Abstract: To solve the drogue tracking problem during the capture phase of autonomous aerial refueling, a trackinglearningdetection (TLD) algorithm was proposed. The algorithm decomposes the drogue tracking task into 3 subtasks: tracking, learning and detection. The tracking component is based on the LucasKanade (LK) method extended with failure detection and chooses tracking points with good performance to track the refueling drogue; and the detector constructs cascaded classifier to sort the picture patches scanned by scanningwindow and returns patches containing object, which integrates with tracking component result and gives final tracking result; and then the PN constraints are introduced into the learning component to correct wrong samples and then learn and update the detector. A scene simulation for aerial refueling was developed based on Creator/Vega Prime. The experimental results of the test on the video of scene simulation show that the success rate of tracking drogue is 95.5%, the average time consumption is about 31.4 ms, and the algorithm can meet well requirements of drogue tracking, such as robustness, precision and real time performance.