Abstract: When the traditional key frame extraction algorithm is applied to the theodolite image sequence, a large number of unstable tracking image frames will be extracted. In order to better retain the stable tracking measurement information of the target, after analyzing the characteristics of the theodolite image sequence, a key frame extraction algorithm for the theodolite image sequence based on the local maximum was constructed. Firstly, the frame difference of the image sequence was calculated by the algorithm. Then, the Hanning window function was used to smooth the frame difference. Finally, based on the smoothed local maximum of frame difference, the key frame was extracted. The experimental results show that the proposed algorithm can better retain the tracking measurement information of the target compared with the traditional frame difference intensity sorting method. The extracted key frames are more uniformly distributed in the entire tracking measurement image sequence, and the scene information contained is more abundant.