Abstract: For the ill-posed problem of super resolution reconstruction, a new adaptive algorithm for image sequence was proposed. The new algorithm was based on the framework of L1-norm. In the new algorithm, the pyramidal algorithm coupled with Lucas-Kanade algorithm was used for images registration to obtain the sub-pixel motion estimation. Displacement operator was introduced to achieve the regular term based on the orthogonal gradient operator and the regularization parameter was determined adaptively. Finally, the steepest descent method was used to solve the minimum of the constraint equation. The simulation experiments and the true sequence experiments show that the method proposed has advantages over spline interpolation, Tikhonov reconstruction and bilateral total variation reconstruction. On the one hand it can provide better reconstructing results, on the other hand the reconstruction time is reduced at the same time since the regularization item is simple.