Abstract: Aiming at the small object detection in infrared searching system, a novel algorithm of adaptive spatial filtering is proposed on the basis of the model analysis of infrared images. The background residual errors are calculated in the algorithm while the background is estimated, and the filtering parameters are adjusted to adapt to the fluctuation of the background according to the values of the residual errors. The multi-scale morphology operator is used in the algorithm. This method is suitable to the infrared small object detection in clutter background, and can reserve the information of objects totally. The method is run on the platform of FPGA, and used in an infrared searching system. The experiment results presented in this paper show that the signal-to-noise ratio (SNR) of the processed images can reach to 3 even if the scene is complex and SNR is low. It has a good result on real-time processing and is easy to be realized in engineering.