Abstract: Frame difference is one of the most widely used methods in moving object detection, for its advantages such as simple algorithm, less data to be processed, easy to be implemented and so on; however, frame difference method usually detects a moving object by using a specific threshold value measured under an invariable lightness condition, which limits its application in variable illumination situations. According to the above problem, the illumination-lightness-threshold table was created based on the lightness in the hue, saturation, lightness (HSL) color space to dynamically adjust the threshold value for different illumination conditions. After simulation, a detection system was implemented by field-programmable gate array (FPGA), and an experimental setup was built to verify the system. Experimental results show that, the recognition rate reaches 93% and 95% for a moving target with 3.0 m/s and 0.5 m/s velocity under 90 W/m² illumination by the adaptive method, which is similar to the method with a large fixed threshold; under 0.59 W/m² illumination, the recognition rate reaches 84% and 92% respectively for the two moving speeds above, which is similar to the fixed threshold method with a small threshold value.