Abstract: Based on modular design concept, a real-time multi-target infrared image processing system with two TMS320DM642 chip as its core components is proposed. Combing with the field programmable gate array, the real-time acquiring, processing and outputting of the image data stream from the infrared focal plane detector were accomplished. In order to increase the data processing speed, the system adopted a two-stage cascaded model and employed parallel pipelining processing and multi-target image concurrent processing, which improved the real-time performance of image processing. Experiment indicates that the system needs 15ms to process each image with a resolution of 320240 to enable multi-target tracking in real-time.