Abstract: The methodology of real time image processing of near-infrared rail abrasion detection system has been studied. In consideration of accurate abrasion value,the location of starting point is very important. For this, two feasible location methods have been offered. One of the two methods has been put into practice. And the results are in line with the actual abrasion values. In addition, the key technology of the extraction of reflection-streak center line has been discussed. For more,Roberts edge detection has been applied and the single-pixel center line is extracted. The method got a good effect and their processing speed is fast,which show that it can meet the demand for real time cross-section detection system.