Abstract: According to the characteristics of the video measuring machine, the optimization of straightness error measuring was introduced. A method to evaluate straightness error with intervaldistance improved algorithm which conforms to minimum zone condition was proposed and the improved edge detection template was designed. Using the method of image measurement experiment of smooth limit gauge, comparing with the traditional edge detection method and gradient search method, a system for measuring straightness error was tested. Experiment results show that the improved edge detection algorithm can improve the measurement speed, the calculation convolution time is reduced by half comparing with the traditional testing templates.The intervaldistance improved algorithm was compared with the gradient searching algorithm, and results show that the relative error of straightness error for the same 8 groups of data is less than 2%, the average computation speed is increased by 0.01 s. It is verified that the computation time can be saved by 4.45 s with the optimized method in the experiment for measuring straightness error of different diameter gauges by imaging instrument . Through different ways of evaluation, the best span for measuring straightness error of 0.078 mm~0.104 mm was advized,which can provide a foundation for realtime detection.