Abstract: Calibration of the system parameters is the basis of three-dimensional measurement system of the structured light, and accuracy of parameter calibration directly affects accuracy of the measurement, in which the projector still has the problems of complex calibration process and low precision. In order to solve this problem, a calibration method is put forward, which is done by projecting circles pattern to a flat with circular signalized points. Based on projective transformation theory, projective corresponding relation is established between projector image coordinates and camera image coordinates through error compensation method, and accurate projector image coordinates of calibration points is obtained. Taking two groups of image coordinates and world coordinates as initial value, nonlinear algorithm is used to optimize the whole system parameters. Experimental results show that the maximum error of the system is less than 0.05 mm and the error root mean square is less than 0.03 mm. The results show that the calibration process is simple and can improve the calibration accuracy and has wide applicability.