Abstract: The calibration of both camera and projector becomes the key element to decide the accuracy of 3D fringe projection measurement, and accuracy of parameter calibration directly affects accuracy of the measurement. By analyzing the imaging degradation model of mark points, a sub-pixel edge detection method was proposed to obtain the high-precision edges based on the combination of Gaussian curve fitting and edge partial area effect, and thus to improve the accuracy of mark point detection. A quick sort matching method of mark points was presented based on the image rectification technic by way of perspective transformation, and the high-precision camera calibration was finished rapidly. Furthermore, by analyzing the phase error in projector calibration, a linear interpolation method was proposed based on radial basis function in order to improve the phase accuracy of circle center of mark points. Then the experimental results show that the edge residual error of mark points is 0.087 1. Compared to the Gaussian curve fitting method, its accuracy is improved by 41%, the average re-projection errors of camera calibration is 0.052 4 pixels, and the re-projection error of projector is 0.120 3 pixels. Compared to the bi-linear interpolation method, its calibration accuracy is increased by 23.9%.