Abstract: There are many quality problems in depth image，for example, images may be combined with blurring, empty holes and noise. In order to solve these problems, from the view point of software,an algorithm guided by color images based on structural characteristics was studied without changing any physical structure of the sensor imaging system,which could realize depth image enhancement and empty holes repairing,as well as improve the quality of the depth image. Through the extraction for the structure of the color image and depth image, the common global features were obtained. After joint bilateral filtering for the structural features,the depth image was enhanced based on Markov random filed (MRF), and the depth enhanced image was finally obtained with low cost. Experimental results show that the algorithm has a better effect in keeping the detail, smoothness and integrity of image edge; moreover, the rootmeansquare error (RMSE) of this algorithm is smaller than other algorithms,which is 0.50693 and 1.16930 for image Teddy and Art,respectively.