Abstract: The depth value of Kinect sensor changes randomly when the depth image is obtained.In order to solve this problem, this paper presents an image repairing method combined with Kalman filtering and multiple frames averaging based on the idea of optimal estimation.Firstly, Kalman filter is used for repairing multiple depth images.The depth value tends to be stable with time recursion in the process of information capture by Kinect sensor.Secondly, multiple frames averaging method is used to determine the final depth image, in order to solve the problem of inaccurate depth value due to the error of Kinect sensor.The experimental results show that, the root mean square error of the algorithm is 38.102 5, the average gradient is 0.471 3, the information entropy is 6.191 8, the edge of depth image of this algorithm is more clearly when compared with the single image restoration.