Repair and error compensation method for depth image based on optimization estimation

Li Liangfu; Zou Bin; Zhou Guoliang; Wang Chao; He Junfeng

doi:10.5768/JAO201839.0101008

Journal of Applied Optics > 2018 > 39(1): 45-50. > DOI: 10.5768/JAO201839.0101008

Li Liangfu, Zou Bin, Zhou Guoliang, Wang Chao, He Junfeng. Repair and error compensation method for depth image based on optimization estimation[J]. Journal of Applied Optics, 2018, 39(1): 45-50. DOI: 10.5768/JAO201839.0101008

Citation:

PDF (3645 KB)

Repair and error compensation method for depth image based on optimization estimation

Xi'an Institute of Applied Optics, Xi'an 710065, China

More Information

Received Date: May 10, 2017
Revised Date: October 28, 2017

Graphical Abstract

Abstract

Abstract

The depth value of Kinect sensor changes randomly when the depth image is obtained.In order to solve this problem, this paper presents an image repairing method combined with Kalman filtering and multiple frames averaging based on the idea of optimal estimation.Firstly, Kalman filter is used for repairing multiple depth images.The depth value tends to be stable with time recursion in the process of information capture by Kinect sensor.Secondly, multiple frames averaging method is used to determine the final depth image, in order to solve the problem of inaccurate depth value due to the error of Kinect sensor.The experimental results show that, the root mean square error of the algorithm is 38.102 5, the average gradient is 0.471 3, the information entropy is 6.191 8, the edge of depth image of this algorithm is more clearly when compared with the single image restoration.
- depth image,
- error repair,
- Kalman filter,
- optimal estimation

FullText(HTML)

References (17)

References

[1]	ZHU Hao, LIU Yebin, FAN Jingtao, et al.Video-based outdoor Human reconstruction[J].IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(4):760-770. doi: 10.1109/TCSVT.2016.2596118
[2]	EISERT P, LIU Yebin, LEE K M, et al.Introduction to the special section on augmented video[J].IEEE Signal Processing Letters, 2017, 27(4):713-715. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=10.1177/135703403773684612
[3]	BONDI E, PALA P, BERRETTI S, et al.Reconstructing high-resolution face models from Kinect depth sequences[J].IEEE Transactions on Information Forensics and Security, 2016, 11(12):2843-2853. doi: 10.1109/TIFS.2016.2601059
[4]	张翼飞, 李良福, 王娇颖, 等.基于超分辨率重建的图像增强算法研究[J].应用光学, 2011, 32 (2):250-255. doi: 10.3969/j.issn.1002-2082.2011.02.013 ZHANG Yifei, LI Liangfu, WANG Jiaoying, et al.An image enhancement algorithm base on super resolution reconstruction[J].Journal of Applied Optics, 2011, 32(2):250-255. doi: 10.3969/j.issn.1002-2082.2011.02.013
[5]	郭连朋, 陈向宁, 刘彬, 等.基于Kinect传感器多深度图像融合的物体三维重建[J].应用光学, 2014, 35 (5):811-816. http://yygx.xml-journal.net/article/id/10488 GUO Lianpeng, CHEN Xiangning, LIU Bin, et al.3D object reconstruction based on fusion of depth images by Kinect sensor[J].Journal of Applied Optics, 2014, 35(5):811-816. http://yygx.xml-journal.net/article/id/10488
[6]	LIU Zhiguang, ZHOU Liuyang, LEUN H, et al.Kinect posture reconstruction based on a local mixture of Gaussian process models[J].IEEE Transactions on Visualization and Computer Graphics, 2016, 22(11):2437-2450. doi: 10.1109/TVCG.2015.2510000
[7]	LI M, MOURIKIS A.High-precision, consistent EKF-based visual-inertial odometry[J].International Journal of Robotics Research, 2013, 32:690-711. doi: 10.1177/0278364913481251
[8]	PIERRE F, AUJOL J F, BUGEAU A, et al.Variational contrast enhancement of gray-scale and RGB images[J].Journal of Mathematical Imaging and Vision, 2017, 57(1):99-116. doi: 10.1007/s10851-016-0670-8
[9]	WANG Anzhi, WANG Minghui.RGB-D salient object detection via minimum barrier distance transform and saliency fusion[J].IEEE Signal Processing Letters, 2017, 24(5):663-667. doi: 10.1109/LSP.2017.2688136
[10]	YANG Qingxiong, YANG Ruigang, DAVIS J, et al.Spatial-depth super resolution for range images[C].New York: IEEE Conference on Computer Vision and Pattern Recognition, 2007: 1-8.
[11]	GWAK S H, PARK E B, Kim H J.Ultrasound image enhancement using Markov random field model[J].ASTL, 2013, 23:96-99.
[12]	TOROK M M, GOLPARVAR-FARD M, KOCHERSBERGER K B.Image-based automated 3D crack detection for post-disaster building assessment[J].Journal of Computing in Civil Engineering, 2014, 28:1-13. doi: 10.1061/(ASCE)CP.1943-5487.0000359
[13]	KLINGBEIL L, NIEUWENHUISEN M, SCHNEIDER J, et al.Towards autonomous navigation of an UAV-based mobile mapping system[C]//4th International Conference on Machine Control & Guidance, 2014.Braunschweig, Germany: ResearchGate, 2014: 1-12.
[14]	LATTANZI D, MILLER G R.3D scene reconstruction for Robotic bridge inspection[J].Journal of Infrastructure Systems, 2014, 20(3):1-12. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=a2132dad50831e2aa13231efde150461
[15]	ENGEL J, SCHOPS T, Cremers D.LSD-SLAM:large-scale direct monocular SLAM[J].European Conference on Computer Vision, 2014, 8690:834-849. http://d.old.wanfangdata.com.cn/NSTLHY/NSTL_HYCC0214745798/
[16]	LA H M, LIM R S, BASILY B B, et al.Mechatronic systems design for an autonomous Robotic system for high-efficiency bridge deck inspection and evaluation[J].IEEE Transactions on Mechatronics, 2013, 18(6):1655-1664. doi: 10.1109/TMECH.2013.2279751
[17]	HENRY P, KRAININ M, Herbst E, et al.RGB-D mapping:using depth cameras for dense 3D modeling of indoor environments[J].Springer Tracts in Advanced Robotics, 2014, 79:477-491.

[1]	WU Fengbo, LIU Yao, ZHU Daixian, WANG Mingbo. Particle filter algorithm based on whale swarm optimization[J]. Journal of Applied Optics, 2021, 42(5): 859-866. DOI: 10.5768/JAO202142.0502006
[2]	LI Yuqi, ZHAO Haitao. Depth estimation based on adaptive fusion of infrared and visible light images progressively[J]. Journal of Applied Optics, 2020, 41(1): 24-32. DOI: 10.5768/JAO202041.0101004
[3]	QU Yepin, ZHANG Chaoran, LYU Yuhai. Maximum likelihood pose estimation using machine vision[J]. Journal of Applied Optics, 2019, 40(2): 253-258. DOI: 10.5768/JAO201940.0202002
[4]	LIU Xiaomin, MA Zhibang, WANG Qiancheng, DU Mengzhu, ZHU Yunfei, MA Fengying, LIANG Erjun. Compression light field reconstruction and depth estimation[J]. Journal of Applied Optics, 2019, 40(2): 179-185. DOI: 10.5768/JAO201940.0201001
[5]	Nie Peiwen, Liu Enhai, Wang Wanping, Tian Hong. Weighted on-orbit calibration method of principal point and focal length for star sensor[J]. Journal of Applied Optics, 2018, 39(6): 827-831. DOI: 10.5768/JAO201839.0601009
[6]	LEI Fei-lin, LIU Jian-wei, WU Yu-jing, ZHAO Chuang-she, SANG Wei, WANG Xin-wei. Application of Kalman filter based fiber opticgyroscope in airborne electro-optical system[J]. Journal of Applied Optics, 2013, 34(1): 188-192.
[7]	ZHANG Yan, QIU Tian-shuang, REN Fu-quan. Robust time delay estimation of distributed optical fiber sensor system[J]. Journal of Applied Optics, 2012, 33(4): 815-820.
[8]	TIAN Jun-wei, SHANG Ya-ceng, WANG Hong-xi, CHENG Gang. Boundary tracking algorithm based on direction estimation[J]. Journal of Applied Optics, 2011, 32(3): 441-445.
[9]	LI Hong-guang, YU Yun-qi, SONG Ya-min. Application of optimal control for stabilization loop of vehicle inertial platform[J]. Journal of Applied Optics, 2007, 28(3): 251-256.
[10]	HONG Han-yu, YU Jiu-yang, CHEN Yi-chao, YI Xin-jian. Optimization restoration algorithm for infrared object turbulencedegraded image[J]. Journal of Applied Optics, 2006, 27(6): 510-515.

Cited By

Cited by

Periodical cited type(10)

1.	公希萌，赵亮凯. 基于三维激光视觉技术的平面设计图像增强和优化研究. 激光杂志. 2023(04): 158-163 .
2.	段锦. 基于Retinex的园林景观图像增强. 微型电脑应用. 2022(11): 45-47+52 .
3.	王康，刘婷，陈子文，李雅薇，郭显久. 计算机图像处理技术在水下图像中的应用. 数字技术与应用. 2021(03): 93-95+161 .
4.	于琳琳. 基于激光全息技术的漆画无损修复研究. 激光杂志. 2021(12): 207-211 .
5.	李毕祥，乐敏. 激光全息数字图像补偿资源云存储平台设计. 激光杂志. 2020(03): 117-121 .
6.	杨森林，孙静，闫曌，李喜龙. 基于深度卷积神经网络的图像帧间补偿研究. 计算机仿真. 2020(01): 452-455 .
7.	李熙莹，朱肯钢. 结合天空分割和局部透射率优化交通图像去雾算法. 计算机与现代化. 2019(05): 51-58 .
8.	冯晶明，苗文娟，畅青. 基于像素值梯度变化的深度图修复算法. 现代计算机. 2019(25): 51-54 .
9.	姚晓峰，须文波，武利秀. 基于深度神经网络的激光图像修复. 激光杂志. 2019(11): 76-79 .
10.	刘振宇，关彤. 基于RGB-D图像的头部姿态检测. 计算机科学. 2019(S2): 334-340 .

Other cited types(8)

Get Citation

PDF

XML

Article views (493) PDF downloads (22) Cited by(18)

Turn off MathJax

Article Contents

Abstract

References

Repair and error compensation method for depth image based on optimization estimation