Research on three-dimensional face measurement and segmentation system
-
摘要: 三维人脸的测量与分割有着广泛的应用需求，是目前重要的研究方向，但三维人脸数据的庞大、无序等问题制约了其快速发展。首先开发了基于结构光方法的三维人脸测量系统，获得以点云形式存储的高精度三维人脸数据。其次，采用保角变换对三维人脸数据进行了预处理，并采用二维卷积神经网络分割结合三维反映射的思路，实现了三维人脸的分割，解决了三维数据的无序性以及旋转性，降低了三维数据分割的时间消耗。实验结果表明，本文提出的三维人脸测量系统精度可达0.5 mm，三维分割的平均交并比可达0.78，二维分割结合三维反映射的整体效率明显高于直接进行三维分割的效率。Abstract: The three-dimensional face measurement and segmentation have extensive application requirements and are the important research direction at present. However, the rapid development is constrained as the three-dimensional face data is huge and unordered. Firstly, the three-dimensional face measurement system based on the structured light method was developed, and the three-dimensional face data with high precision saved in the form of point cloud was obtained. Secondly, after the conformal transformation was adopted to preprocess the three-dimensional face data and the two-dimensional convolutional neural network segmentation combined with the three-dimensional inverse mapping was adopted, the three-dimensional face segmentation was realized, the disorder and rotation of three-dimensional data were solved, and the time consumption of three-dimensional data segmentation was reduced. The experimental results show that the proposed precision of three-dimensional face measurement system can reach to 0.5 mm, and the average intersection-over-union (IoU) of three-dimensional segmentation can reach to 0.78. The overall efficiency of two-dimensional segmentation combined with the three-dimensional inverse mapping is obviously higher than that of the three-dimensional segmentation.
-
-
![]()
图 8 三维人脸分割结果（不同视角）
Figure 8. 3D face segmentation results (from different perspectives)
表 1 本文方法与PointNet分割结果对比
Table 1 Comparison of proposed method and PointNet segmentation results
参数 本文方法 PointNet 训练耗时/min 33.1 120 分割耗时/s < 1 220 准确率（ACC） 0.93 0.90 交并比（IoU） 0.78 0.68 
下载: 导出CSV
-
[1] ZHOU S, XIAO S. 3D face recognition: a survey[J]. Human-centric Computing and Information Sciences,2018,8(1):35-62. doi: 10.1186/s13673-018-0157-2
[2] PATIL H, KOTHARI A, BHURCHANDI K. 3-D face recognition: features, databases, algorithms and challenges[J]. Artificial Intelligence Review,2015,44(3):393-441. doi: 10.1007/s10462-015-9431-0
[3] HUSSMANN, HERMANSKI A. Real-time image processing of TOF range images using a single shot image capture algorithm[C]//2012 IEEE I2MTC - International Instrumentation and Measurement Technology Conference. Graz, Austria: IEEE, 2012: 1551-1555.
[4] STEFAN M, DAVID D, DIRK H, et al. Three-dimensional mapping with time-of-flight cameras[J]. Journal of Field Robotics,2009,26(11/12):934-965. doi: 10.1002/rob.20321
[5] 曾少青. 基于飞行时间法原理的三维成像系统设计[D]. 湖南: 湘潭大学, 2017. ZENG Shaoqing. Design of 3D camera system based on time-of-fight principle[D]. Hu’nan: Xiangtan University, 2017.
[6] FUCHS S, HIRZINGER G. Extrinsic and depth calibration of ToF-cameras[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE: 1-6.
[7] GUO Y, WAN J, LU M, et al. A parts-based method for articulated target recognition in laser radar data[J]. Optik-International Journal for Light and Electron Optics,2013,124(17):2727-2733.
[8] GUO Y, SOHEL F A, BENNAMOUN M, et al. RoPS: a local feature descriptor for 3D rigid objects based on rotational projection statistics[C]//Proceedings of 2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA). Sharjah: IEEE, 2013: 1-6.
[9] QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3d classification and segmentation[C]//Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2016: 77-85.
[10] QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[C]//Conference and Workshop on Neural Information Processing Systems. Long Beach, NIPS, 2017: 1-14.
[11] SU H, JMAPANI V, SUN D, et al. SPLATNet: sparse lattice networks for point cloud processing[C]//Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake: IEEE, 2018: 2530-2539.
[12] ZHANG Z. Flexible camera calibration by viewing a plane from unknown orientations[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Honolulu: IEEE, 1999: 666-673.
[13] SONG Z, HUANG P S, et al. Novel method for structured light system calibration[J]. Optical Engineering,2006,45(8):083601. doi: 10.1117/1.2336196
[14] 朱勇建, 黄振, 马俊飞, 等. 光栅投影三维测量系统标定技术研究[J]. 应用光学,2020,41(5):68-76. ZHU Yongjian, HUANG ZHEN, MA Junfei, et al. Study on calibration method of grating projection 3D measuring system[J]. Journal of Applied Optics,2020,41(5):68-76.
[15] 达飞鹏, 盖绍彦. 光栅投影三维精密测量[M]. 北京: 科学出版社, 2011: 45-62. DA Feipeng, GAI Shaoyan. Grating projection 3D precision measurement[M]. Beijing: Science Press, 2011: 45-62.
[16] 贺文俊, 王加科, 付跃刚, 等. 基于双目立体视觉的高压绝缘子在线检测系统[J]. 应用光学,2018,39(4):528-535. HE Wenjun, WANG Jiake, FU Yuegang, et al. On-line measurement system of high voltage insulator based on binocular stereo vision[J]. Journal of Applied Optics,2018,39(4):528-535.
[17] JUN Y. A piecewise hole filling algorithm in reverse engineering[J]. Computer-Aided Design,2005,37(2):263-270. doi: 10.1016/j.cad.2004.06.012
[18] GU X, WANG Y, CHAN T F, et al. Genus zero surface conformal mapping and its application to brain surface mapping[J]. IEEE Transactions on Medical Imaging,2004,23(8):949-958. doi: 10.1109/TMI.2004.831226
[19] REDDY J N. An introduction to nonlinear finite element analysis[M]. Oxford: Oxford University Press, 2004: 145-152.
[20] KRA I, FARKAS H M. Riemann surfaces[M]. New York: Springer-Verlag, 1992: 221-239.
[21] GU X, YAU S T. Global conformal surface parameterization[C]//Proceedings of Institute of Electrical and Electronics Engineers Visualization. Austin: IEEE, 2004: 267-274.
[22] 尹宝才, 孙艳丰, 王成章, 等. BJUT-3D三维人脸数据库及其处理技术[J]. 计算机研究与发展,2009,46(6):123-132. YIN Baocai, SUN Yanfeng, WANG Chengzhang, et al. BJUT large scale 3D face database and information processing[J]. Journal of Computer Research and Development,2009,46(6):123-132.
[23] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Switzerland: Springer Cham, 2015: 234-241.
[24] LU J, TONG K Y. Visualized insights into the optimization landscape of fully convolutional networks[EB/OL]. [2019-1-20]. https://arxiv.org/abs/1901.08556.
[25] JIANG J F, ZHONG Y Q, ZHANG Q P. Three-dimensional garment surface reconstruction based on ball-pivoting algorithm[J]. Advanced Materials Research,2013(821/822):765-768.
-
期刊类型引用(8)
1. 周会娟，余尚江，陈晋央，陈显，孟晓洁. 一种双面感压式光纤土压力传感器. 兵工学报. 2023(S1): 132-137 . 
百度学术
2. 吕欢祝，钟文博，秦亮，张克非. 聚合物封装的光纤光栅压力传感器的特性研究. 激光杂志. 2020(08): 63-67 . 百度学术
3. 杨洋，赵勇，吕日清，刘兵，郑洪坤，杨洋，王孟军，崔盟军，杨华丽. 多参量一体化光纤传感器及标校系统的研制与开发. 红外与激光工程. 2019(10): 185-191 . 百度学术
4. 吴国军，何少灵，桑卫兵. 温度精度补偿的光纤光栅土压力传感器. 机电工程技术. 2018(06): 47-49 . 百度学术
5. 郭红英，王召巴. 基于光纤光栅的高压固体压力传感器研究. 分析化学. 2017(07): 980-986 . 百度学术
6. 袁斌. 光纤Bragg光栅传感器的通信系统设计与实现. 激光杂志. 2017(12): 67-70 . 百度学术
7. 李凯，赵振刚，李英娜，蔡陈，彭庆军，李川. FBG可变灵敏度压力传感器设计. 传感器与微系统. 2016(06): 69-71 . 百度学术
8. 孙搏，隋青美，王静，曹帅帅，王宁，李海燕，刘斌. 微型布拉格光栅土压力传感器的设计与试验. 仪表技术与传感器. 2016(10): 20-23+27 . 百度学术
其他类型引用(2)
计量
- 文章访问数: 568
- HTML全文浏览量: 291
- PDF下载量: 31
- 被引次数: 10
陕公网安备 61011302001501号 