Nighttime environment perception of driverless vehicles based on improved YOLOv3 network

PEI Jiaxin; SUN Shaoyuan; WANG Yulan; LI Dawei; HUANG Rong

doi:10.5768/JAO201940.0301004

Journal of Applied Optics > 2019 > 40(3): 380-386. > DOI: 10.5768/JAO201940.0301004

PEI Jiaxin, SUN Shaoyuan, WANG Yulan, LI Dawei, HUANG Rong. Nighttime environment perception of driverless vehicles based on improved YOLOv3 network[J]. Journal of Applied Optics, 2019, 40(3): 380-386. DOI: 10.5768/JAO201940.0301004

Citation:

PDF (1157 KB)

Nighttime environment perception of driverless vehicles based on improved YOLOv3 network

PEI Jiaxin^{1, 2,},
SUN Shaoyuan^{1, 2},
WANG Yulan^{1, 2},
LI Dawei^{1, 2},
HUANG Rong^{1, 2}

1.
College of Information Science and Technology, Donghua University, Shanghai 201620, China
2.
Engineering Research Center of Digitized Textile & Fashion Technology(Ministry of Education Donghua University, Shanghai 201620, China

More Information

Received Date: November 25, 2018
Revised Date: January 09, 2019

Graphical Abstract

Abstract

Abstract

Environmental perception is a key task of driverless vehicles at night. An improved YOLOv3 network was proposed to realize the detection of pedestrians and vehicles in infrared images captured by driverless vehicles at night. The problem of estimation of the moving direction of surrounding vehicles is transformed into the problem of estimation of the angle of the surrounding vehicle position. What's more, the network is fused with the depth estimation information to estimate the distance and speed of the surrounding vehicles. Therefore the driverless vehicles can obtain the driving intention of the surrounding vehicles at night. The network has the end-to-end advantage, in which an image is as the input, and the positions of the bounding boxes, the classes and the angle estimation results of the detecting targets are returned directly at the output layer. Moreover, the depth estimation information is combined with the above information to obtain the distance and speed of the surrounding vehicle. The experimental results show that the speed of target detection in the infrared images captured by driverless vehicle is 0.04 s/frame. The effect of angle and speed prediction is good, and the accuracy and real-time performance meet the requirements of practical application.
- infrared image,
- target detection,
- YOLOv3 network,
- angle prediction,
- depth estimation

FullText(HTML)

References (16)

References

[1]	KIESER D. Driverless vehicles and their impact on your portfolio[J]. Equity, 2018, 32(1): 4.
[2]	陈琪.面向无人驾驶的环境感知技术研究[J].科技经济导刊, 2018, 26(26):77. http://www.cnki.com.cn/Article/CJFDTotal-JJKJ201826065.htm CHEN Qi. Research on environment-aware technology for unmanned driving[J]. Science and Technology Economic Guide, 2018, 26 (26): 77. http://www.cnki.com.cn/Article/CJFDTotal-JJKJ201826065.htm
[3]	邹芳喻, 孙韶媛, 席林, 等.具有深度视觉感的车载红外图像彩色化方法[J].激光与光电子学进展, 2013, 50(1):011101. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jgygdzxjz201301013 ZOU Fangyu, SUN Shaoyuan, XI Lin, et al. A method of colorization of vehicle infrared image with deep visual sense[J]. Laser & Optoelectronics Progress, 2013, 50(1):011101. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jgygdzxjz201301013
[4]	高凯珺, 孙韶媛, 姚广顺, 等.基于深度学习的无人车夜视图像语义分割[J].应用光学, 2017, 38(03):421-428. doi: 10.5768/JAO201738.0302007 GAO Kaijun, SUN Shaoyuan, YAO Guangshun, et al. Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J]. Journal of Applied Optics, 2017, 38(03):421-428. doi: 10.5768/JAO201738.0302007
[5]	ANDREOPOULOS A, TSOTSOS J. 50 years of object recognition: directions forward[J]. Computer Vision and Image Understanding, 2013, 117(8):827-891 doi: 10.1016/j.cviu.2013.04.005
[6]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Advances in Newral Information Processing Systems.[s.n.]: [S.l.], 2015: 91-99.
[7]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]. USA: IEEE, 2016: 779-788.
[8]	REDMON J, FARHADI A. Yolov3: an incremental improvement[J/OL]. USA:University of Washington, 2018[2019-05-15]. http://arxiv.org/abs/1804.02767.
[9]	许路, 赵海涛, 孙韶媛.基于深层卷积神经网络的单目红外图像深度估计[J].光学学报, 2016, 36(7):0715002. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201607025 XU Lu, ZHAO Haitao, SUN Shaoyuan.Deep estimation of monocular infrared image based on deep convolutional neural network[J]. Acta Optica Sinica, 2016, 36(7). http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gxxb201607025
[10]	AGRAWAL P, GIRSHICK R, MALIK J. Analyzing the performance of multilayer neural networks for object recognition: ECCV 2014: 13th European Canference on Computer Vision, Zurich, September 6-12, 2014[C]. Benlin: Springer, C2014.
[11]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]. USA: IEEE, 2016: 394.
[12]	LIN T Y, DOLLÁR P, GIRSHICK R B, et al. Feature pyramid networks for object detection[C]//IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas: IEEE, 2016: 936-944.
[13]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]. USA: IEEE, 2015: 3431-3440.
[14]	姚广顺, 孙韶媛, 方建安, 等.基于红外与雷达的夜间无人车场景深度估计[J].激光与光电子学进展, 2017, 54(12):164-170. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jgygdzxjz201712019 YAO Guangshun, SUN Yiyuan, FANG Jian'an at al.Infrared and radar based depth estimation of night unmanned vehicle scene[J]. Laser & Optoelectronics Progress, 2017, 54(12):164-170. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=jgygdzxjz201712019
[15]	JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding[C]//Proceedings of the 22nd ACM international conference on Multimedia.[S.l.]: arXiv, 2014: 675-678.
[16]	Dalal N, Triggs B. Histograms of oriented gradients for human detection[J]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, 1: 886-893. http://cn.bing.com/academic/profile?id=0796294dc798795c12f84f5b2edabaef&encoded=0&v=paper_preview&mkt=zh-cn

[1]	ZHU Simin, ZHAO Haitao. Depth estimation of monocular infrared images based on attention mechanism and graph convolutional neural network[J]. Journal of Applied Optics, 2021, 42(1): 49-56. DOI: 10.5768/JAO202142.0102001
[2]	HE Yelin, ZHANG Jun. Fresnel lens design for human posture estimation sensing[J]. Journal of Applied Optics, 2020, 41(3): 597-602. DOI: 10.5768/JAO202041.0305002
[3]	CHEN Yuru, ZHAO Haitao. Depth estimation based on adaptive pixel-level attention model[J]. Journal of Applied Optics, 2020, 41(3): 490-499. DOI: 10.5768/JAO202041.0302002
[4]	LI Yuqi, ZHAO Haitao. Depth estimation based on adaptive fusion of infrared and visible light images progressively[J]. Journal of Applied Optics, 2020, 41(1): 24-32. DOI: 10.5768/JAO202041.0101004
[5]	QU Yepin, ZHANG Chaoran, LYU Yuhai. Maximum likelihood pose estimation using machine vision[J]. Journal of Applied Optics, 2019, 40(2): 253-258. DOI: 10.5768/JAO201940.0202002
[6]	LIU Xiaomin, MA Zhibang, WANG Qiancheng, DU Mengzhu, ZHU Yunfei, MA Fengying, LIANG Erjun. Compression light field reconstruction and depth estimation[J]. Journal of Applied Optics, 2019, 40(2): 179-185. DOI: 10.5768/JAO201940.0201001
[7]	Li Liangfu, Zou Bin, Zhou Guoliang, Wang Chao, He Junfeng. Repair and error compensation method for depth image based on optimization estimation[J]. Journal of Applied Optics, 2018, 39(1): 45-50. DOI: 10.5768/JAO201839.0101008
[8]	Chen Jian, Wang Lingyang, Chen Dequan, Zhang Maolin. Design of microscope system based on depth from focus[J]. Journal of Applied Optics, 2017, 38(6): 979-984. DOI: 10.5768/JAO201738.0605003
[9]	TIAN Jun-wei, SHANG Ya-ceng, WANG Hong-xi, CHENG Gang. Boundary tracking algorithm based on direction estimation[J]. Journal of Applied Optics, 2011, 32(3): 441-445.
[10]	CHEN Guan-nan, YANG Kun-tao, XIE Zhi-ming, TENG Zhong-jian, CHEN Rong. Edge recognition method for image segmentation based on depth-first traversal[J]. Journal of Applied Optics, 2008, 29(1): 14-17.

Cited By

Cited by

Periodical cited type(12)

1.	周爱国，赵吉林，安山，符长虹. 注重明度感知的通用渐进式无监督图像增强方法. 应用光学. 2024(05): 937-945 . 本站查看
2.	何忧，何晓程，董星. 基于多层激光雷达的3D虚拟环境感知系统. 激光杂志. 2023(01): 199-204 .
3.	张玉亮，赵智龙，付炜平，刘洪吉，熊永平，尹子会. 融合边缘语义信息的单目深度估计. 科学技术与工程. 2022(07): 2761-2769 .
4.	郭贵松，林彬，杨夏，张小虎. 基于斑马鱼图像特征的鱼群检测算法. 应用光学. 2022(02): 257-268 . 本站查看
5.	陈耀祖，谷玉海，成霞，徐小力. 基于优化YOLOv4算法的行驶车辆要素检测方法. 应用光学. 2022(02): 248-256 . 本站查看
6.	刘朔，谷玉海，饶文军，王菊远. 基于优化YOLOv3算法的违章车辆检测方法. 重庆理工大学学报(自然科学). 2021(04): 135-141 .
7.	蔡伟，徐佩伟，杨志勇，蒋昕昊，姜波. 复杂背景下红外图像弱小目标检测. 应用光学. 2021(04): 643-650 . 本站查看
8.	洪波，刘雪芹，秦志亮，马本俊，王飞，刘映锋. 水面运动目标跟踪监控系统的设计与实现. 海洋技术学报. 2021(04): 62-73 .
9.	高晓娟，梅秀庄，白福忠，李萍. 航拍巡线低照度彩色图像增强与电力小部件检测. 内蒙古工业大学学报(自然科学版). 2021(06): 461-467 .
10.	陈裕如，赵海涛. 基于自适应像素级注意力模型的场景深度估计. 应用光学. 2020(03): 490-499 . 本站查看
11.	刘洋，姜涛，段学鹏. 基于YOLOv3的复杂天气条件下人车识别方法的研究. 长春理工大学学报(自然科学版). 2020(06): 57-65 .
12.	侯军占，张卫国，庞澜，杨光，张夏疆，徐晓枫，黄维东. 地面小型无人侦察平台发展及关键技术探讨. 应用光学. 2019(06): 958-964 . 本站查看

Other cited types(10)

Get Citation

PDF

XML

Article views (1416) PDF downloads (64) Cited by(22)

Turn off MathJax

Article Contents

Abstract

References

Nighttime environment perception of driverless vehicles based on improved YOLOv3 network