Real-time splicing method of image data based on embedded panoramic video stream

WANG Jiaoying; LI Liangfu; HE Xi; HUANG Xiying; WANG Jie; ZHOU Guoliang; CAO Yuju; ZHANG Xuhui

doi:10.5768/JAO202344.0202008

Journal of Applied Optics > 2023 > 44(2): 354-364. > DOI: 10.5768/JAO202344.0202008

WANG Jiaoying, LI Liangfu, HE Xi, HUANG Xiying, WANG Jie, ZHOU Guoliang, CAO Yuju, ZHANG Xuhui. Real-time splicing method of image data based on embedded panoramic video stream[J]. Journal of Applied Optics, 2023, 44(2): 354-364. DOI: 10.5768/JAO202344.0202008

Citation:

PDF (6991 KB)

Real-time splicing method of image data based on embedded panoramic video stream

1.
Xi'an Institute of Applied Optics, Xi'an 710065, China
2.
School of Guangzhou, Xidian University, Guangzhou 510555, China

More Information

Received Date: May 12, 2022
Revised Date: June 24, 2022
Available Online: December 02, 2022

Graphical Abstract

Abstract

Abstract

Aiming at the influence of airborne panoramic video stream, a panoramic real-time splicing method based on embedded mode was proposed. The speeded up robust features (SURF) points in the image were extracted to generate the feature descriptors. The matching degree was determined by calculating the Euclidean distance between two feature points. After affine transformation, the Poisson transform was used to realize the fusion and smoothing between images. The above processes were executed concurrently on the target equipment, and customized optimization was carried out according to the characteristics of each process to realize the panoramic real-time splicing. After test, the proposed method realizes the seamless effect at the splicing joint, and the splicing speed reaches 30 Hz, which can meet the requirements of real-time display.
- image splicing,
- embedded,
- SURF descriptor,
- Poisson integration

FullText(HTML)

References (20)

References

[1]	BAY H, ESS A, TUYTELAARS T, et al. Speeded-Up Robust Features (SURF)[J]. Computer Vision and Image Understanding,2008,110:346-359. doi: 10.1016/j.cviu.2007.09.014
[2]	REZ P, GANGNET M, BLAKE A. Poisson image editing[J]. ACM Transactions on Graphics,2003,22(3):313-318. doi: 10.1145/882262.882269
[3]	陈伟, 刘宇, 王亚伟, 等. 基于改进的FAST-SURF快速图像拼接算法[J]. 应用光学,2021,42(4):636-642. doi: 10.5768/JAO202142.0402001 CHEN Wei, LIU Yu, WANG Ya’wei, et al. Fast image stitching algorithm based on improved FAST-SURF[J]. Journal of Applied Optics,2021,42(4):636-642. doi: 10.5768/JAO202142.0402001
[4]	傅子秋, 张晓龙, 余成, 等. 多场景下基于快速相机标定的柱面图像拼接方法[J]. 光电工程,2020,47(4):74-86. FU Ziqiu, ZHANG Xiaolong, YU Cheng, et al. Cylindrical image mosaic method based on fast camera calibration in multi-scene[J]. Opto-Electronic Engineering,2020,47(4):74-86.
[5]	陈浩. 基于SURF特征匹配算法的全景图像拼接[D]. 西安: 西安电子科技大学, 2010. CHEN Hao. Panoramic image stitching based on SURF feature matching algorithm [D]. Xi'an: Xi'an University of Electronic Science and Technology, 2010.
[6]	刘鹏飞, 高如新. 基于相位相关法与改进SURF算法的图像拼接方法[J]. 软件导刊,2019,18(11):157-160. LIU Pengfei, GAO Ruxin. Image stitching method based on phase correlation and improved SURF[J]. Software Guide,2019,18(11):157-160.
[7]	袁丽英, 刘佳, 王飞越. 基于SURF的图像配准改进算法[J]. 探测与控制学报,2020,42(2):65-70. YUAN Liying, LIU Jia, WANG Feiyue. An improved algorithm of SURF image registration[J]. Journal of Detection & Control,2020,42(2):65-70.
[8]	杜承垚, 袁景凌, 陈旻骋, 等. GPU加速与L-ORB特征提取的全景视频实时拼接[J]. 计算机研究与发展,2017,54(6):1316-1325. doi: 10.7544/issn1000-1239.2017.20170095 DU Chengyao, YUAN Jingling, CHEN Mincheng, et al. Real-time panoramic video stitching based on GPU acceleration using local ORB feature extraction[J]. Journal of Computer Research and Development,2017,54(6):1316-1325. doi: 10.7544/issn1000-1239.2017.20170095
[9]	袁启平, 宋金泽, 吴涛. 基于逆透视投影变换的图像拼接方法[J]. 微计算机信息,2010(21):208-210. doi: 10.3969/j.issn.2095-6835.2010.21.081 YUAN Qiping, SONG Jinze, WU Tao. A method for stitching images based on inverse perspective mapping (IPM)[J]. Microcomputer Information,2010(21):208-210. doi: 10.3969/j.issn.2095-6835.2010.21.081
[10]	ADEL E, ELMOGY M, ELBAKRY H. Image stitching based on feature extraction techniques: a survey[J]. International Journal of Computer Applications,2014,99(6):1-8. doi: 10.5120/17374-7818
[11]	吴丽萍, 胡郁. 柱面全景图图像拼接中图像平滑的虚拟现实技术[J]. 科学技术与工程,2017,17(31):271-276. doi: 10.3969/j.issn.1671-1815.2017.31.044 WU Liping, HU Yu. Virtual reality technology of image smoothing in cylindrical panoramic image mosaic[J]. Science Technology and Engineering,2017,17(31):271-276. doi: 10.3969/j.issn.1671-1815.2017.31.044
[12]	ZOU T, DENG F. Research on real-time video stitching technology[C]//2017 4th International Conference on Machinery, Materials and Computer(MACMC 2017). [S. l. ]: Atlantis Press, 2018: 741-744.
[13]	卢嘉铭, 朱哲. 基于GPU加速的实时4K全景视频拼接[J]. 计算机科学,2017,44(8):18-21. LU Jiaming, ZHU Zhe. Real-time 4k panoramic video stitching based on GPU acceleration[J]. Computer Science,2017,44(8):18-21.
[14]	彭凤婷. 全景视频图像融合与拼接算法研究[D]. 成都: 电子科技大学, 2017. PENG Fengting. Research on fusion and stitching algorithms for panoramic video images [D]. Chengdu: University of Electronic Science and Technology, 2017.
[15]	宫妍. 全景图像拼接关键技术研究[D]. 哈尔滨: 哈尔滨工程大学, 2016. GONG Yan. Key technologies for panoramic image stitching [D]. Harbin: Harbin Engineering University, 2016.
[16]	HERRMANN C, WANG C, BOWEN R S, et al. Robust image stitching with multiple registrations[C]//2018 European Conference on Computer Vision. [S. l.: s. n.], 2018: 53-69.
[17]	NVIDIA. NVIDIA Jetson TX2 System-on-Modul, v1.1, 20e[EB/OL]. [2017-12-12]. https://developer.nvidia.com/embedded/jetson-tx2.
[18]	NVIDIA. CUDA C programming Guide, PG-02829-001_v8.0, 2017.5[EB/OL]. [2017-10-15]. https://docs.nvidia.com/cuda/cuda-c-programming-guide.
[19]	NVIDIA. Pascal architecture whitepaper, V1.2, 2017[EB/OL]. [2017-12-13]. https://images.nvidia.com/content/pdf/tesla/whitepaper/pascal-architecture-whitepaper.pdf.
[20]	张刚, 马震环, 雷涛, 等. 基于嵌入式GPU的运动目标分割算法并行优化[J]. 应用光学,2019,40(6):1067-1076. doi: 10.5768/JAO201940.0602004 ZHANG Gang, MA Zhenhuan, LEI Tao, et al. Embedded GPU-based parallel optimization for moving objects segmentation algorithm[J]. Journal of Applied Optics,2019,40(6):1067-1076. doi: 10.5768/JAO201940.0602004

[1]	CUI Yumin, YIN Liju, SUI Liguo, ZHOU Hui, DENG Yulin. Compression and reconstruction of photon counting integral imaging[J]. Journal of Applied Optics, 2023, 44(2): 295-306. DOI: 10.5768/JAO202344.0202001
[2]	YU Haibin, CHEN Beixi, PAN Zhifeng, REN Weifeng, ZHANG Honggang. Arrangement of microlens and image restoration technology of photon integrated interferometric imaging system[J]. Journal of Applied Optics, 2022, 43(2): 213-220. DOI: 10.5768/JAO202243.0201005
[3]	CHEN Wei, LIU Yu, WANG Yawei, SUN Jing, JI Ting, ZHAO Qinglin. Fast image stitching algorithm based on improved FAST-SURF[J]. Journal of Applied Optics, 2021, 42(4): 636-642. DOI: 10.5768/JAO202142.0402001
[4]	Zhang Lei, Jiao Xiaoxue, Sun Yu, Liu Shaopeng, Xie Yan, Zhao Xing. 3D shape acquisition technology of object with largedepth based on integral imaging[J]. Journal of Applied Optics, 2017, 38(4): 587-591. DOI: 10.5768/JAO201738.0402002
[5]	Zhang Tian-yi, Ji Hang-xin, Hou Yong-hui, Hu Zhong-wen, Wang Lei. Integral field spectroscopy imaging technology[J]. Journal of Applied Optics, 2015, 36(4): 531-536. DOI: 10.5768/JAO201536.0401007
[6]	JI Li-e, YANG Feng-bao, WANG Zhi-she, CHEN Lei. SAR and visible image registration method-based on edge and SURF algorithm[J]. Journal of Applied Optics, 2013, 34(5): 809-814.
[7]	CHEN Hua, HU Chun-hai. Human body limbs detection algorithm based on integral image[J]. Journal of Applied Optics, 2011, 32(5): 904-908.
[8]	WANG Jiao-ying, CHEN Wei-dong, LI Liang-fu. A Seamless image mosaic based on feature invariant description[J]. Journal of Applied Optics, 2011, 32(1): 59-64.
[9]	LI Fu-wei, ZHANG Yun-qiang. Influence of integration time on IRFPA imaging system[J]. Journal of Applied Optics, 2008, 29(5): 727-730.
[10]	ZHOU Wei-dong, FENG Qi-bo, KUANG Cui-fang. Study on Image Description Methods[J]. Journal of Applied Optics, 2005, 26(3): 27-31.

Cited By

Cited by

Periodical cited type(2)

1.	李兴春，樊新龙，张小军，周虹，官春林，顾乃庭. 基于遗传算法的变形反射镜快速设计. 激光与光电子学进展. 2023(01): 348-355 .
2.	周浩，王欣，刘强，贾建军，舒嵘. 渐变厚度主动变形镜的研究. 光学学报. 2021(24): 218-227 .

Other cited types(2)

Get Citation

PDF

XML

Article views (233) PDF downloads (64) Cited by(4)

Turn off MathJax

Article Contents

Abstract

References

Real-time splicing method of image data based on embedded panoramic video stream