Real-time implementation of multi-source image fusion algorithm

HUANG Xiying; WANG Jie; WANG Jiaoying; YAN Li; LU Yang; LI Liangfu

doi:10.5768/JAO202243.0402002

Journal of Applied Optics > 2022 > 43(4): 676-681. > DOI: 10.5768/JAO202243.0402002

HUANG Xiying, WANG Jie, WANG Jiaoying, YAN Li, LU Yang, LI Liangfu. Real-time implementation of multi-source image fusion algorithm[J]. Journal of Applied Optics, 2022, 43(4): 676-681. DOI: 10.5768/JAO202243.0402002

Citation:

PDF (1600 KB)

Real-time implementation of multi-source image fusion algorithm

Xi'an Institute of Applied Optics, Xi'an 710065, China

More Information

Received Date: May 06, 2022
Revised Date: June 10, 2022
Available Online: June 22, 2022

Graphical Abstract

Abstract

Abstract

Aiming at the real-time requirements of image fusion in the infrared and visible light, a real-time implementation method to solve the current high-definition or ultrahigh-definition multi-source Laplacian pyramid image fusion was proposed. Based on the video data stream, a parallel processing pipeline architecture of Laplacian pyramid system was designed. The time delay and optimization ideas between the pipelines were analyzed. All the delay time differences were compensated through on-chip cache to achieve the equal length of pipeline and guarantee the integrity of data processing in the whole algorithm. This method could realize 5-level Laplacian pyramid fusion of dual-channel 1080×1920@60 Hz video images on XILINX 7 series field programmable gate array (FPGA) and above. The experimental results show that this real-time method has better fusion effect, with only 10.535 ms of one-frame image fusion and the processing delay is less than 1 ms.
- image fusion,
- Laplacian pyramid,
- FPGA,
- real-time processing,
- Laplacian average

FullText(HTML)

References (15)

References

[1]	王士乐. 多源图像信息融合与增强技术研究[D]. 上海: 东华大学, 2012. WANG Shile. Research of information fusion and enhancement of multi-source image[D]. Shanghai: Donghua University, 2012.
[2]	延翔. 多源图像融合算法研究[D]. 西安: 西安电子科技大学, 2018. YAN Xiang. Research on algorithm for multi-source image fusion[D]. Xi’an: Xidian University, 2018.
[3]	林子慧. 基于多尺度变换的红外与可见光图像融合技术研究[D]. 北京: 中国科学院大学, 2019. LIN Zihui. Research on infrared and visible image fusion based on multi-scale transform[D]. Beijing: University of Chinese Academy of Sciences, 2019.
[4]	吕勇. 多视场红外与可见光图像融合配准技术研究[D]. 西安: 西安工业大学, 2020. LYU Yong. Research on multi-filed infrared and visible image fusion registration technology[D]. Xi’an: Xi’an Technological University, 2020.
[5]	李宇琦, 赵海涛. 基于红外和可见光图像逐级自适应融合的场景深度估计[J]. 应用光学,2020,41(1):24-32. doi: 10.5768/JAO202041.0101004 LI Yuqi, ZHAO Haitao. Depth estimation based on adaptive fusion of infrared and vision light images progressively[J]. Journal of Applied Opitics,2020,41(1):24-32. doi: 10.5768/JAO202041.0101004
[6]	陈清江, 汪泽百, 柴昱洲. 改进VGG网络的多聚焦图像的融合方法[J]. 应用光学,2020,14(3):500-507. CHEN Qingjiang, WANG Zebai, CHAI Yuzhou. Multi-focus image fusion method based on improved VGG network[J]. Journal of Applied Opitics,2020,14(3):500-507.
[7]	黄文博, 严华. 基于两尺度分解和特征提取的红外与可见光图像融合[J]. 现代计算机,2021,16(6):148-153. HUANG Wenbo, YAN Huan. Infrared and visible image fusion based on two-scale decomposition and extraction[J]. Modern Computer,2021,16(6):148-153.
[8]	赵鑫. 基于DSP的多源图像融合系统研究[D]. 安徽: 安徽理工大学, 2020. ZHAO Xin. Research on multi-source image fusion system based on DSP[D]. Anhui: Anhui University of Science and Technology, 2020.
[9]	葛飞. 基于多核DSP的图像融合技术研究[D]. 北京: 北京理工大学, 2016. GE Fei. Research of image fusion based on multicore DSP[D]. Beijing: Beijing Institute of Technology, 2016.
[10]	杨峰. 多DSP图像融合算法设计及优化[D]. 江苏: 南京理工大学, 2014. YANG Feng. Design and optimization of image fusion algorithm based on multicore DSP[D]. Jiangsu: Nanjing Institute of Technology, 2014.
[11]	刘冰. 基于FPGA的图像融合校正算法设计与实现[D]. 北京: 北京理工大学, 2018. LIU Bing. Design and implementation of imgae fusion correction algorithm based on FPGA. Beijing: Beijing Institute of Technology, 2018.
[12]	汤伟. 基于FPGA的红外图像与微光图像融合研究[D]. 湖南: 中南大学, 2012. TANG Wei. Infrared and low-light-level image fusion research based on FPGA. Hunan: Central South University, 2012.
[13]	张昊. 红外与CCD图像的融合研究及FPGA设计[D]. 西安: 西安科技大学, 2015. ZhANG Hao. Infrared and CCD image fusion research and hardware design based on FPGA. Xi’an: Xi’an University of Science and Technology, 2015.
[14]	卢蓉, 高昆, 倪国强, 等. 基于FPGA的多分辨率图像融合系统实时实现的研究[J]. 激光与红外, 2007, 37(7): 1018-1021. LU Rong, GAO Kun, NI Guoqiang, et al. Study on real-time implementation of multi-resolution image fusion system based on FPGA[J]. Laser & Infrared. 2007, 37(7): 1018-1021.
[15]	马永强, 侯宏旭, 王顺利. 基于归一化标量权重映射与融合金字塔的彩图对比度增强算法[J]. 计算机应用研究,2015,32(10):3187-3190. doi: 10.3969/j.issn.1001-3695.2015.10.072 MA Yongqiang, HOU Hongxu, WANG Shunli. Image contrast enhancement algorithm based on normalized scalar weight map coupled fusion pyramid[J]. Application Research of Compluters,2015,32(10):3187-3190. doi: 10.3969/j.issn.1001-3695.2015.10.072

[1]	ZHU Jinquan, YANG Xueli, SUN Kecheng, LIU Donghui, NIU Zhigang. Design and application of photoelectric auxiliary decision system for FPSO crude oil offloading[J]. Journal of Applied Optics, 2022, 43(5): 870-878. DOI: 10.5768/JAO202243.0501007
[2]	LI Dong, GUO Xi, WANG Di, MA Pengbo, SUN Weidong, WANG Mingji. Laser detection of oil-gas diffusion in central drain pipe of floating-roof tank[J]. Journal of Applied Optics, 2021, 42(5): 913-918. DOI: 10.5768/JAO202142.0507001
[3]	WU Bin, YANG Yanzhao, YING Chengping, LIU Hongyuan, ZHANG Peng, WANG Hengfei. Application of terahertz spectroscopy in THDCPD isomers detection[J]. Journal of Applied Optics, 2020, 41(4): 786-790. DOI: 10.5768/JAO202041.0409903
[4]	FU Hongtao, YANG Erlong, LI Cunlei, LIU Jianmei, DONG Chi, SONG Lijia, GUO Chunping. Quantitative identification of hydrocarbon concentration in drilling fluid based on laser Raman spectroscopy[J]. Journal of Applied Optics, 2019, 40(4): 692-698. DOI: 10.5768/JAO201940.0407003
[5]	Liu Jianmei, Li Cunlei, Gao Peng, Wang Rui, Zhu Ning, Fu Hongtao. Identification method of crude oil in petroleum pipeline based on laser Raman detection technology[J]. Journal of Applied Optics, 2018, 39(3): 436-441. DOI: 10.5768/JAO201839.0307002
[6]	Sun Chen, Zhao Yiwu, An Zhongde, Fu Qiang, Zhan Juntong, Duan Jin. Effect of concentration on propagation characteristics of polarized laser in oil-mist diffusion[J]. Journal of Applied Optics, 2017, 38(6): 1012-1017. DOI: 10.5768/JAO201738.0607002
[7]	FENG Rui-shu, LI Wei-wei, ZHOU Qing-li, MU Kai-jun, ZHANG Liang-liang, ZHANG Cun-lin. Vibrational spectrum of RDX investigated with terahertz time-domain spectroscopy[J]. Journal of Applied Optics, 2009, 30(6): 907-910.
[8]	MA Yun-bin, HU Zhi-xin, YANG Jing, MA Jun-peng. Oil pipeline security monitoring system based on fiber Bragg grating sensor[J]. Journal of Applied Optics, 2009, 30(3): 505-509.
[9]	CHENG Shu-chun, ZHANG Yan-ping. Application of fiber grating sensor for oil leak detection in oil industry[J]. Journal of Applied Optics, 2008, 29(3): 441-443.
[10]	WANG Zhong-dong, WANG Yu-tian. A fiberoptic sensing liquid level measuring system for oil storage tanks[J]. Journal of Applied Optics, 2006, 27(1): 69-72.

Cited By

Cited by

Periodical cited type(5)

1.	袁美桂，魏志强，轩新想. 一种新的机载光电侦察系统目标定位精度分析方法. 光学技术. 2024(02): 215-219 .
2.	刘召庆，张芳，朱镭，贾兆辉，文江华，秦川，张兰兰. 巡飞弹目标定位精度分析. 应用光学. 2022(04): 592-598 . 本站查看
3.	王冠，王惠林，骞琨，沈宇，边赟. 机载光电系统目标定位大气折射修正研究. 应用光学. 2022(04): 641-647 . 本站查看
4.	马经帅，于洵，刘晓宇，韩峰，丁良华. 高精度光电跟踪系统中伺服稳定控制算法研究. 应用光学. 2021(04): 597-607 . 本站查看
5.	秦川，陶忠，桑蔚，张鹏，海云波，梅甫麟. 基于粒子滤波的运动目标光电定位仿真研究. 应用光学. 2020(01): 10-17 . 本站查看

Other cited types(2)

Get Citation

PDF

XML

Article views (637) PDF downloads (98) Cited by(7)

Turn off MathJax

Article Contents

Abstract

References

Real-time implementation of multi-source image fusion algorithm