FPGA-based real-time atmospheric turbulence image restoration algorithm and its implementation

LI Yawei; ZHANG Hong; WU Lingfan; YANG Yifan; CHEN Hao

doi:10.5768/JAO202142.0602002

Journal of Applied Optics > 2021 > 42(6): 1017-1024. > DOI: 10.5768/JAO202142.0602002

LI Yawei, ZHANG Hong, WU Lingfan, YANG Yifan, CHEN Hao. FPGA-based real-time atmospheric turbulence image restoration algorithm and its implementation[J]. Journal of Applied Optics, 2021, 42(6): 1017-1024. DOI: 10.5768/JAO202142.0602002

Citation:

PDF (1531 KB)

FPGA-based real-time atmospheric turbulence image restoration algorithm and its implementation

School of Astronautics, Beihang University, Beijing 102206, China

More Information

Received Date: March 10, 2021
Revised Date: August 21, 2021
Available Online: October 10, 2021

Graphical Abstract

Abstract

Abstract

Due to the influence of atmospheric turbulence, the images obtained by the remote optical imaging equipment will appear serious degradation phenomena, such as geometric distortion, motion blur and defocus blur, etc. At present, the mainstream turbulence restoration algorithms usually rely on the non-rigid registration, reconstruction or finding rare lucky regions from long-term video sequences. These methods require huge calculated amount or complete video data in advance, which cannot meet the real-time requirements of actual application scenarios. Therefore, a real-time turbulence restoration algorithm that could be implemented in field programmable gate array (FPGA) was proposed. This method used the randomness of atmospheric turbulence to filter the image in the time domain through the continuous frames and solved the problem of image geometric distortion. Then, the Wiener filter in the frequency domain was converted into a convolution in the spatial domain, which was easier to implement, and solved the problem of image blur. The experimental results show that the proposed algorithm not only meets the real-time requirements, but also effectively realizes the restoration of turbulence images.
- atmospheric turbulence,
- image restoration,
- FPGA,
- real-time image processing

FullText(HTML)

References (23)

References

[1]	ZHU X, MILANFAR P. Stabilizing and deblurring atmospheric turbulence[C]//2011 IEEE International Conference on Computational Photography (ICCP). USA: IEEE, 2011: 1-8.
[2]	刘彦飞, 代永红, 周浩天, 等. 振动与远场湍流对机载精跟踪终端的影响[J]. 光子学报,2015,44(7):706001-706008. doi: 10.3788/gzxb20154407.0706001 LIU Yanfei, DAI Yonghong, ZHOU Haotian, et al. Effect of vibration and far field turbulence on the fine tracking of airborne optical ccommunication terminal[J]. Acta Photonica Sinica,2015,44(7):706001-706008. doi: 10.3788/gzxb20154407.0706001
[3]	洪汉玉, 喻九阳, 陈以超, 等. 红外目标湍流退化图像的优化复原算法[J]. 应用光学,2006,27(6):510-515. HONG Hanyu, YU Jiuyang, CHEN Yichao, et al. Optimization restoration algorithm for infrared object turbulence degraded image[J]. Journal of Applied Optics,2006,27(6):510-515.
[4]	张利, 孙传东, 何俊华. 基于成像自适应光学的水下成像系统研究[J]. 应用光学,2010,31(5):690-694. ZHANG Li, SUN Chuandong, HE Junhua. Underwater imaging system based on adaptive optics[J]. Journal of Applied Optics,2010,31(5):690-694.
[5]	MASAO S, SHIN Y, MASAYUKI T, et al. Super-resolution from image sequence under influence of hot-air optical turbulence[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage. USA: IEEE, 2008: 1-8.
[6]	ZHU X, MILANFAR P. Image reconstruction from videos distorted by atmospheric turbulence[C]//Visual Information Processing and Communication. International Society for Optics and Photonics. USA: SPIE, 2010: 75430S.
[7]	LAU C P, LAI Y H, LUI L M. Restoration of atmospheric turbulence-distorted images via RPCA and quasiconformal maps[J]. Inverse Problems,2017,35:074002.
[8]	ZHU X, MILANFAR P. Removing atmospheric turbulence via space-invariant deconvolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(1):157-170. doi: 10.1109/TPAMI.2012.82
[9]	LAU C P, LAI Y H, LUI L M. Variational models for joint subsampling and reconstruction of turbulence-degraded images[J]. Journal of Science Computer,2019,78:1488-1525. doi: 10.1007/s10915-018-0833-4
[10]	FRIED D L. Probability of getting a lucky short-exposure image through turbulence[J]. Journal of the Optical Society of America,1978,68(12):1651-1658. doi: 10.1364/JOSA.68.001651
[11]	JOHN S, VORONTSOV M A. Multiframe selective information fusion from robust error estimation theory[J]. IEEE Transactions on Image Processing,2005,14(5):577-584. doi: 10.1109/TIP.2005.846022
[12]	AUBAILLY M, VORONTSOV M A, CARHART G W, et al. Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach[C]//Atmospheric Optics: Models, Measurements, and Target-in-the-Loop Propagation III. USA: SPIE, 2009, 7463: 74630C.
[13]	鲁啸天, 杨天鸣, 金伟其, 等. 水面波动和水体湍流退化图像的复原方法[J]. 应用光学,2017,38(1):42-55. LU Xiaotian, YANG Tianming, JIN Weiqi, et al. Correction methods for water fluctuation and underwater turbulence degraded imaging[J]. Journal of Applied Optics,2017,38(1):42-55.
[14]	OREIFEJ O, LI X, SHAH M. Simultaneous video stabilization and moving object detection in turbulence[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(2):450-462.
[15]	OREIFEJ O, SHU G, PACE T, et al. A two-stage reconstruction approach for seeing through water[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2011: 1153-1160.
[16]	LAU C P, LAI Y H, LUI L M. Restoration of atmospheric turbulence-distorted images via RPCA and quasiconformal maps[J]. Inverse Problems, 2019, 35(7): 074002.
[17]	ZON N, KIRYATI N. Unified functional framework for restoration of image sequences degraded by atmospheric turbulence[C]//2017 Energy Minimization Methods in Computer Vision and Pattern Recognition. Switzerland: Springer, Cham, 2017: 10746.
[18]	COHEN B, AVRIN V, BELITSKY M, et al. Generation of a restored image from a video sequence recorded under turbulence effects[J]. Optical Enginering,1997,36:3312-3317. doi: 10.1117/1.601592
[19]	GAL R, KIRYATI N, SOCHEN N A. Progress in the restoration of image sequences degraded by atmospheric turbulence[J]. Pattern Recognition Letters,2014,48:8-14. doi: 10.1016/j.patrec.2014.04.007
[20]	MAO Y, GILLES J. Turbulence stabilization[C]//2012 SPIE Conference on Defense, Security and Sensing. USA: SPIE, 2012: 83550H.
[21]	LAU C P, CASTILLO C D, CHELLAPPA R. ATFaceGAN: single face semantic aware image restoration and recognition from atmospheric turbulence[C]//IEEE Transactions on Biometrics, Behavior, and Identity Science. USA: IEEE,2015: 240-251.
[22]	PAUL N, DE CHILLAZ A, COLLETTE J L. On-line restoration for turbulence degraded video in nuclear power plant reactors[J]. Signal, Image and Video Processing,2015(3):601-610.
[23]	Carasso A S, Bright D S, Vladar A E. APEX method and real-time blind deconvolution of scanning electron microscope imagery[J]. Optical Engineering,2002,41:2499-2514. doi: 10.1117/1.1499970

[1]	SU Zhengcheng, LI Jun, WU Fan, LAN Shiqi, YAN Bo. Effect of temperature-induced half-wave voltage characteristics of Y-waveguide modulator on resonant fiber optic gyroscope[J]. Journal of Applied Optics, 2024, 45(2): 461-466. DOI: 10.5768/JAO202445.0208003
[2]	LIU Junhan, QU Tianliang, ZHANG Xuan, LIU Yanqing, XIONG Changxin. Fabrication of ultra-high Q factor and millimeter-scale crystal echo wall microcavity[J]. Journal of Applied Optics, 2023, 44(4): 742-747. DOI: 10.5768/JAO202344.0401006
[3]	HUANG Yueruihan, MA Jiajun, ZHANG Zipeng, LIU Jianning, JIANG Junbiao. Scale factor compensation technology of laser gyro with total reflection prism[J]. Journal of Applied Optics, 2023, 44(2): 444-449. DOI: 10.5768/JAO202344.0207003
[4]	HU Tieli, WANG Honghong, LI Siwei, CAO Feng, HU Xinyi, FAN Zhe, YANG Yuxin, GUO Jian, YOU Yue, YANG Ke, LI Hui, YU Yang. Research on temperature control and self-tuning for 30℃～420℃ blackbody[J]. Journal of Applied Optics, 2023, 44(2): 392-397. DOI: 10.5768/JAO202344.0203005
[5]	CHEN Wei, LIU Yu, LI Hongtao, SUN Jing, YAN Ning. Improved ViBe algorithm based on adaptive threshold and dynamic update factor[J]. Journal of Applied Optics, 2022, 43(3): 444-452. DOI: 10.5768/JAO202243.0302004
[6]	ZHOU Wenqing, FEI Yuming, HONG Guijie, YING Guangyao, YE Xin. Research on magnetic temperature characteristics of zero error on high precision fiber-optic gyro[J]. Journal of Applied Optics, 2020, 41(1): 220-227. DOI: 10.5768/JAO202041.0108001
[7]	WANG Xuelian, WU Zhifeng, SONG Guicai, LU Xiaofeng, DAI Caihong. Measurement of absolute linearity using laser covering large-scale dynamic range[J]. Journal of Applied Optics, 2019, 40(4): 681-685. DOI: 10.5768/JAO201940.0407001
[8]	Wan Xun, Xie Liangping. Temperature field analysis and structure redesign of fiber optic gyroscopeWan Xun, Xie Liangping[J]. Journal of Applied Optics, 2016, 37(3): 353-358. DOI: 10.5768/JAO201637.0301006
[9]	ZHI Jian-hui, DONG Xin-min, KONG Xing-wei, WANG Xu-feng. Analysis of external influence factors in camera calibration[J]. Journal of Applied Optics, 2014, 35(2): 286-291.
[10]	QIAO Xue-guang, LI Ting, WANG Hong-liang, JIA Zhen-an, LIU Qin-peng, WANG Xiang-yu. Response characteristics of high temperature resistant fiber Bragg grating[J]. Journal of Applied Optics, 2007, 28(2): 209-211.

Cited By

Cited by

Periodical cited type(6)

1.	杨朝政，李淑英. 基于虚拟现实技术的激光三维图像优化系统设计. 激光杂志. 2023(04): 152-157 .
2.	郏露锋，张凯亮. 基于LabVIEW的大视场激光视频图像采集系统. 激光杂志. 2023(04): 259-264 .
3.	张宇驰，陈义，李鸿. 基于视频技术的煤矿在线应急预警系统的研究与应用. 煤炭技术. 2022(02): 189-193 .
4.	姜睿林，陈春晓，崔建良，徐俊琪. 基于FPGA的内窥镜窄带光生成方法研究. 生物医学工程研究. 2022(01): 7-12 .
5.	张泽宇，张弘，伍凌帆，杨一帆，李旭亮. 基于FPGA的实时Bayer解马赛克算法与实现. 应用光学. 2022(02): 240-247 . 本站查看
6.	格日勒，王刚，柳智鑫. 基于视频技术的煤矿在线应急预警系统研究. 能源与环保. 2021(09): 41-45 .

Other cited types(4)

Get Citation

PDF

XML

Article views (960) PDF downloads (147) Cited by(10)

Turn off MathJax

Article Contents

Abstract

References

FPGA-based real-time atmospheric turbulence image restoration algorithm and its implementation