基于FPGA的低照度条件下EBAPS图像混合噪声去除算法

夏皓天; 钱芸生; 王逸伦; 郎怡政

doi:10.5768/JAO202243.0604006

基于FPGA的低照度条件下EBAPS图像混合噪声去除算法

doi: 10.5768/JAO202243.0604006

南京理工大学电子工程与光电技术学院，江苏南京 210094

基金项目: 国家自然科学基金“叶企孙”科学基金项目(U2141239)

详细信息

作者简介:
夏皓天（1998—），男，硕士研究生，主要从事光电信息探测、微光成像技术与微光图像处理研究。E-mail：xiahaotian0422@163.com

通讯作者:
钱芸生（1968—），男，教授，博导，主要从事光电成像器件、系统与相关测试技术研究。 E-mail：yshqian@njust.edu.cn

中图分类号: TN206
计量
- 文章访问数: 146
- HTML全文浏览量: 57
- PDF下载量: 17
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-01
- 修回日期: 2022-09-24
- 网络出版日期: 2022-10-11
- 刊出日期: 2022-11-14

Mixed noise removal algorithm of EBAPS image under low illumination condition based on FPGA

College of Electronic Engineering and Optoelectronic Technology, Nanjing University of Science and Technology, Nanjing 210094, China

摘要

摘要: 为了解决单一的中值滤波和高斯滤波算法对低照度图像中同时存在的脉冲噪声和泊松噪声抑制效果不佳、边缘细节保护不足的问题，提出一种基于可编程逻辑门阵列(field programmable gate array，FPGA)的开关中值-高斯融合(open and close mix-median-Gaussian，OCMMG)滤波算法。首先，利用最小四方向差值检测每个像素点的异常程度，根据脉冲噪声判别阈值分配权重，进行第1步滤波处理；然后，利用四方向边缘检测算法提取图像边缘，根据设置的边缘置信度表征值进行第2步滤波处理；最后，用电子轰击有源像素传感器(electron bombarded active pixel sensor，EBAPS)在1×10⁻³ lx照度条件下采集的图像，基于FPGA对其进行实时图像处理。实验结果表明，FPGA处理结果与软件仿真处理结果相符。该算法相比于中值滤波和高斯滤波算法，峰值信噪比分别提高了3.23%和16.34%，结构相似性分别提高了14.66%和33.86%，边缘保持指数分别提高了0.49%和4.21%，能够有效去除EBAPS图像的混合噪声，并满足实时性要求。
- FPGA /
- 图像降噪 /
- 低照度 /
- EBAPS /
- 混合噪声 /
- 边缘检测
Abstract: In order to solve the problem that single median filtering and gaussian filtering algorithm is not effective in suppressing impulse noise and poisson noise simultaneously in low illumination image, and the edge detail protection is insufficient, an open and close mix-median-gaussian (OCMMG) filtering algorithm based on field programmable gate array (FPGA) was proposed. Firstly, the minimum four-direction difference was used to detect the anomaly degree of each pixel point, the weight was allocated according to the threshold of pulse noise discrimination, and the first step was filtering. Then, the four-direction edge detection algorithm was used to extract image edges, and the second step was filtered according to the set edge confidence characterization value. Finally, the images collected by electron bombarded active pixel sensor (EBAPS) under the condition of 1×10⁻³ lx illumination were processed by FPGA in real time. The experimental results show that the FPGA processing results are consistent with the software simulation processing results. Compared with the median filtering and gaussian filtering algorithm, the peak signal-to-noise ratio (PSNR) of the algorithm is improved by 3.23% and 16.34%, the structural similarity is improved by 14.66% and 33.86%, and the edge retention index is improved by 0.49% and 4.21%, respectively, which can effectively remove the mixed noise of EBAPS image and meet the real-time requirements.
- field programmable gate array /
- image denoising /
- low illumination /
- electron bombarded active pixel sensor /
- mixed noise /
- edge detection

HTML全文

图 1 系统总体框架设计

Fig. 1 Design diagram of overall framework for system

下载: 全尺寸图片幻灯片

图 2 算法模块流程图

Fig. 2 Flow chart of algorithm module

下载: 全尺寸图片幻灯片

图 3 3×3模板生成图

Fig. 3 Diagram of 3×3 template generation

下载: 全尺寸图片幻灯片

图 4 3×3矩阵映射图

Fig. 4 Diagram of 3×3 matrix mapping

下载: 全尺寸图片幻灯片

图 5 最小四方向差值流程图

Fig. 5 Flow chart of minimum four-direction differences

下载: 全尺寸图片幻灯片

图 6 快速中值滤波模块流程图

Fig. 6 Flow chart of fast median filter module

下载: 全尺寸图片幻灯片

图 7 边缘检测模块流程图

Fig. 7 Flow chart of edge detection module

下载: 全尺寸图片幻灯片

图 8 高斯滤波模板

Fig. 8 Diagram of gaussian filter template

下载: 全尺寸图片幻灯片

图 9 仿真测试图

Fig. 9 Diagram of simulation test

下载: 全尺寸图片幻灯片

图 10 照度1的去噪结果对比图

Fig. 10 Comparison diagram of denoising results of illuminance 1

下载: 全尺寸图片幻灯片

图 11 照度2的去噪结果对比图

Fig. 11 Comparison diagram of denoising results of illuminance 2

下载: 全尺寸图片幻灯片

图 12 照度3的去噪结果对比图

Fig. 12 Comparison diagram of denoising results of illuminance 3

下载: 全尺寸图片幻灯片

图 13 硬件设备及其内部结构图

Fig. 13 Hardware device and its internal structure diagram

下载: 全尺寸图片幻灯片

图 14 高压电源与照度计

Fig. 14 High-voltage power supply and illuminometer

下载: 全尺寸图片幻灯片

图 15 静态靶标实验结果对比图

Fig. 15 Comparison of experimental results of static targets

下载: 全尺寸图片幻灯片

图 16 静态SF实验结果对比图

Fig. 16 Comparison of experimental results of static SF graphics

下载: 全尺寸图片幻灯片

图 17 夜晚外部环境实验结果对比图

Fig. 17 Comparison of experimental results of external environment at night

下载: 全尺寸图片幻灯片

图 18 运动目标实验结果对比图

Fig. 18 Comparison of experimental results of moving targets

下载: 全尺寸图片幻灯片

图 19 仿真软件图像处理运行时间

Fig. 19 Image processing running time of simulation software

下载: 全尺寸图片幻灯片

图 20 FPGA图像处理运行时间

Fig. 20 Image processing running time of FPGA

下载: 全尺寸图片幻灯片

表 1 照度1的测试图去噪后评价结果

Table 1 Evaluation results of test map after denoising of illuminance 1

滤波算法	Lena			Goldhill			Pepper
滤波算法	PSNR	SSIM	EPI	PSNR	SSIM	EPI	PSNR	SSIM	EPI
噪声图	21.499 4	0.294 2	0.906 4	21.385 5	0.412 4	0.911 9	21.361 2	0.322 5	0.924 4
中值滤波	31.966 5	0.514 3	0.990 6	29.479 4	0.588 5	0.984 8	31.131 6	0.490 2	0.991 3
高斯滤波	28.930 4	0.460 0	0.980 9	27.960 8	0.570 8	0.978 3	28.330 8	0.452 9	0.983 4
文献[24]	29.537 0	0.450 8	0.983 8	28.520 8	0.588 3	0.981 3	28.875 0	0.467 0	0.985 6
文献[25]	28.824 4	0.519 3	0.986 9	27.398 7	0.616 4	0.983 1	27.562 0	0.494 5	0.987 0
本文算法	33.006 8	0.583 2	0.993 6	31.176 6	0.685 4	0.989 7	32.237 3	0.560 4	0.993 3

下载: 导出CSV

表 2 照度2的测试图去噪后评价结果

Table 2 Evaluation results of test map after denoising of illuminance 2

滤波算法	Lena			Goldhill			Pepper
滤波算法	PSNR	SSIM	EPI	PSNR	SSIM	EPI	PSNR	SSIM	EPI
噪声图	21.131 3	0.164 5	0.796 3	21.633 6	0.225 3	0.752 0	21.848 4	0.177 3	0.787 9
中值滤波	28.558 6	0.401 0	0.984 0	33.145 6	0.479 3	0.974 0	33.880 7	0.378 8	0.981 7
高斯滤波	27.195 8	0.322 8	0.955 7	29.416 4	0.407 5	0.940 1	29.759 3	0.317 6	0.953 2
文献[24]	27.253 9	0.304 6	0.966 9	30.225 4	0.409 2	0.951 4	30.282 3	0.307 1	0.959 8
文献[25]	28.669 0	0.409 1	0.979 4	31.267 9	0.501 6	0.969 1	30.906 6	0.377 3	0.973 3
本文算法	28.737 5	0.442 1	0.987 2	34.417 5	0.567 1	0.980 6	34.791 6	0.431 0	0.985 2

下载: 导出CSV

表 3 照度3的测试图去噪后评价结果

Table 3 Evaluation results of test map after denoising of illuminance 3

滤波算法	Lena			Goldhill			Pepper
滤波算法	PSNR	SSIM	EPI	PSNR	SSIM	EPI	PSNR	SSIM	EPI
噪声图	21.644 9	0.126 4	0.608 4	21.617 5	0.194 0	0.674 5	21.549 4	0.138 9	0.653 9
中值滤波	35.779 9	0.345 5	0.968 8	34.227 3	0.451 8	0.968 5	35.414 4	0.342 8	0.974 0
高斯滤波	29.757 1	0.260 3	0.888 7	29.529 6	0.365 9	0.913 5	29.631 2	0.261 9	0.908 9
文献[24]	31.664 7	0.257 5	0.925 7	31.170 1	0.374 8	0.940 0	31.350 6	0.267 5	0.937 8
文献[25]	34.312 5	0.354 4	0.962 2	32.972 5	0.482 4	0.964 6	33.657 7	0.350 5	0.967 3
本文算法	36.776 0	0.387 9	0.975 2	35.399 4	0.531 8	0.975 9	36.527 8	0.388 7	0.979 9

下载: 导出CSV

表 4 仿真软件与FPGA算法客观评价一致性检验

Table 4 Consistency test of objective evaluation of simulation software and FPGA algorithm

参数	PSNR			SSIM			EPI
参数	仿真软件	FPGA	误差/%	仿真软件	FPGA	误差/%	仿真软件	FPGA	误差/%
靶标1×10⁻¹ lx	32.606 7	32.384 3	0.68	0.604 6	0.647 7	6.65	0.958 3	0.993 4	3.53
靶标1×10⁻² lx	30.476 2	30.384 3	0.30	0.529 7	0.577 0	8.20	0.931 5	0.955 0	2.46
靶标1×10⁻³ lx	34.291 6	34.168 5	0.36	0.527 5	0.564 6	6.57	0.930 2	0.938 1	0.84
SF 1×10⁻¹ lx	33.197 8	33.047 7	0.45	0.563 6	0.605 3	6.89	0.982 9	0.995 5	1.27
SF 1×10⁻² lx	30.779 7	30.604 9	0.57	0.535 4	0.554 9	3.51	0.907 8	0.968 6	6.28
SF 1×10⁻³ lx	35.385 0	35.668 3	0.79	0.505 6	0.561 1	9.89	0.933 7	0.960 1	2.75
夜晚外部环境	35.004 3	35.368 4	1.03	0.617 8	0.635 3	2.75	0.989 2	0.997 0	0.78

下载: 导出CSV

参考文献(30)

[1]	王彦斌, 张德锋, 周旋风, 等. CCD光敏面上规则点阵的产生机理[J]. 光学精密工程,2021,29(6):1242-1250. WANG Yanbin, ZHANG Defeng, ZHOU Xuanfeng, et al. Formation mechanism of regular lattices on photosensitive surface of CCD[J]. Optics and Precision Engineering,2021,29(6):1242-1250.
[2]	BOLDUC E, FACCIO D, LEACH J. Acquisition of multiple photon pairs with an EMCCD camera[J]. Journal of Optics,2017,19(5):054006. doi: 10.1088/2040-8986/aa52d8
[3]	MORAN S E, ULICH B L, ELKINS W P, et al. Intensified CCD (ICCD) dynamic range and noise performance[J]. Proceedings of SPIE - the International Society for Optical Engineering,1997,3173:430-457.
[4]	COATES C G, DE NVIR D J, MCHALE N G, et al. Optimizing low-light microscopy with back-illuminated electron multiplying charge-coupled device: enhanced sensitivity, speed, and resolution[J]. Journal of Biomedical Optics,2004,9(6):1244. doi: 10.1117/1.1805559
[5]	蔡宁静, 曾祥忠, 王波. EBCMOS相机盲元检测方法的研究[J]. 应用光学,2021,42(1):137-142. doi: 10.5768/JAO202142.0103007 CAI Ningjing, ZENG Xiangzhong, WANG Bo. Research on blind element detection method of EBCMOS camera[J]. Journal of Applied Optics,2021,42(1):137-142. doi: 10.5768/JAO202142.0103007
[6]	AEBI V W, JAMES B J. Electron bombarded active pixel sensor: US20140267641 A1 [P]. 2003-05-02.
[7]	GEORGE M. Electron bombarded back-illuminated CCD sensors for lowlight level imaging applications[J]. SPIE,1995,2415:211-235. doi: 10.1117/12.206518
[8]	唐小东. EBAPS电子轰击性能测试技术研究[D]. 南京: 南京理工大学, 2019. TANG Xiaodong. Research on EBAPS electron bombardment performance test technology [D]. Nanjing: Nanjing University of Science and Technology, 2019.
[9]	KURATA H, OTSUGA K, KOTABE A, et al. Random telegraph signal in flash memory: its impact on scaling of multilevel flash memory beyond the 90-nm node[J]. IEEE Journal of Solid-State Circuits,2007,42(6):1362-1369. doi: 10.1109/JSSC.2007.897158
[10]	MAJID A, MAHMOOD M T, CHOI T S. A novel noise-free pixels based impulse noise filtering [C]// IEEE International Conference on Image Processing. USA: IEEE, 2010:125-128.
[11]	康宇, 赵冬青, 上官鹏, 等. 基于FPGA的图像边缘保护高斯滤波算法实现[J]. 电子设计工程,2021,29(6):94-98. doi: 10.14022/j.issn1674-6236.2021.06.020 KANG Yu, ZHAO Dongqing, SHANGGUAN Peng, et al. Realizaton of image edge protection Gauss filter algorithm based on FPGA[J]. International Electronic Elements,2021,29(6):94-98. doi: 10.14022/j.issn1674-6236.2021.06.020
[12]	ANSCOMBE F J. The transformation of poisson, binomial and negative-binomial data[J]. Biometrika,1948,35:246-254. doi: 10.1093/biomet/35.3-4.246
[13]	FRYZLEWICZ P, NASON G P. A Haar-Fisz algorithm for poisson intensity estimation[J]. Journal of Computational and Graphical Statistics,2004,13(3):621-638. doi: 10.1198/106186004X2697
[14]	许玉婷, 吴志芳, 王强, 等. 基于剪切波变换的辐射图像泊松噪声降噪技术研究[J]. 原子能科学技术,2022,56(3):577-584. XU Yuting, WU Zhifang, WANG Qiang, et al. Denoise technology for radiation image with poisson noise based on shearlet transform[J]. Atomic Energy Science and Technology,2022,56(3):577-584.
[15]	王光新, 王正明, 谢美华, 等. 泊松噪声模糊图像的边缘保持变分复原算法[J]. 光电子·激光,2007,18(3):359-363. doi: 10.3321/j.issn:1005-0086.2007.03.028 WANG Guangxin, WANG Zhengming, XIE Meihua, et al. An edge preserving variational restoration method of blurred image corrupted by poisson noise[J]. Journal of Optoelectronics·Laser,2007,18(3):359-363. doi: 10.3321/j.issn:1005-0086.2007.03.028
[16]	陈明轩. 数字图像中随机值脉冲噪声的滤除算法研究[D]. 北京: 北京信息科技大学, 2013: 4. CHEN Mingxuan. Research on filtering algorithm of random value impulse noise in digital image[D]. Beijing: Beijing Information Science and Technology University, 2013: 4.
[17]	邹瑜. 一种易于FPGA实现的椒盐噪声滤除算法[J]. 电子测试,2021(20):40-41. doi: 10.3969/j.issn.1000-8519.2021.20.015 ZOU Yu. A salt and pepper noise filtering algorithm easy to implement on FPGA[J]. Electronic Test,2021(20):40-41. doi: 10.3969/j.issn.1000-8519.2021.20.015
[18]	左振鹏. 数字图像自适应去噪算法的FPGA实现[J]. 电子技术应用,2012,38(11):14-15. doi: 10.3969/j.issn.0258-7998.2012.11.006 ZUO Zhenpeng. A new adaptive switching filter for digital image on FPGA[J]. Application of Electronic Technology,2012,38(11):14-15. doi: 10.3969/j.issn.0258-7998.2012.11.006
[19]	王丽红, 胡长宏, 范鲜红, 等. 自适应中值滤波器优化及其FPGA实现[J]. 哈尔滨理工大学学报,2021,26(5):68-75. doi: 10.15938/j.jhust.2021.05.009 WANG Lihong, HU Changhong, FAN Xianhong, et al. Optimization of adaptive median filter and FPGA implementation[J]. Journal of Harbin University of Science and Technology,2021,26(5):68-75. doi: 10.15938/j.jhust.2021.05.009
[20]	LIU Y H, GAO K, NI G Q. An improved trilateral filter for Gaussian and impulse noise removal [C]// International Conference on Industrial Mechatronics & Automation. USA: IEEE, 2010:14566891.
[21]	XU Guangyu, TAN Jianqing, ZHONG Jinqin. An improved trilateral filter for image denoising using an effective impulse detector [C]//Proceedings of the 4th International Congress on Image and Signal Processing. USA: IEEE, 2011: 90-94.
[22]	JUNG S W. Enhancement of image and depth map using adaptive joint trilateral filter[J]. IEEE Transactions on Circuits and Systems for Video Technology,2013,23(2):258-269. doi: 10.1109/TCSVT.2012.2203734
[23]	丁继生, 卫伟, 杨依忠, 等. 一种基于FPGA的开关中值滤波算法研究[J]. 合肥工业大学学报(自然科学版),2016,39(4):490-493. DING Jisheng, WEI Wei, YANG Yizhong, et al. Research on switching median filter algorithm based on FPGA[J]. Journal of Hefei University of Technology:Nature Science Edition,2016,39(4):490-493.
[24]	陈伟, 党榕. 混合噪声图像的去噪算法研究及FPGA实现[J]. 电子测量技术,2021,44(16):135-140. CHEN Wei, DANG Rong. Research on denoising algorithm of mixed noise image and FPGA implementation[J]. Electronic Measurement Technology,2021,44(16):135-140.
[25]	涂友钢. 单兵夜视眼镜视频信号处理技术研究[D]. 南京: 南京理工大学, 2019. TU Yougang. Research on video signal processing technology of soldier night vision glasses [D]. Nanjing: Nanjing University of Science and Technology, 2019.
[26]	褚翔, 祝连庆, 娄小平, 等. 基于改进Sobel算子的动态自动调焦算法研究[J]. 应用光学,2017,38(2):237-242. CHU Xiang, ZHU Lianqing, LOU Xiaoping, et al. Dynamic auto focus algorithm based on improved Sobel operator[J]. Journal of Applied Optics,2017,38(2):237-242.
[27]	宁永慧, 刘辉, 赵庆磊, 等. 大面阵高帧频CMOS成像电子学系统设计[J]. 光学精密工程,2019,27(5):1167-1177. doi: 10.3788/OPE.20192705.1167 NING Yonghui, LIU Hui, ZHAO Qinglei, et al. High-frame frequency imaging system of large area CMOS image sensor[J]. Optics and Precision Engineering,2019,27(5):1167-1177. doi: 10.3788/OPE.20192705.1167
[28]	KELE O, YLMAZ M A, TEKALP A M, et al. On the computation of PSNR for a set of images or video [C]// 2021 Picture Coding Symposium (PCS). USA: IEEE, 2021: 9477470.
[29]	韩昊男, 钱锋, 吕建威, 等. 图像去雾方法质量评价[J]. 光学精密工程,2022,20(6):721-733. HAN Haonan, QIAN Feng, LYU Jianwei. Image dehazing method quality assessment[J]. Optics and Precision Engineer,2022,20(6):721-733.
[30]	JUNKE L, XIN J. EPI-neighborhood distribution based light field depth estimation [C]//IEEE International Conference on Acoustics, Speech and Signal Processing. USA: IEEE, 2020: 2003-2007.