Multi-projection color correction based on free-form deformation technology
-
摘要:
为了解决多通道投影显示系统中各投影仪投影画面之间颜色不一致的问题，设计了一种基于自由变形技术的多投影颜色校正方法。首先，通过Bernstein基函数建立自由变形技术模型，建立各投影仪原始图像和摄像机拍摄图像之间的颜色转换关系；其次，采集原始图像集和摄像机拍摄的投影显示画面集，确定自由变形技术模型的参数；然后，通过Matlab分析验证投影图像各颜色通道之间是相互影响的；最后，对原始图像进行颜色扭曲，将本文颜色校正方法与传统方法进行直方图相似性评估对比。实验结果表明：对比广义颜色校正方法和基于B样条曲线的颜色校正方法，本文方法将投影画面的颜色强度平均差值在B通道减少了2.50，在G通道减少了2.34，在R通道减少了3.57，直方图的相关性提高了8.9%，巴氏距离降低了9.7%。基于自由变形技术的多投影颜色校正方法使投影画面衔接流畅，给用户带来更好的沉浸感受。
-
关键词:
- 多通道投影系统 /
- 颜色校正 /
- 自由变形技术 /
- Bernstein基函数
Abstract:In order to solve the problem of color inconsistency between the projection pictures of various projectors in a multi-channel projection display system, a multi-projection color correction method based on free-form deformation technology was designed. Firstly, the free-form deformation model was established by Bernstein basis function, and the color conversion relationship between the original image of each projector and the image taken by the camera was established. Then, collected the original image set and the projection display picture set taken by the camera to determine the parameters of the free-form deformation technology model. Moreover, through Matlab analysis, it was verified that each color channel of projection image was affected by each other. Finally, the original image was color distorted, and the proposed color correction method was compared with the traditional method for histogram similarity evaluation. The experimental results show that, compared with the generalized color correction method and the color correction method based on B-spline curve, the proposed method reduces the average difference of color intensity of the projected image by 2.50 in the B channel, 2.34 in the G channel, and 3.57 in the R channel. The correlation of histogram is increased by 8.9%, and the Pap distance is reduced by 9.7%.The multi-projection color correction method based on free-form deformation makes the projection images smoothly connected, and brings the user a better immersion experience.
-
引言
短波红外是指波长为1 μm~2.5 μm的红外辐射，其利用目标反射环境中的短波红外辐射来实现探测。短波红外可以提供可见光、近红外、中波红外和长波红外所不能提供的信息，填补近红外和中波红外成像之间的光谱空缺，实现在3个大气红外透射窗口的“无缝隙探测”，对在红外波段全面获取目标的信息具有重要意义[1-7]。
在国外，短波红外探测器及成像系统在机载平台上已经得到广泛运用，逐渐成为机载光电吊舱主要光电传感器之一。国内机载光电系统探测/感知的光谱波段涉及可见光、近红外、中波红外和长波红外，短波红外成像系统在机载平台上运用较少。
1 短红外探测技术在机载光电吊舱上的运用
在国外，短波红外成像系统已成为机载光电吊舱低照度光电传感器，FLIR公司推出的Star SAFIRE 380-HDc、Star SAFIRE 380-HLD等机载光电吊舱内集成铟镓砷短波红外成像系统，可对车辆目标上的激光光斑进行可视化探测，将红外/彩色视频图像与短波红外视频图像进行融合，增强态势感知能力，如图 1和图 2所示。
![]()
图 2 Star SAFIRE 380-HLD光电吊舱对激光光斑探测Figure 2. Laser spot detected by Star SAFIRE 380-HLD EO pod2 短波红外探测器灵敏度计算
2.1 短波红外探测器参数
以Xenics公司生产的短波红外探测器XSW-640为例，探测器主要参数如下。
探测器类型：InGaAs COMS二极管阵列
像元阵列和尺寸：640像素×512像素，20 μm×20 μm
响应波段：900 nm~1 700 nm
量子效率：大于65%(1 000 nm~1 600 nm)
积分时间范围：1 μs~40 ms(调节补偿为1 μs)
噪声(实测值/1 ms积分时间)： < 500 electrons(高动态)/ < 95 electrons(高增益)
动态范围：66 dB(高动态范围)/52 dB(高增益)
![]()
图 3 XSW-640短波红外探测器外形及量子效率Figure 3. XSW-640 SWIR detector and quantum efficiency curves2.2 探测器灵敏度计算
在1ms积分时间内，短波红外探测器输出信噪比为
$$ ^{\frac{S}{N}=\frac{\left( {{D}_{\text{QE}}}\cdot ~\frac{P}{hf}\text{ }~ \right)ev}{~{{i}_{\text{dark}}}~}} $$ (1) 式中：DQE为量子效率(计算中取量子效率为65%)；P为入射光功率；hf为单光子能量；idark为噪声电流；e为单个电子电荷电量；v为电荷运动速度。当SNR取值为1时，可计算得到P，从而可得到探测灵敏度Hmin为3.6×10-2 lx。
2.3 典型环境下短波红外照度
短波红外基于目标对环境辐射或照射激光反射成像，λ1~λ2波段内的辐射照度为
$$ ^{{{E}_{{{\lambda }_{1}}\tilde{\ }{{\lambda }_{2}}}}={{E}_{\text{total}}}\times ({{F}_{0\tilde{\ }{{\lambda }_{2}}}}-{{F}_{0\tilde{\ }{{\lambda }_{1}}}})} $$ (2) 式中F0~λ是波长从0~λ的黑体辐射占0~+∞黑体辐射的百分比，根据文献1给出的环境照度值，计算得到不同环境下0.9 μm~1.7 μm波段照度Eatm如表 1所示。
表 1 不同条件下0.9 μm~1.7 μm波段环境照度Table 1. Environmental illuminance in 0.9 μm ~ 1.7 μm under different environments天空状态 环境照度Eatm/lx 散射太阳光 (2.446~4.9)×103 阴天 2.446×102 黄昏/阴暗天 2.446×101 黎明/暮光 2.446 微明 2.446×10-1 2.4 特定距离上的激光照度
激光测照器对于特定距离的目标进行测照，被照射目标上的激光照度Elaser为
$$ ^{{{E}_{\text{laser}}}=~\frac{{{P}_{t}}~{{\tau }_{t}}}{\pi {{\theta }^{2}}~{{R}^{2}}}~~{{\text{e}}^{-\mu R}}~} $$ (3) 式中：Pt为激光照射功率，75 mJ/20 Hz；τt为发射光学系统透过率，取0.9；μ为激光发射机到激光光斑之间大气平均衰减系数；θ为激光束散角，0.11 mrad；R为激光发射机到激光光斑的距离。
3 激光光斑和环境探测输出信噪比
信噪比S/N定义为输出信号与随机噪声均方根的比值，根据GJB2705-1996要求，相机探测输出信噪比S/N最小值不小于10。对激光光斑和环境探测输出信噪比为[8-10]
$$ \begin{matrix} {{(\frac{S}{N})}_{\text{target}}}=~\frac{{{\rho }_{\text{target}}}\left( {{E}_{\text{laser}}}+{{E}_{\text{atm}}} \right){{\tau }_{r}}{{\theta }^{2}}{{A}_{r}}}{{{S}_{\text{d}}}{{H}_{\text{min}}}~}\text{exp}(-\mu R)~~ \\ {{(\frac{S}{N})}_{\text{atm}}}=~\frac{{{\rho }_{\text{atm}}}{{E}_{\text{atm}}}{{\tau }_{r}}{{\theta }^{2}}{{A}_{r}}}{{{S}_{\text{d}}}{{H}_{\text{min}}}}~\text{ }~\text{ exp}(-\mu R) \\ \end{matrix} $$ (4) 式中：ρtarget为目标反射率，黑底铬绿涂料平均反射率为0.24[11]；ρatm为环境反射率，靶场环境平均反射率为0.36；Hmin为探测器阈值灵敏度，3.6×10-2 lx；τr为接收光学系统透过率，取0.75；Ar为接收面积，接收口径为Φ61 mm；Sd为探测器面积，1.28×10-4 m2。
在1km大气能见度条件下，短波红外热像仪对于环境和激光光斑探测输出信噪比如图 3所示。当环境照度分别为0.244 6 lx和2.446 lx时，短波红外热像无法实现特定距离处环境清晰成像；当环境照度分别为24.46 lx和244.6 lx时，能够对0.68 km和1.66 km处环境景物进行清晰成像。当环境照度分别为0.244 6 lx、2.446 lx、24.46 lx和244.6 lx时，能够对1.05 km、1.06 km、1.1 km和1.51 km处激光光斑清晰成像。
采用某InGaAs短波红外热像仪在50 m高度对外界场景进行成像实验，该热像仪采用TEC制冷，探测器分辨率为320像素×256像素、像元尺寸为25 μm×25 μm，光学视场为20°×15°、光学口径为10 mm。
在1 km大气能见度、阴天下午(预计环境照度为244.6 lx)条件下，能够实现对2 km处楼层探测；在1km大气能见度、阴天晚上(预计环境照度为1 lx)条件下，无法对0.15 km处楼层探测。如图 5所示, 实验结果与仿真计算相符。
4 激光光斑和背景探测输出对比度
对于特定距离处，目标与背景对比度C0为
$$ ^{{{C}_{0}}=\text{ }\frac{\left( {{E}_{\text{laser}}}+{{E}_{\text{atm}}} \right){{\rho }_{t\text{arget}}}-{{E}_{\text{atm}}}{{\rho }_{\text{atm}}}\text{ }~\text{ }}{\left( {{E}_{\text{laser}}}+{{E}_{\text{atm}}} \right){{\rho }_{\text{target}}}\text{+}{{E}_{\text{atm}}}{{\rho }_{\text{atm}}}~}} $$ (5) 短波红外热像仪探测输出目标与背景对比度C为
$$ ^{\begin{smallmatrix} C=~{{C}_{0}}\times \text{MT}{{\text{F}}_{\text{A}}}\times \text{MT}{{\text{F}}_{\text{LOS}}}\times \text{MT}{{\text{F}}_{\text{Optics}}}\times \\ \ \ \ \ \ \ \text{MT}{{\text{F}}_{\text{detector}}}\times \text{MT}{{\text{F}}_{\text{E}}}~ \end{smallmatrix}} $$ (6) 式中：MTFA为大气对比度调制传递函数，包括含大气通道传递函数和大气湍流传递函数；MTFLOS为瞄准线稳定性调制传递函数；MTFOptics为光学系统调制传递函数；MTFdetector为光电成像探测器调制传递函数；MTFE为信号处理调制传递函数。
不同大气能见度条件下，短波红外热像对于激光光斑和背景探测输出对比度如图 6所示。大气能见度为1 km、探测输出对比度阈值为0.02，环境照度分别为0.244 6 lx、2.446 lx、24.46 lx和244.6 lx时，短波红外热像对激光光斑探测距离分别为1.55 km、1.43 km、1.16 km和0.81 km。对于高环境照度条件下，激光照度随着距离的增加而降低，当目标辐射亮度和环境辐射亮度相等时，短波红外热像探测输出目标背景对比度为0，无法区分目标和背景。如图 6(b)曲线所示，当环境照度为244.6 lx，激光测照距离为1.79 km时，短波红外热像仪探测输出目标/背景对比度为0；在2.64 km处，探测输出目标背景对比度为0.022。仿真结果表明：1)由于激光照度的降低，探测输出目标背景对比度随着测照距离的增加而降低；2)特定距离下，探测输出目标背景对比度随着环境照度的增加而降低；3)探测输出目标背景对比度随着大气能见度的增加而增加。
5 结论
由于短波红外热像仪具有穿透能力强，可以对激光光斑进行可视化探测等特点，在机载平台上已经得到广泛的运用。基于文中激光测照器和短波红外热像仪设定参数，计算结果表明：短波红外热像仪对激光光斑探测距离与大气能见度、环境照度、阈值信噪比、阈值对比度、探测器灵敏度等众多因素有关，在大气能见度1 km~3 km、环境照度1 lx~1 000 lx、阈值信噪比2.25~10、阈值对比度0.01~0.02条件下，短波红外对激光光斑探测距离为0 km~3.5 km。
短波红外对于激光光斑的探测，还存在如下规律：
1) 当环境照度较低时，受限于探测灵敏度，短波红外对于环境态势(周围树木、道路等)探测能力差；
2) 当环境照度较高，或目标背景和辐射源(太阳)成为镜面反射时，受限于探测输出对比度，激光光斑将无法从背景中区分；
3) 当环境照度较高时，对于短波红外热像进行滤光设计(以激光波段为中心)，可降低环境照度，提高激光光斑的探测信噪比和探测距离。
-
![]()
图 1 多投影显示系统颜色校正流程图
Figure 1. Color correction flow chart of multi-projection display system
![]()
图 2 图像经投影后RGB颜色强度的变化波动图
Figure 2. Fluctuation diagram of RGB color intensity after projection
![]()
图 3 不同方法颜色校正后投影图像的直方图
Figure 3. Histogram of projection images after color correction with different methods
![]()
图 4 基于自由变形技术颜色校正后的投影图像
Figure 4. Projection image after color correction based on free-form deformation technology
![]()
图 5 四通道CAVE系统颜色校正前后效果图
Figure 5. Effect picture of four channel CAVE system before and after color correction
表 1 RGB各通道输出颜色强度值的极差和均方根误差
Table 1 Range and root mean square error of output color intensity values of RGB channels
输入的颜色通道 输出的颜色强度值 输入变化的
颜色通道输入固定的
颜色通道样本极差 样本标准差 样本均方根
误差B G 36.426 5 11.920 1 22.740 2 R 26.759 4 8.246 8 22.860 5 G B 49.295 9 17.519 5 24.256 9 R 21.806 4 6.393 7 11.878 5 R B 37.010 5 13.584 2 17.911 8 G 10.056 3 3.587 7 22.824 3 
下载: 导出CSV
-
[1] WANG X, YAN K, LIU Y. Automatic geometry calibration for multi-projector display systems with arbitrary continuous curved surfaces[J]. IET Image Processing,2019,13(7):1050-1055. doi: 10.1049/iet-ipr.2018.5575
[2] HEINZ M, BRUNNETT G. Optimized GPU-based post-processing for stereoscopic multi-projector display systems[J]. Virtual Reality,2019,23(1):45-60. doi: 10.1007/s10055-018-0352-5
[3] MOUTAFIS B E, GRAVVANIS G A, FILELIS-PAPADOPOULOS C K. Hybrid multi-projection method using sparse approximate inverses on GPU clusters[J]. The International Journal of High Performance Computing Applications,2020,34(3):282-305. doi: 10.1177/1094342020905637
[4] BU F, WANG X, GAO B. A multi-projection deep computation model for smart data in internet of things[J]. Future Generation Computer Systems,2019,93:68-76. doi: 10.1016/j.future.2018.09.060
[5] WANG M Y, HAN Y, WANG R, et al. A software-hardware cooperative method for multi-projector seamless tiled display system[J]. IEICE Electronics Express,2015,12:20141104. doi: 10.1587/elex.12.20141104
[6] 韩成, 张超, 秦贵和, 等. 大型正交多幕投影系统光辐射补偿算法[J]. 吉林大学学报（工学版）, 2015 (4): 1266-1273. HAN Cheng, ZHANG Chao, QIN Guihe, et al. Optical radiation compensation algorithm for large-scale orthogonal multi screen projection system [J]. Journal of Jilin University( Engineering Edition), 2015 (4): 1266-1273.
[7] ZHAO S, ZHAO M, DAI S. Automatic registration of multi-projector based on coded structured light[J]. Symmetry,2019,11(11):556-563.
[8] TEHRANI M A, GOPI M, MAJUMDER A. Automated geometric registration for multi-projector displays on arbitrary 3D shapes using uncalibrated devices[J]. IEEE Transactions on Visualization and Computer Graphics,2019,27(4):2265-2279.
[9] 王修晖, 王康健, 陆慧娟. 背投式多投影系统中各向异性问题的研究[J]. 中国图像图形学报,2015,20(4):499-505. WANG Xiuhui, WANG Kangjian, LU Huijuan. Study on anisotropy in rear projection multi projection system[J]. Chinese Journal of Image and Graphics,2015,20(4):499-505.
[10] 侯培国, 张铮, 宋涛, 等. Lab 的多投影颜色校正及亮度融合技术[J]. 光学精密工程, 2021, 29(7): 1667-1677. HOU Peiguo, ZHANG Zheng, SONG Tao, et al. Multi projection color correction and brightness fusion technology of lab[J]. Optical Precision Engineering, 2021, 29 (7): 1667-1677.
[11] 邵佳丰, 罗晨, 周怡君, 等. 基于自由变形技术的分流叶片形状优化设计[J]. 航空动力学报, 2021, 36(6): 1315-1323. SHAO Jiafeng, LUO Chen, ZHOU Yijun, et al. Shape optimization design of splitter blade based on free deformation technology[J]. Journal of Aeronautical Power, 2021, 36 (6): 1315-1323.
[12] ASKARIAN B S, POURREZA H, NALBANDIAN S. Scalable and view-independent calibration of multi-projector display for arbitrary uneven surfaces[J]. Machine Vision and Applications,2019,30(7):1191-1207.
[13] INDRASWARI R, KURITA T, ARIFIN A Z, et al. Multi-projection deep learning network for segmentation of 3D medical images[J]. Pattern Recognition Letters,2019,125:791-797. doi: 10.1016/j.patrec.2019.08.003
[14] KURTH P, LANGE V, SIEGL C, et al. Auto-calibration for dynamic multi-projection mapping on arbitrary surfaces[J]. IEEE Transactions on Visualization and Computer Graphics,2018,24(11):2886-2894. doi: 10.1109/TVCG.2018.2868530
[15] FUJITA T, HATANO K, KIJIMA S, et al. Online combinatorial optimization with multiple projections and its application to scheduling problem[J]. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences,2018,101(9):1334-1343.
[16] LIU X Z, LIN Y, XU X G, et al. Multi-projector calibration based on virtual viewing space[J]. International Journal of Virtual Reality,2019,19(3):16-30.
[17] HU H M, FANG W, LI B, et al. An adaptive multi-projection metric learning for person re-identification across non-overlapping cameras[J]. IEEE Transactions on Circuits and Systems for Video Technology,2018,29(9):2809-2821.
[18] ZHANG Q, ZHANG C, LING J, et al. A generic multi-projection-center model and calibration method for light field cameras[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(11):2539-2552.
计量
- 文章访问数: 182
- HTML全文浏览量: 101
- PDF下载量: 40
陕公网安备 61011302001501号 