| Citation: |
WEI Wei, CHEN Fen, ZHANG Huabo, LUO Yingguo, ZHANG Peng, PENG Zongju. Light field images super-resolution based on sub-pixel and gradient guide[J]. Journal of Applied Optics, 2024, 45(5): 956-965. DOI: 10.5768/JAO202445.0502003
|
For the problem of low spatial resolution of light field images captured by light field cameras, a super-resolution method for light field images based on sub-pixel and gradient guide was proposed. A multiple sub-pixel information extraction module was designed, which divided the sub-aperture images into four image stacks: horizontal, vertical, diagonal and anti-diagonal, and extracted the sub-pixel information of each image stack separately. Meanwhile, considering that the gradient prior could provide effective clues for predicting high-frequency details, the gradient multiple sub-pixel information of the sub-aperture images was fused in the reconstruction process. Experimental results on five publicly available databases show that the proposed method not only generally outperforms the existing methods in terms of objective indexes, but also has better performance in the subjective visual effect, which the edge texture details are clearer.
| [1] |
JIA C, SHI F, ZHAO M. Object detection based on light field imaging[C]// Proceedings of 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design, May 4-6, 2022, Hangzhou, China. New York: IEEE, 2022: 239-244.
|
| [2] |
HAN L, SHI Z, ZHENG S N, et al. Light field depth estimation using RNN and CRF[C]// Proceedings of 2022 7th International Conference on Image, Vision and Computing, July 26-28, 2022, Xi’an, China. New York: IEEE, 2022: 725-729.
|
| [3] |
宋力争, 林冬云, 彭侠夫, 等. 基于深度学习的patch-match双目三维重建[J]. 应用光学, 2022, 43(3): 436-443. doi: 10.5768/JAO202243.0302003
SONG Lizheng, LIN Dongyun, PENG Xiafu, et al. Patch-match binocular 3D reconstruction based on deep learning[J]. Journal of Applied Optics, 2022, 43(3): 436-443. doi: 10.5768/JAO202243.0302003
|
| [4] |
XU Y F, ZHAO J K. Deep multi-levels edge-guided network for super-resolution[C]// Proceedings of 2022 19th International Computer Conference on Wavelet Active Media Technology and Information Processing, December 16-18, 2022, Chengdu, China. New York: IEEE, 2022: 1-5.
|
| [5] |
LIU Y H, LI S M, LIU A Q. Two-way guided super-resolution reconstruction network based on gradient prior[C]// Proceedings of 2021 IEEE International Conference on Image Processing, September 19-22, 2021, Anchorage, AK, USA. New York: IEEE, 2021: 1819-1823.
|
| [6] |
YOON Y, JEON H G, YOO D, et al. Light field image super-resolution using convolutional neural network[J]. IEEE Signal Processing Letters, 2017, 24(6): 848-852. doi: 10.1109/LSP.2017.2669333
|
| [7] |
WANG Y Q, WANG L G, YANG J G, et al. Spatial-angular interaction for light field image super-resolution[C]// Proceedings of the 16th European Conference on Computer Vision, August 23-28, 2020, Glasgow, United Kingdom. Berlin: Springer, 2020: 290-308.
|
| [8] |
YEUNG H W F, HOU J H, CHEN X M, et al. Light field spatial super-resolution using deep efficient spatial-angular separable convolution[J]. IEEE Transactions on Image Processing, 2019, 28(5): 2319-2330. doi: 10.1109/TIP.2018.2885236
|
| [9] |
WU G C, ZHAO M D, WANG L Y, et al. Light field reconstruction using deep convolutional network on EPI[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 1638-1646.
|
| [10] |
安平, 陈欣, 陈亦雷, 等. 基于视点图像与EPI特征融合的光场超分辨率[J]. 信号处理, 2022, 38(9): 1818-1830.
AN Ping, CHEN Xin, CHEN Yilei, et al. Light field super-resolution based on viewpoint image and EPI feature fusion[J]. Signal Processing, 2022, 38(9): 1818-1830.
|
| [11] |
ZHANG S, LIN Y, SHENG H. Residual networks for light field image super-resolution[C]// Proceeding of 2019 IEEE Conference on Computer Vision and Pattern Recognition, June 16-21, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 11038-11047.
|
| [12] |
ZHANG S, CHANG S, LIN Y. End-to-end light field spatial super-resolution network using multiple epipolar geometry[J]. IEEE Transactions on Image Processing, 2021, 30: 5956-5968. doi: 10.1109/TIP.2021.3079805
|
| [13] |
JIN J, HOU J H, CHEN J, et al. Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization[C]// Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition, June 13-19, 2020, Seattle, WA, USA. New York: IEEE, 2020: 2260-2269.
|
| [14] |
WANG Y Q, YANG J G, WANG L G, et al. Light field image super-resolution using deformable convolution[J]. IEEE Transactions on Image Processing, 2021, 30: 1057-1071. doi: 10.1109/TIP.2020.3042059
|
| [15] |
WANG S Z, ZHOU T F, LU Y, et al. Detail-preserving transformer for light field image super-resolution[J]. AAAI Conference on Artificial Intelligence, 2022, 36(3): 2522-2530. doi: 10.1609/aaai.v36i3.20153
|
| [16] |
CAO J Z, LI Y W, ZHANG K, et al. Video super-resolution transformer[EB/OL]. [2023-06-15]. https://arxiv.org/abs/2106.06847.
|
| [17] |
DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. [2023-06-15]. https://www.cnblogs.com/lucifer1997/p/18210509.
|
| [18] |
HUI Z, GAO X B, YANG Y C, et al. Lightweight image super-resolution with information multi-distillation network[C]// Proceedings of the 27th ACM International Conference on Multimedia, October, 2019, New York, NY, USA. New York: ACM, 2019: 2024-2032.
|
| [19] |
RERABEK M, EBRAHIMI T. New light field image dataset[C]// Proceedings of 8th International Conference on Quality of Multimedia Experience. Lisbon: Portugal, 2016: 2409457.
|
| [20] |
HONAUER K, JOHANNSEN O, KONDERMANN D, et al. A dataset and evaluation methodology for depth estimation on 4D light fields[C]// 13th Asian Conference on Computer Vision. Taiwan: Springer , 2016, 19-34.
|
| [21] |
WANNER S, MEISTER S, GOLDLUECKE B. Datasets and benchmarks for densely sampled 4D light fields[J]. Vision, Modelling and Visualization, 2013, 13: 225-226.
|
| [22] |
PENDU M L, JIANG X R, GUILLEMOT C. Light field in painting propagation via low rank matrix completion[J]. IEEE Transactions on Image Processing, 2018, 27(4): 1981-1993. doi: 10.1109/TIP.2018.2791864
|
| [23] |
VAISH V, ADAMS A. The (new) standford light field archive[EB/OL]. [2023-06-15]. http://lightfield.stanford.edu/.
|
| [24] |
KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C] //IEEE Conference on Computer Vision and Pattern Recognition, June 26-July 1, 2016, Las Vegas, USA. New York: IEEE, 2016: 1646-1654.
|
| [25] |
LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Honolulu, USA. New York: IEEE, 2017: 1132-1140.
|
WeChat
You are the 2904064th visitor, Welcome!
Address: No.9, west section of No.3 electronic road, Xi'an Post Code: 710065 Tel: 029-88288172
Copyright © Editorial Department of Journal of Applied Optics
陕公网安备 61011302001501号 
DownLoad: