Laser & Optoelectronics Progress, Volume. 58, Issue 18, 1811014(2021)

Stereo Image Super-Resolution: A Survey

Yingqian Wang, Longguang Wang, Zhengyu Liang, Wei An, and Jungang Yang*
Author Affiliations
  • College of Electronic Science and Technology, National University of Defense Technology, Changsha, Hunan 410073, China
  • show less
    References(85)

    [1] Dong X Y, Sun X, Jia X P et al. Remote sensing image super-resolution using novel dense-sampling networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 1618-1633(2021).

    [2] Dong X Y, Wang L G, Sun X et al. Remote sensing image super-resolution using second-order multi-scale networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 3473-3485(2021).

    [4] Huang Y W, Shao L, Frangi A F. Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 5787-5796(2017).

    [5] Oktay O, Ferrante E, Kamnitsas K et al. Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation[J]. IEEE Transactions on Medical Imaging, 37, 384-395(2017).

    [6] Hu F, Lin Y, Hou M D et al. Super-resolution reconstruction of cytoskeleton image based on deep learning[J]. Acta Optica Sinica, 40, 2410001(2020).

    [7] Xiao K, Tian L J, Wang Z Y. Fast super-resolution fluorescence microscopy imaging with low signal-to-noise ratio based on deep learning[J]. Chinese Journal of Lasers, 47, 1007002(2020).

    [8] Bar-Zion A, Tremblay-Darveau C, Solomon O et al. Fastvascular ultrasound imaging with enhanced spatial resolution and background rejection[J]. IEEE Transactions on Medical Imaging, 36, 169-180(2017).

    [9] Seibel H, Goldenstein S, Rocha A. Eyes on the target: super-resolution and license-plate recognition in low-quality surveillance videos[J]. IEEE Access, 5, 20020-20035(2017).

    [10] Pang Y W, Cao J L, Wang J et al. JCS-net: joint classification and super-resolution network for small-scale pedestrian detection in surveillance images[J]. IEEE Transactions on Information Forensics and Security, 14, 3322-3331(2019).

    [11] Chen L, Pan J S, Li Q. Robust face image super-resolution via joint learning of subdivided contextual model[J]. IEEE Transactions on Image Processing, 28, 5897-5909(2019).

    [12] Tan T D, Guo Z M. Research of hand positioning and gesture recognition based on binocular vision[C]. //2011 IEEE International Symposium on VR Innovation, March 19-20, 2011, Singapore., 311-315(2011).

    [13] Oskiper T, Sizintsev M, Branzoi V et al. Augmented reality binoculars[J]. IEEE Transactions on Visualization and Computer Graphics, 21, 611-623(2015).

    [14] Jia B X, Chen J, Zhang K X. Drivable road reconstruction for intelligent vehicles based on two-view geometry[J]. IEEE Transactions on Industrial Electronics, 64, 3696-3706(2017).

    [15] Gong W, Qin L, Ren G F et al. Binocular stereo matching algorithm based on multidimensional feature fusion[J]. Laser & Optoelectronics Progress, 57, 161501(2020).

    [16] Cheng M Y, Gai S Y, Da F P. A stereo-matching neural network based on attention mechanism[J]. Acta Optica Sinica, 40, 1415001(2020).

    [17] Yang W M, Zhang X C, Tian Y P et al. Deep learning for single image super-resolution: a brief review[J]. IEEE Transactions on Multimedia, 21, 3106-3121(2019).

    [18] Wang Z H, Chen J, Hoi S C H. Deep learning for image super-resolution: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2166, 32217470(2020).

    [19] Tang Y Q, Pan H, Zhu Y P et al. A survey of image super-resolution reconstruction[J]. Acta Electronica Sinica, 48, 1407-1420(2020).

    [20] Qu Y Y, Chen R, Li C H et al. Survey of single image super-resolution reconstruction with deep learning[J]. Journal of Xiamen University (Natural Science), 60, 555-570, 641-642(2021).

    [21] Menze M, Geiger A. Objectscene flow for autonomous vehicles[C]. //2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA., 3061-3070(2015).

    [22] Jeon D S, Baek S H, Choi I et al. Enhancing the spatial resolution of stereo images using a parallax prior[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA., 1721-1730(2018).

    [23] Bhavsar A V, Rajagopalan A N. Resolution enhancement in multi-image stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1721-1728(2010).

    [24] Yan B, Ma C X, Bare B et al. Disparity-aware domain adaptation in stereo image restoration[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA., 13176-13184(2020).

    [25] Khamis S, Fanello S, Rhemann C et al. StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction[M]. //Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11219, 596-613(2018).

    [27] Wang L G, Wang Y Q, Liang Z F et al. Learning parallax attention for stereo image super-resolution[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA., 12242-12251(2019).

    [28] Wang L G, Guo Y L, Wang Y Q et al. Parallax attention for unsupervised stereo correspondence learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6899, 32976095(2020).

    [29] Song W, Choi S, Jeong S et al. Stereoscopic image super-resolution with stereo consistent feature[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 12031-12038(2020).

    [30] Ying X Y, Wang Y Q, Wang L G et al. A stereo attention module for stereo image super-resolution[J]. IEEE Signal Processing Letters, 27, 496-500(2020).

    [31] Xu Q Y, Wang L G, Wang Y Q et al. Deep bilateral learning for stereo image super-resolution[J]. IEEE Signal Processing Letters, 28, 613-617(2021).

    [33] Wang Y Q, Wang L G, Yang J G et al. Flickr1024: a large-scale dataset for stereo image super-resolution[C]. //2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), October 27-28, 2019, Seoul, Korea (South), 3852-3857(2019).

    [34] Geiger A, Lenz P, Urtasun R. Arewe ready for autonomous driving? The KITTI vision benchmark suite[C]. //2012 IEEE Conference on Computer Vision and Pattern Recognition, June 16-21, 2012, Providence, RI, USA., 3354-3361(2012).

    [35] Scharstein D, Szeliski R, Zabih R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[C]. //Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), December 9-10, 2001, Kauai, HI, USA., 131-140(2001).

    [36] Scharstein D, Szeliski R. High-accuracy stereo depth maps using structured light[C]. //2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, June 18-20, 2003, Madison, WI, USA., 1-8(2003).

    [37] Scharstein D, Pal C. Learning conditional random fields for stereo[C]. //2007 IEEE Conference on Computer Vision and Pattern Recognition, June 17-22, 2007, Minneapolis, MN, USA., 1-8(2007).

    [38] Hirschmuller H, Scharstein D. Evaluation of cost functions for stereo matching[C]. //2007 IEEE Conference on Computer Vision and Pattern Recognition, June 17-22, 2007, Minneapolis, MN, USA., 1-8(2007).

    [39] Scharstein D, Hirschmüller H, Kitajima Y et al. High-resolution stereo datasets with subpixel-accurate ground truth[M]. //Jiang X Y, Hornegger J, Koch R. Pattern recognition. Lecture notes in computer science, 8753, 31-42(2014).

    [40] Schöps T, Schönberger J L, Galliani S et al. A multi-view stereo benchmark with high-resolution images and multi-camera videos[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 2538-2547(2017).

    [41] Agustsson E, Timofte R. NTIRE 2017 challenge on single image super-resolution: Dataset and study[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), July 21-26, 2017, Honolulu, HI, USA, 1122-1131(2017).

    [42] Timofte R, Agustsson E, van Gool L et al. NTIRE 2017 challenge on single image super-resolution: methods and results[C]. //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 21-26, Honolulu, HI, USA, 114-125(2017).

    [43] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]. //2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA., 1646-1654(2016).

    [44] Lim B, Son S, Kim H et al. Enhanced deep residual networks for single image super-resolution[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), July 21-26, 2017, Honolulu, HI, USA., 1132-1140(2017).

    [45] Zhang Y L, Tian Y P, Kong Y et al. Residual dense network for image super-resolution[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA., 2472-2481(2018).

    [46] Zhang Y L, Li K P, Li K et al. Image super-resolution using very deep residual channel attention networks[M]. //Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11211, 294-310(2018).

    [48] Dai J F, Qi H Z, Xiong Y W et al. Deformable convolutional networks[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy., 764-773(2017).

    [49] Zhu X Z, Hu H, Lin S et al. Deformable ConvNets v2: more deformable, better results[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA, 9300-9308(2019).

    [51] Wang Y Q, Yang J G, Wang L G et al. Light field image super-resolution using deformable convolution[J]. IEEE Transactions on Image Processing, 30, 1057-1071(2020).

    [52] Yang F Z, Yang H, Fu J L et al. Learning texture transformer network for image super-resolution[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA., 5790-5799(2020).

    [53] Tian Y P, Zhang Y L, Fu Y et al. TDAN: temporally-deformable alignment network for video super-resolution[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA, 3357-3366(2020).

    [54] Ying X Y, Wang L G, Wang Y Q et al. Deformable 3D convolution for video super-resolution[J]. IEEE Signal Processing Letters, 27, 1500-1504(2020).

    [57] Kim T H, Sajjadi M S M, Hirsch M et al. Spatio-temporal transformer network for video restoration[M]. //Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11207, 111-127(2018).

    [58] Pang J H, Sun W X, Yang C X et al. Zoom and learn: generalizing deep stereo matching to novel domains[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA, 2070-2079(2018).

    [59] Guo X Y, Li H S, Yi S et al. Learning monocular depth by distilling cross-domain stereo networks[M]. //Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11215, 506-523(2018).

    [60] Tonioni A, Tosi F, Poggi M et al. Real-time self-adaptive deep stereo[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA., 195-204(2019).

    [61] Liu R, Yang C X, Sun W X et al. StereoGAN: bridging synthetic-to-real domain gap by joint optimization of domain translation and stereo matching[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA, 12754-12763(2020).

    [63] Efrat N, Glasner D, Apartsin A et al. Accurate blur models vs. image priors in single image super-resolution[C]. //2013 IEEE International Conference on Computer Vision, December 1-8, 2013, Sydney, NSW, Australia., 2832-2839(2013).

    [64] Yang C Y, Ma C, Yang M H. Single-imagesuper-resolution: a benchmark[M]. //Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science, 8692, 372-386(2014).

    [65] Gu J J, Lu H N, Zuo W M et al. Blind super-resolution with iterative kernel correction[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA., 1604-1613(2019).

    [67] Gu J J, Lu H N, Zuo W M et al. Blind super-resolution with iterative Kernel correction[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA., 1604-1613(2019).

    [69] Michaeli T, Irani M. Nonparametric blind super-resolution[C]. //2013 IEEE International Conference on Computer Vision, December 1-8, 2013, Sydney, NSW, Australia., 945-952(2013).

    [70] Shocher A, Cohen N, Irani M. Zero-shot super-resolution using deep internal learning[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA., 3118-3126(2018).

    [71] Yuan Y, Liu S Y, Zhang J W et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 18-22, 2018, Salt Lake City, UT, USA., 814-823(2018).

    [72] Li Z, Zhang Y. Hyperspectral anomaly detection via image super-resolution processing and spatial correlation[J]. IEEE Transactions on Geoscience and Remote Sensing, 59, 2307-2320(2021).

    [73] Ji H, Gao Z, Mei T C et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 17, 676-680(2020).

    [74] Wang L, Li D, Zhu Y S et al. Dual super-resolution learning for semantic segmentation[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA., 3773-3782(2020).

    [75] Zhang Z P, Zhang K P. FarSee-net: real-time semantic segmentation by efficient multi-scale context aggregation and feature space super-resolution[C]. //2020 IEEE International Conference on Robotics and Automation (ICRA), May 31-August 31, 2020, Paris, France, 8411-8417(2020).

    [76] Lei S, Shi Z W, Wu X et al. Simultaneous super-resolution and segmentation for remote sensing images[C]. //IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, July 28-August 2, 2019, Yokohama, Japan., 3121-3124(2019).

    [77] Chen Y, Tai Y, Liu X M et al. FSRNet: end-to-end learning face super-resolution with facial priors[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA, 2492-2501(2018).

    [78] Chen C F, Gong D H, Wang H et al. Learning spatial attention for face super-resolution[J]. IEEE Transactions on Image Processing, 30, 1219-1231(2021).

    [80] Liang Z F, Guo Y L, Feng Y L et al. Stereo matching using multi-level cost volume and multi-scale feature constancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 300-315(2021).

    [81] Liang Z F, Feng Y L, Guo Y L et al. Learning for disparity estimation through feature constancy[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA., 2811-2820(2018).

    [82] Li X J, Huang H, Zhao H L et al. Learning a convolutional neural network for propagation-based stereo image segmentation[J]. The Visual Computer, 36, 39-52(2020).

    [83] Chen Y L, Liu S, Shen X Y et al. DSGN: deep stereo geometry network for 3D object detection[C]. //2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13-19, 2020, Seattle, WA, USA, 12533-12542(2020).

    [84] Qin Z Y, Wang J L, Lu Y. Triangulation learning network: from monocular to stereo 3D object detection[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA, 7607-7615(2019).

    [85] Li P L, Chen X Z, Shen S J. Stereo R-CNN based 3D object detection for autonomous driving[C]. //2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 15-20, 2019, Long Beach, CA, USA., 7636-7644(2019).

    Tools

    Get Citation

    Copy Citation Text

    Yingqian Wang, Longguang Wang, Zhengyu Liang, Wei An, Jungang Yang. Stereo Image Super-Resolution: A Survey[J]. Laser & Optoelectronics Progress, 2021, 58(18): 1811014

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Imaging Systems

    Received: Jun. 1, 2021

    Accepted: Jun. 26, 2021

    Published Online: Aug. 28, 2021

    The Author Email: Yang Jungang (yangjungang@nudt.edu.cn)

    DOI:10.3788/LOP202158.1811014

    Topics