Chinese Optics, Volume. 16, Issue 3, 479(2023)

Recent progress of non-line-of-sight imaging reconstruction algorithms in typical imaging modalities

Lu-da ZHAO1,2、*, Xiao DONG1,2,3, Shi-long XU1,2,3, Yi-hua HU1,2,3、*, Xin-yuan ZHANG1,2,3, and Yi-cheng ZHONG4
Author Affiliations
  • 1College of Electronic Engineering, National University of Defence Technology, Hefei 230037, China
  • 2State Key Laboratory of Pulsed Power Laser Technology, National University of Defense Technology, Hefei 230037, China
  • 3Anhui Province Key Laboratory of Electronic Restriction, National University of Defense Technology, Hefei 230037, China
  • 4Unit 77126 of the PLA, Kaiyuan 661600, China
  • show less
    References(91)

    [1] [1] RAMESH R, DAVIS J. 5d timelight transpt matrix: What can we reason about scene properties[EBOL]. [20080301] [20080301]. http:dspace.mt.eduhle1721.167888.

    [2] [2] KIRMANI A, HUTCHISON T, DAVIS J, et al.. Looking around the cner using transient imaging[C]. 2009 IEEE 12th International Conference on Computer Vision, IEEE, 2009: 159166.

    [4] MAEDA T, SATAT G, SWEDISH T, et al. Recent advances in imaging around corners[J]. arXiv preprint arXiv:, 05613, 2019(1910).

    [5] GENG R X, HU Y, CHEN Y. Recent advances on non-line-of-sight imaging: Conventional physical models, deep learning, and new scenes[J]. APSIPA Transactions on Signal and Information Processing, 11, e1(2021).

    [7] WU SH K, ZHANG Y N. Research progress of passive non-line-of-sight imaging methods[J]. Optoelectronic Technology, 41, 87-93(2021).

    [12] [12] NAIK N, ZHAO SH, VELTEN A, et al.. Single view reflectance capture using multiplexed scattering timeofflight imaging[C]. Proceedings of the 2011 SIGGRAPH Asia Conference, Association f Computing Machinery, 2011: 171.

    [19] HEIDE F, O’TOOLE M, ZANG K, et al. Non-line-of-sight imaging with partial occluders and surface normals[J]. ACM Transactions on Graphics, 38, 22(2019).

    [21] [21] PHARKAR R, VELTEN A, BARDAGJY A, et al.. Estimating motion size of moving nonlineofsight objects in cluttered environments[C]. CVPR 2011, IEEE, 2011: 265272.

    [22] [22] HEIDE F, XIAO L, HEIDRICH W, et al.. Diffuse mirrs: 3D reconstruction from diffuse indirect illumination using inexpensive timeofflight senss[C]. Proceedings of the 2014 IEEE Conference on Computer Vision Pattern Recognition, IEEE, 2014: 32223229.

    [23] [23] TSAI C Y, KUTULAKOS K N, NARASIMHAN S G, et al.. The geometry of firstreturning photons f nonlineofsight imaging[C]. Proceedings of the 2017 IEEE Conference on Computer Vision Pattern Recognition, IEEE, 2017: 23362344.

    [24] KADAMBI A, ZHAO H, SHI B X, et al. Occluded imaging with time-of-flight sensors[J]. ACM Transactions on Graphics, 35, 15(2016).

    [25] [25] TSAI C Y, SANKARANARAYANAN A C, GKIOULEKAS I. Beyond volumetric albedoa surface optimization framewk f nonlineofsight imaging[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 15451555.

    [26] ISERINGHAUSEN J, HULLIN M B. Non-line-of-sight reconstruction using efficient transient rendering[J]. ACM Transactions on Graphics, 39, 8(2020).

    [27] [27] AHN B, DAVE A, VEERARAGHAVAN A, et al.. Convolutional approximations to the general nonlineofsight imaging operat[C]. Proceedings of the 2019 IEEECVF International Conference on Computer Vision, IEEE, 2019: 78887898.

    [28] CHEN J H, ZHANG Y, GUO SH SH, et al. Joint estimation of NLOS building layout and targets via sparsity-driven approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 60, 5114513(2022).

    [29] [29] LIU X CH, BAUER S, VELTEN A. Analysis of feature visibility in nonlineofsight measurements[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 1013210140.

    [30] LINDELL D B, WETZSTEIN G, O'TOOLE M. Wave-based non-line-of-sight imaging using fast f-k migration[J]. ACM Transactions on Graphics, 38, 116(2019).

    [31] [31] YOUNG S I, LINDELL D B, GIROD B, et al.. Nonlineofsight surface reconstruction using the directional lightcone transfm[C]. Proceedings of the 2020 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2020: 14041413.

    [33] [33] ISOGAWA M, CHAN D, YUAN Y, et al.. Efficient nonlineofsight imaging from transient sinograms[C]. Proceedings of the 16th European Conference on Computer Vision, Springer, 2020: 193208.

    [40] [40] LIU X CH, VELTEN A. The role of wigner distribution function in nonlineofsight imaging[C]. 2020 IEEE International Conference on Computational Photography (ICCP), IEEE, 2020: 112.

    [45] [45] XIN SH M, NOUSIAS S, KUTULAKOS K N, et al.. A they of fermat paths f nonlineofsight shape reconstruction[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 67936802.

    [46] [46] SMITH B M, O''TOOLE M, GUPTA M. Tracking multiple objects outside the line of sight using speckle imaging[C]. Proceedings of the 2018 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2018: 62586266.

    [47] SASAKI T, LEGER J R. Light-field reconstruction from scattered light using plenoptic data[J]. Proceedings of SPIE, 10772, 1077203(2018).

    [49] [49] SAUNDERS C, BOSE R, MURRAYBRUCE J, et al.. Multidepth computational periscopy with an dinary camera[C]. ICASSP 20202020 IEEE International Conference on Acoustics, Speech Signal Processing (ICASSP), IEEE, 2020: 92999305.

    [50] [50] BARADAD M, YE V, YEDIDIA A B, et al.. Inferring light fields from shadows[C]. Proceedings of the 2018 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2018: 62676275.

    [51] [51] BOUMAN K L, YE V, YEDIDIA A B, et al.. Turning cners into cameras: Principles methods[C]. Proceedings of the 2017 IEEE International Conference on Computer Vision, IEEE, 2017: 22892297.

    [52] SEIDEL S W, MURRAY-BRUCE J, MA Y T, et al. Two-dimensional non-line-of-sight scene estimation from a single edge occluder[J]. IEEE Transactions on Computational Imaging, 7, 58-72(2020).

    [53] [53] SEIDEL S W, MA Y T, MURRAYBRUCE J, et al.. Cner occluder computational periscopy: estimating a hidden scene from a single photograph[C]. 2019 IEEE International Conference on Computational Photography (ICCP), IEEE, 2019: 19.

    [54] [54] YEDIDIA A B, BARADAD M, THRAMPOULIDIS C, et al.. Using unknown occluders to recover hidden scenes[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 1222312231.

    [55] [55] MAEDA T, WANG Y Q, RASKAR R, et al.. Thermal nonlineofsight imaging[C]. 2019 IEEE International Conference on Computational Photography (ICCP), IEEE, 2019: 111.

    [56] [56] TANAKA K, MUKAIGAWA Y, KADAMBI A. Polarized nonlineofsight imaging[C]. Proceedings of the 2020 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2020: 21332142.

    [62] [62] CHOPITE J G, HULLIN M B, W M, et al.. Deep nonlineofsight reconstruction[C]. Proceedings of the 2020 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2020: 957966.

    [65] [65] CHEN W ZH, DANEAU S, BROSSEAU C, et al.. Steadystate nonlineofsight imaging[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 67836792.

    [66] [66] CHRAN S, JAYASURIYA S. Adaptive lighting f datadriven nonlineofsight 3d localization object identification[C]. Proceedings of the 30th British Machine Vision Conference 2019, BMVC, 2019.

    [69] MUSARRA G, CARAMAZZA P, TURPIN A, et al. Detection, identification, and tracking of objects hidden from view with neural networks[J]. Proceedings of the SPIE, 10978, 1097803(2019).

    [70] [70] SU SH CH, HEIDE F, WETZSTEIN G, et al.. Deep endtoend timeofflight imaging[C]. Proceedings of the 2018 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2018: 63836392.

    [71] CHEN Y, QU B, LU X Q. Robust speckle-autocorrelation non-line-of-sight imaging with generative adversarial networks[J]. Proceedings of the SPIE, 12083, 120830B(2022).

    [73] [73] LEI X, HE L Y, TAN Y X, et al.. Direct object recognition without lineofsight using optical coherence[C]. Proceedings of the 2019 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2019: 1172911738.

    [74] AITTALA M, SHARMA P, MURMANN L, et al. Computational mirrors: blind inverse light transport by deep matrix factorization[J]. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Curran Associates Inc., 1283(2019).

    [76] CHEN W ZH, WEI F Y, KUTULAKOS K N, et al. Learned feature embeddings for non-line-of-sight imaging and recognition[J]. ACM Transactions on Graphics, 39, 230(2020).

    [77] [77] MU F ZH, MO S CH, PENG J Y, et al.. Physics to the rescue: deep nonlineofsight reconstruction f highspeed imaging[JOL]. IEEE Transaction on Pattern Analysis Machine Intelligence, 2022: 112.

    [78] [78] GRAU J, PLACK M, HAEHN P, et al.. Occlusion fields: an implicit representation f nonlineofsight surface reconstruction[JOL]. arXiv preprint arXiv: 2203.08657, 2022.

    [79] [79] TANCIK M, SATAT G, RASKAR R. Flash photography f datadriven hidden scene recovery[JOL]. arXiv preprint arXiv: 1810.11710, 2018.

    [80] GENG R X, HU Y, LU ZH, et al. Passive non-line-of-sight imaging using optimal transport[J]. IEEE Transactions on Image Processing, 31, 110-124(2021).

    [82] [82] ISOGAWA M, YUAN Y, O''TOOLE M, et al.. Optical nonlineofsight physicsbased 3D human pose estimation[C]. Proceedings of the 2020 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2020: 70117020.

    [83] [83] HE Y, ZHANG D H, HU Y, et al. . Nonlineofsight imaging with radio signals[C]. 2020 AsiaPacific Signal Infmation Processing Association Annual Summit Conference (APSIPA ), IEEE, 2020: 1116.

    [84] [84] GALINDO M, MARCO J, O''TOOLE M, et al.. A dataset f benchmarking timeresolved nonlineofsight imaging[C]. ACM SIGGRAPH 2019 Posters, Association f Computing Machinery, 2019: 73.

    [86] NETRAPALLI P, JAIN P, SANGHAVI S. Phase retrieval using alternating minimization[J]. Proceedings of the 26th International Conference on Neural Information Processing Systems, Curran Associates Inc., 2796-2804(2013).

    [87] [87] KUPYN O, BUDZAN V, MYKHAILYCH M, et al.. DeblurGAN: blind motion deblurring using conditional adversarial wks[C]. Proceedings of the 2018 IEEECVF Conference on Computer Vision Pattern Recognition, IEEE, 2018: 81838192.

    Tools

    Get Citation

    Copy Citation Text

    Lu-da ZHAO, Xiao DONG, Shi-long XU, Yi-hua HU, Xin-yuan ZHANG, Yi-cheng ZHONG. Recent progress of non-line-of-sight imaging reconstruction algorithms in typical imaging modalities[J]. Chinese Optics, 2023, 16(3): 479

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Review

    Received: Aug. 24, 2022

    Accepted: Nov. 25, 2022

    Published Online: May. 31, 2023

    The Author Email:

    DOI:10.37188/CO.2022-0186

    Topics