Researching | Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Advanced Photonics Nexus, Volume. 3, Issue 2, 026010(2024)

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Su Wu^1、†, Chan Huang², Jing Lin³, Tao Wang^1,4, Shanshan Zheng^1,4, Haisheng Feng^1,4, and Lei Yu^1、*

Author Affiliations

¹Chinese Academy of Sciences, Anhui Institute of Optics and Fine Mechanics, Hefei, China

²Hefei University of Technology, School of Physics, Department of Optical Engineering, Hefei, China

³Hefei Normal University, Department of Chemical and Chemical Engineering, Hefei, China

⁴University of Science and Technology of China, Science Island Branch of Graduate School, Hefei, China

show less

Full Text Get PDF

Figures&Tables (11)

References (47)

Paper Information

References(47)

[1] X. Liu et al. Non-line-of-sight imaging using phasor-field virtual wave optics. Nature, 572, 620-623(2019).

[2] B. David et al. Wave-based non-line-of-sight imaging using fast f-k migration. ACM Trans. Graph., 38, 116(2019).

[3] T. Maeda et al. Thermal non-line-of-sight imaging, 1-11(2019).

[4] Y. Altmann et al. Quantum-inspired computational imaging. Science, 361, eaat2298(2018).

[5] W. Yang et al. None-line-of-sight imaging enhanced with spatial multiplexing. Opt. Express, 30, 5855-5867(2022).

[6] J. H. Nam et al. Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second. Nat. Commun., 12, 6526(2021).

[7] F. Heide et al. Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph., 38, 1-10(2019).

[8] C. Saunders, J. Murray-Bruce, V. K. Goyal. Computational periscopy with an ordinary digital camera. Nature, 565, 472(2019).

[9] J. Liu et al. Photon-efficient non-line-of-sight imaging. IEEE Trans. Comput. Imaging, 8, 639-650(2022).

[10] D. Faccio, A. Velten, G. Wetzstein. Non-line-of-sight imaging. Nat. Rev. Phys., 2, 318-327(2020).

[11] F. Mu et al. Physics to the rescue: deep non-line-of-sight reconstruction for high-speed imaging. IEEE Trans. Pattern Anal. Mach. Intell., 1-12(2022).

[12] T. Maeda et al. Recent advances in imaging around corners(2019).

[13] C. A. Metzler, D. B. Lindell, G. Wetzstein. Keyhole imaging: non-line-of-sight imaging and tracking of moving objects along a single optical path. IEEE Trans. Comput. Imaging, 7, 1-12(2020).

[14] D. Zhu, W. Cai. Fast non-line-of-sight imaging with two-step deep remapping. ACS Photonics, 9, 2046-2055(2022).

[15] A. Kirmani et al. Looking around the corner using transient imaging, 159-166(2009).

[16] A. Velten et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun., 3, 745(2012).

[17] M. O’Toole, D. B. Lindell, G. Wetzstein. Real-time non-line-of-sight imaging, 1-2(2018).

[18] M. O’Toole, D. B. Lindell, G. Wetzstein. Confocal non-line-of-sight imaging based on the light-cone transform. Nature, 555, 338-341(2018).

[19] S. I. Young et al. Non-line-of-sight surface reconstruction using the directional light-cone transform, 1407-1416(2020).

[20] X. Liu, S. Bauer, A. Velten. Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems. Nat. Commun., 11, 1645(2020).

[21] W. Chen et al. Steady-state non-line-of-sight imaging, 6790-6799(2019).

[22] C. A. Metzler et al. Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging. Optica, 7, 63-71(2020).

[23] S. Tong, A. M. Alessio, P. E. Kinahan. Noise and signal properties in PSF-based fully 3D PET image reconstruction: an experimental evaluation. Phys. Med. Biol., 55, 1453(2010).

[24] J. D. Rego, K. Kulkarni, S. Jayasuriya. Robust lensless image reconstruction via PSF estimation, 403-412(2021).

[25] G. Demoment. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Trans. Acoust. Speech Signal Process., 37, 2024-2036(1989).

[26] S. Stute, C. Comtat. Practical considerations for image-based PSF and blobs reconstruction in PET. Phys. Med. Biol., 58, 3849(2013).

[27] A. K. Trull et al. Point spread function based image reconstruction in optical projection tomography. Phys. Med. Biol., 62, 7784(2017).

[28] B. Zhu et al. Image reconstruction by domain-transform manifold learning. Nature, 555, 487-492(2018).

[29] L. Song et al. High-accuracy image formation model for coded aperture snapshot spectral imaging. IEEE Trans. Comput. Imaging, 8, 188-200(2022).

[30] K. C. Lee et al. Design and single-shot fabrication of lensless cameras with arbitrary point spread functions. Optica, 10, 72(2023).

[31] H. J. Zhu, B. C. Han, B. Qiu. Survey of astronomical image processing methods. Lect. Notes Comput. Sci., 9219, 420-429(2015).

[32] J. Smith et al. Image-to-image translation for wavefront and PSF estimation. Proc. SPIE, 12185, 121852L(2022).

[33] S. Ludwig et al. Image reconstruction and enhancement by deconvolution in scatter-plate microscopy. Opt. Express, 27, 23049-23058(2019).

[34] X. Shao et al. Applications of both time of flight and point spread function in brain PET image reconstruction. Nucl. Med. Commun., 37, 422-427(2016).

[35] D. Kidera et al. The edge artifact in the point-spread function-based PET reconstruction at different sphere-to-background ratios of radioactivity. Ann. Nucl. Med., 30, 97-103(2016).

[36] X. Xie et al. Extended depth-resolved imaging through a thin scattering medium with PSF manipulation. Sci. Rep., 8, 4585(2018).

[37] C. Pei et al. Dynamic non-line-of-sight imaging system based on the optimization of point spread functions. Opt. Express, 29, 32349-32364(2021).

[38] L. B. Wolff, S. K. Nayar, M. Oren. Improved diffuse reflection models for computer vision. Int. J. Comput. Vis., 30, 55-71(1998).

[39] C. Tian et al. Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multimedia, 23, 1489-1502(2020).

[40] J. Yamanaka, S. Kuwashima, T. Kurita. Fast and accurate image super resolution by deep CNN with skip connection and network in network. Lect. Notes Comput. Sci., 10635, 217-225(2017).

[41] Z. Wang et al. Multi-memory convolutional neural network for video super-resolution. IEEE Trans. Image Process., 28, 2530-2544(2018).

[42] R. Kuschmierz et al. Ultra-thin 3D lensless fiber endoscopy using diffractive optical elements and deep neural networks. Light Adv. Manuf., 2, 415-424(2021).

[43] H. Zhu et al. DNF: diffractive neural field for lensless microscopic imaging. Opt. Express, 30, 18168-18178(2022).

[44] Q. Li et al. Imaging reconstruction through strongly scattering media by using convolutional neural networks. Opt. Commun., 477, 126341(2020).

[45] Y. Sun et al. Image reconstruction through dynamic scattering media based on deep learning. Opt. Express, 27, 16032-16046(2019).

[46] P. Netrapalli, P. Jain, S. Sanghavi. Phase retrieval using alternating minimization, 2796-2804(2013).

[47] E. J. Candes, X. Li, M. Soltanolkotabi. Phase retrieval via Wirtinger flow: theory and algorithms. IEEE Trans. Inf. Theory, 61, 1985-2007(2015).

Tools

Get Citation

Copy Citation Text

Su Wu, Chan Huang, Jing Lin, Tao Wang, Shanshan Zheng, Haisheng Feng, Lei Yu. Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction[J]. Advanced Photonics Nexus, 2024, 3(2): 026010

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Research Articles

Received: Nov. 8, 2023

Accepted: Feb. 27, 2024

Published Online: Mar. 19, 2024

The Author Email: Yu Lei (yulei@aiofm.ac.cn)

DOI:10.1117/1.APN.3.2.026010

Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology