Passive non-line-of-sight imaging for moving targets with an event camera

Conghe Wang; Yutong He; Xia Wang; Honghao Huang; Changda Yan; Xin Zhang; Hongwei Chen

doi:10.3788/COL202321.061103

Chinese Optics Letters, Volume. 21, Issue 6, 061103(2023)

Passive non-line-of-sight imaging for moving targets with an event camera

Conghe Wang¹, Yutong He², Xia Wang^1、*, Honghao Huang², Changda Yan¹, Xin Zhang¹, and Hongwei Chen^2、**

¹Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China

²Beijing National Research Center for Information Science and Technology (BNRist), Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

show less

References(28)

[1] D. Faccio, A. Velten, G. Wetzstein. Non-line-of-sight imaging. Nat. Rev. Phys., 2, 318(2020).

[2] A. Kirmani, T. Hutchison, J. Davis, R. Raskar. Looking around the corner using transient imaging. IEEE 12th International Conference on Computer Vision (ICCV), 159(2009).

[3] T. Maeda, G. Satat, T. Swedish, L. Sinha, R. Raskar. Recent advances in imaging around corners(2019).

[4] M. La Manna, F. Kine, E. Breitbach, J. Jackson, T. Sultan, A. Velten. Error backprojection algorithms for non-line-of-sight imaging. IEEE Trans. Pattern Anal. Mach. Intell., 41, 1615(2019).

[5] M. O’Toole, D. B. Lindell, G. Wetzstein. Confocal non-line-of-sight imaging based on the light-cone transform. Nature, 555, 338(2018).

[6] T. Sasaki, C. Hashemi, J. R. Leger. Passive 3D location estimation of non-line-of-sight objects from a scattered thermal infrared light field. Opt. Express, 29, 43642(2021).

[7] J. Boger-Lombard, O. Katz. Passive optical time-of-flight for non-line-of-sight localization. Nat. Commun., 10, 3343(2019).

[8] C. Pei, A. Zhang, Y. Deng, F. Xu, J. Wu, D. U.-L. Li, H. Qiao, L. Fang, Q. Dai. Dynamic non-line-of-sight imaging system based on the optimization of point spread functions. Opt. Express, 29, 32349(2021).

[9] A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, R. Raskar. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun., 3, 745(2012).

[10] X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, A. Velten. Non-line-of-sight imaging using phasor-field virtual wave optics. Nature, 572, 620(2019).

[11] R. Geng, Y. Hu, Z. Lu, C. Yu, H. Li, H. Zhang, Y. Chen. Passive non-line-of-sight imaging using optimal transport. IEEE Trans. Image Process., 31, 110(2021).

[12] K. Tanaka, Y. Mukaigawa, A. Kadambi. Polarized non-line-of-sight imaging. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2136(2020).

[13] O. Katz, P. Heidmann, M. Fink, S. Gigan. Non-invasive singleshot imaging through scattering layers and around corners via speckle correlations. Nat. Photonics, 8, 784(2014).

[14] M. Tancik, T. Swedish, G. Satat, R. Raskar. Data-driven non-line-of-sight imaging with a traditional camera. Imaging and Applied Optics, IW2B.6(2018).

[15] J. He, S. Wu, R. Wei, Y. Zhang. Non-line-of-sight imaging and tracking of moving objects based on deep learning. Opt. Express, 30, 16758(2022).

[16] C. A. Metzler, F. Heide, P. Rangarajan, M. M. Balaji, A. Viswanath, A. Veeraraghavan, R. G. Baraniuk. Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging. Optica, 7, 63(2020).

[17] C. Zhou, C.-Y. Wang, Z. Liu. Non-line-of-sight imaging off a phong surface through deep learning(2020).

[18] A. Zhang, J. Wu, J. Suo, L. Fang, H. Qiao, D. D.-U. Li, S. Zhang, J. Fan, D. Qi, Q. Dai, C. Pei. Single-shot compressed ultrafast photography based on U-net network. Opt. Express, 28, 39299(2020).

[19] G. Gallego, T. Delbruck, G. M. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis, D. Scaramuzza. Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell., 44, 154(2020).

[20] A. Amir, B. Taba, D. Berg, T. Melano, J. McKinstry, C. Di Nolfo, T. Nayak, A. Andreopoulos, G. Garreau, M. Mendoza, J. Kusnitz, M. Debole, S. Esser, T. Delbruck, M. Flickner, D. Modha. A low power, fully event-based gesture recognition system. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7388(2017).

[21] H. Rebecq, R. Ranftl, V. Koltun, D. Scaramuzza. High speed and high dynamic range video with an event camera. IEEE Trans. Pattern Anal. Mach. Intell., 43, 1964(2021).

[22] S. Schaefer, D. Gehrig, D. Scaramuzza. AEGNN: asynchronous event-based graph neural networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12361(2022).

[23] C. Saunders, J. Murray-Bruce, V. K. Goyal. Computational periscopy with an ordinary digital camera. Nature, 565, 472(2019).

[24] X. Lagorce, G. Orchard, F. Galluppi, B. E. Shi, R. B. Benosman. HOTS: a hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39, 1346(2017).

[25] C. Yan, X. Wang, X. Zhang, X. Li. Adaptive event address map denoising for event cameras. IEEE Sens. J., 22, 3417(2022).

[26] C. Wang, X. Wang, C. Yan, K. Ma. Feature representation and compression methods for event-based data. IEEE Sens. J., 23, 5109(2023).

[27] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 586(2018).

[28] S. Chan, R. E. Warburton, G. Gariepy, J. Leach, D. Faccio. Non-line-of-sight tracking of people at long range. Opt. Express, 25, 10109(2017).

Cited By

[1] Shuo Zhu, Zhou Ge, Chutian Wang, Jing Han, Edmund Y. Lam.