Infrared and Laser Engineering, Volume. 51, Issue 8, 20220563(2022)

Scattering imaging with deep learning: Physical and data joint modeling optimization (invited)

Enlai Guo, Yingjie Shi, Shuo Zhu, Qianqian Cheng, Yi Wei, Jinye Miao, and Jing Han*
Author Affiliations
  • Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense Laboratory, Nanjing University of Science and Technology, Nanjing 210094, China
  • show less
    References(60)

    [1] X Zhan, J Gao, Y Gan, et al. Roles of scattered and ballistic photons in imaging through scattering media: A deep learning-based study. arXiv, 2207.10263(2022).

    [2] S M Popoff, G Lerosey, R Carminati, et al. Measuring the transmission matrix in optics: An approach to the study and control of light propagation in disordered media. Physical Review Letters, 104, 100601(2010).

    [3] Y Li, Y Xue, L Tian. Deep speckle correlation: A deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

    [4] M W Matthès, Y Bromberg, Rosny J de, et al. Learning and avoiding disorder in multimode fibers. Physical Review X, 11, 021060(2021).

    [5] A d’Arco, F Xia, A Boniface, et al. Physics-based neural network for non-invasive control of coherent light in scattering media. Optics Express, 30, 30845-30856(2022).

    [6] S Zhu, E Guo, J Gu, et al. Imaging through unknown scattering media based on physics-informed learning. Photonics Research, 9, B210-B219(2021).

    [7] S Zhu, E Guo, J Gu, et al. Efficient color imaging through unknown opaque scattering layers via physics-aware learning. Optics Express, 29, 40024-40037(2021).

    [8] M Liao, S Zheng, S Pan, et al. Deep-learning-based ciphertext-only attack on optical double random phase encryption. Opto-Electronic Advances, 4, 05200016(2021).

    [9] O Katz, P Heidmann, M Fink, et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nature Photonics, 8, 784-790(2014).

    [10] I Freund, M Rosenbluh, S Feng. Memory effects in propagation of optical waves through disordered media. Physical Review Letters, 61, 2328-2331(1988).

    [11] S Li, M Deng, J Lee, et al. Imaging through glass diffusers using densely connected convolutional networks. Optica, 5, 803-813(2018).

    [12] M Lyu, H Wang, G Li, et al. Learning-based lensless imaging through optically thick scattering media. Advanced Photonics, 1, 036002(2019).

    [13] N Borhani, E Kakkava, C Moser, et al. Learning to see through multimode fibers. Optica, 5, 960-966(2018).

    [14] B Rahmani, D Loterie, G Konstantinou, et al. Multimode optical fiber transmission with a deep learning network. Light: Science & Applications, 7, 1-11(2018).

    [15] Z Wang, X Jin, Q Dai. Non-invasive imaging through strongly scattering media based on speckle pattern estimation and deconvolution. Scientific Reports, 8, 9088(2018).

    [16] S Cheng, H Li, Y Luo, et al. Artificial intelligence-assisted light control and computational imaging through scattering media. Journal of Innovative Optical Health Sciences, 12, 1930006(2019).

    [17] Y Sun, J Shi, L Sun, et al. Image reconstruction through dynamic scattering media based on deep learning. Optics Express, 27, 16032-16046(2019).

    [18] M Yang, Z H Liu, Z D Cheng, et al. Deep hybrid scattering image learning. Journal of Physics D: Applied Physics, 52, 115105(2019).

    [19] K Yamazaki, R Horisaki, J Tanida. Imaging through scattering media based on semi-supervised learning. Applied Optics, 59, 9850-9854(2020).

    [20] X Lai, Q Li, Z Chen, et al. Reconstructing images of two adjacent objects passing through scattering medium via deep learning. Optics Express, 29, 43280-43291(2021).

    [21] Y Luo, S Yan, H Li, et al. Towards smart optical focusing: Deep learning-empowered dynamic wavefront shaping through nonstationary scattering media. Photonics Research, 9, B262-B278(2021).

    [22] Y Sun, X Wu, Y Zheng, et al. Scalable non-invasive imaging through dynamic scattering media at low photon flux. Optics and Lasers in Engineering, 144, 106641(2021).

    [23] S Zheng, H Wang, S Dong, et al. Incoherent imaging through highly nonstatic and optically thick turbid media based on neural network. Photonics Research, 9, B220-B228(2021).

    [24] Y Shi, E Guo, L Bai, et al. Prior-free imaging unknown target through unknown scattering medium. Optics Express, 30, 17635-17651(2022).

    [25] B Song, C Jin, J Wu, et al. Deep learning image transmission through a multimode fiber based on a small training dataset. Optics Express, 30, 5657-5672(2022).

    [26] W Tahir, H Wang, L Tian. Adaptive 3D descattering with a dynamic synthesis network. Light: Science & Applications, 11, 42(2022).

    [27] A Turpin, I Vishniakou, Seelig J d. Light scattering control in transmission and reflection with neural networks. Optics Express, 26, 30911-30929(2018).

    [28] I Kang, S Pang, Q Zhang, et al. Recurrent neural network reveals transparent objects through scattering media. Optics Express, 29, 5316-5326(2021).

    [29] E Guo, S Zhu, Y Sun, et al. Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect. Optics Express, 28, 2433-2446(2020).

    [30] Q Cheng, L Bai, J Han, et al. Super-resolution imaging through the diffuser in the near-infrared via physically-based learning. Optics and Lasers in Engineering, 159, 107186(2022).

    [31] E Guo, Y Shi, L Bai, et al. Imaging complex targets through a scattering medium based on adaptive encoding. Photonics, 9, 467(2022).

    [32] Q Cheng, E Guo, J Gu, et al. De-noising imaging through diffusers with autocorrelation. Applied Optics, 60, 7686-7695(2021).

    [33] S Zhu, E Guo, Q Cui, et al. Locating and imaging through scattering medium in a large depth. Sensors, 21, 90(2020).

    [34] G Satat, M Tancik, O Gupta, et al. Object classification through scattering media with deep learning on time resolved measurement. Optics Express, 25, 17466-17479(2017).

    [35] P Caramazza, A Boccolini, D Buschek, et al. Neural network identification of people hidden from view with a single-pixel, single-photon detector. Scientific Reports, 8, 11945(2018).

    [36] Q Zhao, H Li, Z Yu, et al. Speckle-based optical cryptosystem and its application for human face recognition via deep learning. arXiv, 2201.11844(2022).

    [37] U Kürüm, P R Wiecha, R French, et al. Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array. Optics Express, 27, 20965-20979(2019).

    [38] E Guo, Y Sun, S Zhu, et al. Single-shot color object reconstruction through scattering medium based on neural network. Optics and Lasers in Engineering, 136, 106310(2021).

    [39] Y Gao, W Xu, Y Chen, et al. Deep learning-based photoacoustic imaging of vascular network through thick porous media. IEEE Transactions on Medical Imaging, 41, 2191-2204(2022).

    [40] Q Li, J Zhao, Y Zhang, et al. Imaging reconstruction through strongly scattering media by using convolutional neural networks. Optics Communications, 477, 126341(2020).

    [41] L Sun, J Shi, X Wu, et al. Photon-limited imaging through scattering medium based on deep learning. Optics Express, 27, 33120-33134(2019).

    [42] X Li, J Shi, X Wu, et al. Photon limited imaging through disordered media: information extraction by exploiting the photon’s quantum nature via deep learning. Applied Physics B, 128, 1-13(2022).

    [43] J Han, J Miao, Y Shi, et al. Photon-limited imaging through scattering medium based on speckle coding. Optik, 255, 168643(2022).

    [44] K Liu, H Zhang, B Zhang, et al. Hybrid optimization algorithm based on neural networks and its application in wavefront shaping. Optics Express, 29, 15517-15527(2021).

    [45] J Zhao, Y Sun, H Zhu, et al. Deep-learning cell imaging through anderson localizing optical fiber. Advanced Photonics, 1, 066001(2019).

    [46] P Fan, T Zhao, L Su. Deep learning the high variability and randomness inside multimode fibers. Optics Express, 27, 20241-20258(2019).

    [47] H Wu, X Meng, X Yang, et al. Single shot real-time high-resolution imaging through dynamic turbid media based on deep learning. Optics and Lasers in Engineering, 149, 106819(2022).

    [48] G Barbastathis, A Ozcan, G Situ. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [49] J Rosen, Aguiar H B de, V Anand, et al. Roadmap on chaos-inspired imaging technologies (CI2-Tech). Applied Physics B, 128, 1-26(2022).

    [50] Y Ba, G Zhao, A Kadambi. Blending diverse physical priors with neural networks. arXiv, 1910.00201(2019).

    [51] H Yu, B Han, L Bai, et al. Untrained deep learning-based fringe projection profilometry. APL Photonics, 7, 016102(2022).

    [52] M R Kellman, E Bostan, N A Repina, et al. Physics-based learned design: optimized coded-illumination for quantitative phase imaging. IEEE Transactions on Computational Imaging, 5, 344-353(2019).

    [53] H Wu, Q Li, X Meng, et al. Cryptographic analysis on an optical random-phase-encoding cryptosystem for complex targets based on physics-informed learning. Optics Express, 29, 33558-33571(2021).

    [54] [54] Stewart R, Ermon S. Labelfree supervision of neural wks with physics domain knowledge[C]ThirtyFirst AAAI Conference on Artificial Intelligence, 2017.

    [55] J Pan, J Dong, Y Liu, et al. Physics-based generative adversarial models for image restoration and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 2449-2462(2020).

    [56] C A Metzler, F Heide, P Rangarajan, et al. Deep-inverse correlography: Towards real-time high-resolution non-line-of-sight imaging. Optica, 7, 63-71(2020).

    [57] F Wang, Y Bian, H Wang, et al. Phase imaging with an untrained neural network. Light: Science & Applications, 9, 1-7(2020).

    [58] K Monakhova, V Tran, G Kuo, et al. Untrained networks for compressive lensless photography. Optics Express, 29, 20913-20929(2021).

    [59] S Diamond, V Sitzmann, F Heide, et al. Unrolled optimization with deep priors. arXiv, 1705.08041(2017).

    [60] [60] Li R, Cheong L F, Tan R T. Heavy rain image restation: Integrating physics model conditional adversarial learning[C]Proceedings of the IEEECVF Conference on Computer Vision Pattern Recognition, 2019: 16331642.

    CLP Journals

    [1] Wei Wang, Xunming Cai, Xin Zhao, Wenbin Ma. High quality and rapid imaging of single-shot optical speckle[J]. Infrared and Laser Engineering, 2023, 52(12): 20230345

    [2] Haofeng Hu, Yizhao Huang, Zhen Zhu, Qianwen Ma, Jingsheng Zhai, Xiaobo Li. Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)[J]. Infrared and Laser Engineering, 2024, 53(3): 20240057

    Tools

    Get Citation

    Copy Citation Text

    Enlai Guo, Yingjie Shi, Shuo Zhu, Qianqian Cheng, Yi Wei, Jinye Miao, Jing Han. Scattering imaging with deep learning: Physical and data joint modeling optimization (invited)[J]. Infrared and Laser Engineering, 2022, 51(8): 20220563

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Special issue——Scattering imaging and non-line-of-sight imaging

    Received: Aug. 10, 2022

    Accepted: --

    Published Online: Jan. 9, 2023

    The Author Email: Han Jing (eohj@njust.edu.cn)

    DOI:10.3788/IRLA20220563

    Topics