Infrared and Laser Engineering, Volume. 51, Issue 8, 20220563(2022)

Scattering imaging with deep learning: Physical and data joint modeling optimization (invited)

Enlai Guo, Yingjie Shi, Shuo Zhu, Qianqian Cheng, Yi Wei, Jinye Miao, and Jing Han*
Author Affiliations
  • Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense Laboratory, Nanjing University of Science and Technology, Nanjing 210094, China
  • show less
    References(60)

    [1] Zhan X, Gao J, Gan Y, et al. Roles of scattered and ballistic photons in imaging through scattering media: A deep learning-based study[J]. arXiv, 2207.10263(2022).

    [2] Popoff S M, Lerosey G, Carminati R, et al. Measuring the transmission matrix in optics: An approach to the study and control of light propagation in disordered media[J]. Physical Review Letters, 104, 100601(2010).

    [3] Li Y, Xue Y, Tian L. Deep speckle correlation: A deep learning approach toward scalable imaging through scattering media[J]. Optica, 5, 1181-1190(2018).

    [4] Matthès M W, Bromberg Y, de Rosny J, et al. Learning and avoiding disorder in multimode fibers[J]. Physical Review X, 11, 021060(2021).

    [5] d’Arco A, Xia F, Boniface A, et al. Physics-based neural network for non-invasive control of coherent light in scattering media[J]. Optics Express, 30, 30845-30856(2022).

    [6] Zhu S, Guo E, Gu J, et al. Imaging through unknown scattering media based on physics-informed learning[J]. Photonics Research, 9, B210-B219(2021).

    [7] Zhu S, Guo E, Gu J, et al. Efficient color imaging through unknown opaque scattering layers via physics-aware learning[J]. Optics Express, 29, 40024-40037(2021).

    [8] Liao M, Zheng S, Pan S, et al. Deep-learning-based ciphertext-only attack on optical double random phase encryption[J]. Opto-Electronic Advances, 4, 05200016(2021).

    [9] Katz O, Heidmann P, Fink M, et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations[J]. Nature Photonics, 8, 784-790(2014).

    [10] Freund I, Rosenbluh M, Feng S. Memory effects in propagation of optical waves through disordered media[J]. Physical Review Letters, 61, 2328-2331(1988).

    [11] Li S, Deng M, Lee J, et al. Imaging through glass diffusers using densely connected convolutional networks[J]. Optica, 5, 803-813(2018).

    [12] Lyu M, Wang H, Li G, et al. Learning-based lensless imaging through optically thick scattering media[J]. Advanced Photonics, 1, 036002(2019).

    [13] Borhani N, Kakkava E, Moser C, et al. Learning to see through multimode fibers[J]. Optica, 5, 960-966(2018).

    [14] Rahmani B, Loterie D, Konstantinou G, et al. Multimode optical fiber transmission with a deep learning network[J]. Light: Science & Applications, 7, 1-11(2018).

    [15] Wang Z, Jin X, Dai Q. Non-invasive imaging through strongly scattering media based on speckle pattern estimation and deconvolution[J]. Scientific Reports, 8, 9088(2018).

    [16] Cheng S, Li H, Luo Y, et al. Artificial intelligence-assisted light control and computational imaging through scattering media[J]. Journal of Innovative Optical Health Sciences, 12, 1930006(2019).

    [17] Sun Y, Shi J, Sun L, et al. Image reconstruction through dynamic scattering media based on deep learning[J]. Optics Express, 27, 16032-16046(2019).

    [18] Yang M, Liu Z H, Cheng Z D, et al. Deep hybrid scattering image learning[J]. Journal of Physics D: Applied Physics, 52, 115105(2019).

    [19] Yamazaki K, Horisaki R, Tanida J. Imaging through scattering media based on semi-supervised learning[J]. Applied Optics, 59, 9850-9854(2020).

    [20] Lai X, Li Q, Chen Z, et al. Reconstructing images of two adjacent objects passing through scattering medium via deep learning[J]. Optics Express, 29, 43280-43291(2021).

    [21] Luo Y, Yan S, Li H, et al. Towards smart optical focusing: Deep learning-empowered dynamic wavefront shaping through nonstationary scattering media[J]. Photonics Research, 9, B262-B278(2021).

    [22] Sun Y, Wu X, Zheng Y, et al. Scalable non-invasive imaging through dynamic scattering media at low photon flux[J]. Optics and Lasers in Engineering, 144, 106641(2021).

    [23] Zheng S, Wang H, Dong S, et al. Incoherent imaging through highly nonstatic and optically thick turbid media based on neural network[J]. Photonics Research, 9, B220-B228(2021).

    [24] Shi Y, Guo E, Bai L, et al. Prior-free imaging unknown target through unknown scattering medium[J]. Optics Express, 30, 17635-17651(2022).

    [25] Song B, Jin C, Wu J, et al. Deep learning image transmission through a multimode fiber based on a small training dataset[J]. Optics Express, 30, 5657-5672(2022).

    [26] Tahir W, Wang H, Tian L. Adaptive 3D descattering with a dynamic synthesis network[J]. Light: Science & Applications, 11, 42(2022).

    [27] Turpin A, Vishniakou I, d Seelig J. Light scattering control in transmission and reflection with neural networks[J]. Optics Express, 26, 30911-30929(2018).

    [28] Kang I, Pang S, Zhang Q, et al. Recurrent neural network reveals transparent objects through scattering media[J]. Optics Express, 29, 5316-5326(2021).

    [29] Guo E, Zhu S, Sun Y, et al. Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect[J]. Optics Express, 28, 2433-2446(2020).

    [30] Cheng Q, Bai L, Han J, et al. Super-resolution imaging through the diffuser in the near-infrared via physically-based learning[J]. Optics and Lasers in Engineering, 159, 107186(2022).

    [31] Guo E, Shi Y, Bai L, et al. Imaging complex targets through a scattering medium based on adaptive encoding[J]. Photonics, 9, 467(2022).

    [32] Cheng Q, Guo E, Gu J, et al. De-noising imaging through diffusers with autocorrelation[J]. Applied Optics, 60, 7686-7695(2021).

    [33] Zhu S, Guo E, Cui Q, et al. Locating and imaging through scattering medium in a large depth[J]. Sensors, 21, 90(2020).

    [34] Satat G, Tancik M, Gupta O, et al. Object classification through scattering media with deep learning on time resolved measurement[J]. Optics Express, 25, 17466-17479(2017).

    [35] Caramazza P, Boccolini A, Buschek D, et al. Neural network identification of people hidden from view with a single-pixel, single-photon detector[J]. Scientific Reports, 8, 11945(2018).

    [36] Zhao Q, Li H, Yu Z, et al. Speckle-based optical cryptosystem and its application for human face recognition via deep learning[J]. arXiv, 2201.11844(2022).

    [37] Kürüm U, Wiecha P R, French R, et al. Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array[J]. Optics Express, 27, 20965-20979(2019).

    [38] Guo E, Sun Y, Zhu S, et al. Single-shot color object reconstruction through scattering medium based on neural network[J]. Optics and Lasers in Engineering, 136, 106310(2021).

    [39] Gao Y, Xu W, Chen Y, et al. Deep learning-based photoacoustic imaging of vascular network through thick porous media[J]. IEEE Transactions on Medical Imaging, 41, 2191-2204(2022).

    [40] Li Q, Zhao J, Zhang Y, et al. Imaging reconstruction through strongly scattering media by using convolutional neural networks[J]. Optics Communications, 477, 126341(2020).

    [41] Sun L, Shi J, Wu X, et al. Photon-limited imaging through scattering medium based on deep learning[J]. Optics Express, 27, 33120-33134(2019).

    [42] Li X, Shi J, Wu X, et al. Photon limited imaging through disordered media: information extraction by exploiting the photon’s quantum nature via deep learning[J]. Applied Physics B, 128, 1-13(2022).

    [43] Han J, Miao J, Shi Y, et al. Photon-limited imaging through scattering medium based on speckle coding[J]. Optik, 255, 168643(2022).

    [44] Liu K, Zhang H, Zhang B, et al. Hybrid optimization algorithm based on neural networks and its application in wavefront shaping[J]. Optics Express, 29, 15517-15527(2021).

    [45] Zhao J, Sun Y, Zhu H, et al. Deep-learning cell imaging through anderson localizing optical fiber[J]. Advanced Photonics, 1, 066001(2019).

    [46] Fan P, Zhao T, Su L. Deep learning the high variability and randomness inside multimode fibers[J]. Optics Express, 27, 20241-20258(2019).

    [47] Wu H, Meng X, Yang X, et al. Single shot real-time high-resolution imaging through dynamic turbid media based on deep learning[J]. Optics and Lasers in Engineering, 149, 106819(2022).

    [48] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging[J]. Optica, 6, 921-943(2019).

    [49] Rosen J, de Aguiar H B, Anand V, et al. Roadmap on chaos-inspired imaging technologies (CI2-Tech)[J]. Applied Physics B, 128, 1-26(2022).

    [50] Ba Y, Zhao G, Kadambi A. Blending diverse physical priors with neural networks[J]. arXiv, 1910.00201(2019).

    [51] Yu H, Han B, Bai L, et al. Untrained deep learning-based fringe projection profilometry[J]. APL Photonics, 7, 016102(2022).

    [52] Kellman M R, Bostan E, Repina N A, et al. Physics-based learned design: optimized coded-illumination for quantitative phase imaging[J]. IEEE Transactions on Computational Imaging, 5, 344-353(2019).

    [53] Wu H, Li Q, Meng X, et al. Cryptographic analysis on an optical random-phase-encoding cryptosystem for complex targets based on physics-informed learning[J]. Optics Express, 29, 33558-33571(2021).

    [54] [54] Stewart R, Ermon S. Labelfree supervision of neural wks with physics domain knowledge[C]ThirtyFirst AAAI Conference on Artificial Intelligence, 2017.

    [55] Pan J, Dong J, Liu Y, et al. Physics-based generative adversarial models for image restoration and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 2449-2462(2020).

    [56] Metzler C A, Heide F, Rangarajan P, et al. Deep-inverse correlography: Towards real-time high-resolution non-line-of-sight imaging[J]. Optica, 7, 63-71(2020).

    [57] Wang F, Bian Y, Wang H, et al. Phase imaging with an untrained neural network[J]. Light: Science & Applications, 9, 1-7(2020).

    [58] Monakhova K, Tran V, Kuo G, et al. Untrained networks for compressive lensless photography[J]. Optics Express, 29, 20913-20929(2021).

    [59] Diamond S, Sitzmann V, Heide F, et al. Unrolled optimization with deep priors[J]. arXiv, 1705.08041(2017).

    [60] [60] Li R, Cheong L F, Tan R T. Heavy rain image restation: Integrating physics model conditional adversarial learning[C]Proceedings of the IEEECVF Conference on Computer Vision Pattern Recognition, 2019: 16331642.

    CLP Journals

    [1] Wei Wang, Xunming Cai, Xin Zhao, Wenbin Ma. High quality and rapid imaging of single-shot optical speckle[J]. Infrared and Laser Engineering, 2023, 52(12): 20230345

    [2] Haofeng Hu, Yizhao Huang, Zhen Zhu, Qianwen Ma, Jingsheng Zhai, Xiaobo Li. Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)[J]. Infrared and Laser Engineering, 2024, 53(3): 20240057

    Tools

    Get Citation

    Copy Citation Text

    Enlai Guo, Yingjie Shi, Shuo Zhu, Qianqian Cheng, Yi Wei, Jinye Miao, Jing Han. Scattering imaging with deep learning: Physical and data joint modeling optimization (invited)[J]. Infrared and Laser Engineering, 2022, 51(8): 20220563

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Special issue——Scattering imaging and non-line-of-sight imaging

    Received: Aug. 10, 2022

    Accepted: --

    Published Online: Jan. 9, 2023

    The Author Email: Jing Han (eohj@njust.edu.cn)

    DOI:10.3788/IRLA20220563

    Topics