Various scattering media, such as cloud, haze, turbid solution, biological soft tissue, frosted glass, are widely present in nature and daily life. Seeing into and through scattering media has always been a hotspot in research. However, light field propagation in the medium is extremely complex and cannot be accurately described. Despite of the development of a variety of techniques, such as wavefront shaping, optical phase conjugation, scattering matrix measurement, speckle autocorrelation and deconvolution, it still remains at the stage of proof-of-concept with thin scattering media in laboratory.

With the rapid development of big data and AI, deep learning has been introduced into imaging through scattering media. It has attracted widespread attention due to no requirement on physical modeling, simple data processing, improved imaging thickness, and better tolerance to dynamic states. That said, learning-based research is usually case by case demonstration, which can hardly be generalized to common scenarios. Also, since the physical model is unknown, the reason for the prediction failure is unknown; even if it is successful, there is a lack of confidence in the reliability of the results, which seriously hampers its wide applications.

Exploring the law of optical field propagation in scattering media, the principle of information transmission, and the physical mechanism and boundary of different imaging methods have always been the focus of the team led by Dr. Honglin Liu, Shanghai Institute of Optics and Fine Mechanics (SIOM), Chinese Academy of Sciences. Considering that deep learning has no physical model, through collaboration with Prof. Puxiang Lai at the Hong Kong Polytechnic University, the team designed a scheme to study the relationship between image prediction ability and sampling position under adjustable proportion of ballistic light.

It has been revealed that the deep learning actually extracts target information from ballistic and scattered light simultaneously, due to the different sensitivity of ballistic and scattered light distributions to media position changes: the generalization of the network is improved by enhancing the utilization of ballistic light but worsened by increasing the weight of scattered light. Moreover, the network can only recognize known scattering media, and an increased number of media will also deteriorate the prediction quality. In the study, a physical model of deep learning for imaging through scattering media is established, and the physical origin of network generalization and physical boundary of practical application are clarified. Relevant research results were recently published in Photonics Research, Volume 11, No. 6, 2023 [Xuyu Zhang, Shengfu Cheng, Jingjing Gao, Yu Gan, Chunyuan Song, Dawei Zhang, Songlin Zhuang, Shensheng Han, Puxiang Lai, Honglin Liu. Physical origin and boundary of scalable imaging through scattering media: a deep learning-based exploration[J]. Photonics Research, 2023, 11(6): 1038].

As shown in Fig. 1(a), a target is loaded on a digital micromirror device (DMD). Light reflected from the DMD is scattered by a diffuser and then collected by a camera. Two different strategies are used for data collection: first, training data is collected at a single location, and data at the original position and positions with different shifts is recorded for testing; second, data from multiple locations is collected to constitute a training dataset, and the network is tested by data at multiple original locations and locations with different shifts. The results show that with ballistic photons, training with data from multiple locations can improve the generalization capability of the network, while without ballistic photons, the network has no generalization regardless of the strategy. The ballistic composition does not change with the position of the diffuser, which is the origin of generalization. The coding of scattering components at different locations has different features, and the training with a large amount of data enable the network to recognize specific features.

Fig.1 (a) Schematic of experimental setup. (b, c) Prediction results at different locations with and without ballistic photons. (d) Trend curves of image quality with position shift w/o ballistic photons.

At present, the understanding of the physical mechanism of imaging through scattering media is still very superficial. The majority of methods in the field are based on approximations under strong constraints. For example, in optical phase conjugation (OPC), a spherical wave emitted by a point source transmits through a scattering medium. Even under paraxial condition, the lengths of different paths are different, the exit wavefront is not in a plane, so propagation of a generated conjugate wavefront in a plane is not an ideal reverse process. As the medium thickness increases, the path length difference increases dramatically, and the wavefront distortion aggravates till no more conjugation remains. Under the condition that path length differences, medium absorption, and polarization changes are negligible, the conjugated wave can return to the origin source position to achieve focusing. Physically, OPC cannot compensate for scattering effect of thick media for focusing and imaging. In addition, in many proof-of-concept demos of various methods, residual ballistic photons have been used, although maybe unconsciously. As the medium thickness increases, not only the proportion of ballistic photons decreases, but also the position sensitivity of the scattered light increases due to thermal motions and various perturbations difficult to shield, resulting in elevated difficulty in detecting and using the scattered light. The results of this study not only establish a physical model of deep learning in imaging through scattering media, but also reshape the understanding of physical mechanisms of other imaging methods, which could address the fundamental reason why the bottleneck of medium thickness has not yet been broken through.

In the next step, the team will quantify the efficiency of image reconstruction from scattered light under various coding features, optimize the model, and expand the scenarios to the difference in utilization of ballistic and scattering components for lens and lensless systems under incoherent illumination. We will also investigate new mechanisms and methods for the detection and utilization of dynamic speckle fields, laying a foundation for going beyond the thickness limits for imaging through scattering media.