Optical sensing is fundamental for a great number of applications. Most of conventional optical sensors rely on direct interception and capture of photons. They suffer from intrinsic limitations, including the damage of the device under intense light irradiation. The information collected is also limited as one sensor is mostly designed to characterize one specific optical parameter. Inverse sensing by scattered photons has been invented to overcome these limitations. Scattered photons contain abundant information, including the morphology of the scattering centers as well as the wavelength, intensity and direction of the incident light^1-5. However, previous study of inverse sensing proves to be extremely challenging due to the complicated and nonlinear nature of the scattered photons^6-13. If the scattered photons can be effectively analyzed, it can overcome many limitations in optical sensing. Instead of relying on the light-matter interaction and physically capturing photons, a new sensing concept collects much information from the spatial distribution of the scattering photons. The detection unit is no longer fully restricted by the material properties which are used to make the optical sensor^14-16.

How to efficiently analyze the information carried by scattered photons is the key challenge. Previous scientific researches aim to derive a general solution to the inverse sensing problems^{13, 17, 18}. Due to the huge computational resources required, such a general solution is beyond reach for most of the practical situations. Furthermore, conventional computational methods are also not effective for scattered photon analysis. Commonly used simulation platforms include: a) Finite-Difference Time-Domain method¹⁹ (Lumerical), b) transmission line matrix based on either the time domain or finite integration technique²⁰ (CST, SIMULIA), c) cross-platform finite element analysis based on partial differential equations²¹ (COMSOL Multiphysics), and d) ray-tracing model geometrical optical solver²² (Zemax). All these methods cannot handle inverse sensing problems. Light source is always required to be pre-defined. Hence these computational methods are not suitable for the inverse analysis of the scattered photons.

In this paper, a more efficient solution is provided. It is proposed to study inverse sensing problems from a different perspective. Instead of aiming at a general solution, it is more efficient and straightforward to reconstruct scattered photons in a specific and well-understood environment. Our solution is based on convolutional neural networks (CNN) deep learning (DL)^23-28. The foundation of this method is the strong capability of DL to inversely analyze complicated spatial light distribution in a given environment. Based on it, a multifunctional "inverse sensing" method is demonstrated, which is as efficient as its direct sensing counterpart. Compared to a general solution, our solution is also useful for a large number of circumstances. Most optical sensors are designed to function in a specific environment, installed at a fixed location, and run for a pre-set purpose. Examples are data reader for blue-ray disks, surveillance cameras, optical sensors in communication networks, and detectors for various industrial monitoring purposes.

Specifically, in the first part, the fundamental physics and DL toolkit are studied to efficiently analyze the scattered photons' profiles, which overcome various challenges for direct sensing. Its capability and potentials are demonstrated to inversely detect multiple optical parameters, such as direction and intensity. In the second part, key features, advantages, and applications are discussed. Firstly, the function and capability are dynamic. Compared to the conventional optical sensors with a fixed function, this DL based method can upgrade after continuous data training. Secondly, multi-functional detection is achieved in complicated situations. Multiple parameters can be monitored simultaneously, even if the light signal overlaps with each other. Thirdly, this method removes many limitations for conventional optical sensors' implementation. This inverse sensing method is non-contact, which provides accurate remote measurement even if the path of the optical signal is further away from the sensor. Fourthly, without the direct exposure to the intensive light signal, the non-contact feature makes the device hardly get saturated or damaged. It is demonstrated in the later section that this method provides a nearly unlimited measurement range, which is only restricted by the database to train the DL model. The same method can detect the bio-dyes fluorescence with 0.42 nW continuous emissions and laser pulse at a peak power up to 7.2 GW (pulse energy/pulse duration). Fifthly, as the foundation of such analysis is the DL^{26, 28}, it is less affected by the environment, which offers strong capability to select signal out of noise.

Scattered photons are abundant in most optical applications. There are many sources in the environment. One primary source for most of the optical devices is the scattering centers in air, in which light is scattered by molecules of various sizes (air molecules and dust as demonstrated in Fig. 1(a). Beyond the scattering centers in the light path, the reflective interfaces also contribute to light scattering (Fig. 1(b)). The properties of scattered photons depend on many factors including both the light source and environment. Hence, scattered photons provide unique opportunities for optical detection. However, to directly capture the information proves to be challenging. Scattered photons mostly come from multiple sources. Characterization by simply measuring the intensity at selected points has been applied in some systems, for example, dynamic light scattering (DLS) which is used to determine the size distribution profile of small particles in suspension. However, the amount of information collected is highly limited. Alternatively, a more comprehensive collection is available by studying the spatial distribution of the scattered photons in the environment (Figs. 1(a) and 1(b)). Wavelength, incident direction, and intensity of the light are some of the key parameters that decide the scattered photon profile. Thus, it is possible to characterize these parameters by analyzing the profile of the scattered photon (Fig. 1(c)). The analysis of the scattered photon is an inverse sensing problem. The light source is unknown, which requires a novel inverse analysis approach.

We apply the CNN DL to develop our method, which belongs to the supervised learning paradigms^{26, 28-31}. This method is dedicated to deriving the CNN function between a complicated input and a simple output (Fig. 2). Such a function is governed by a set of parameters. The proper set of parameters is decided by epochs of "training" processes, in which a pair of training data, composed of the input with certain features and its corresponding output value, is analyzed to modify the parameters. The modification of the parameters is based on CNN with certain optimization approaches to process a large quantity of data. The "training" process repeats until the set of the parameters can give accurate prediction between the input and output. For our application scope, we design a specific DL workflow modified with the pre-imaging process for the optimization of optical applications, which is described in the supplementary materials. The training data is the optical image of the scattered photon profile and the corresponding physical parameters of interests. The softmax function is selected as the activation function to output the probability distribution over different classes:

1 $ f\left( {{x_i}} \right) = \frac{{{\rm{exp}}\left( {{x_i}} \right)}}{{\mathop \sum \nolimits_{j = 1}^K \;{\rm{exp}}({x_j})}}, $

where f denotes the softmax output, which is the probability that predicted class i; x is the input value for the softmax function; K is the number of the classes for identification. Cross entropy loss is selected as the loss function, which is the function that optimization function needs to minimize through training:

2 $L(x) = - \mathop \sum \limits_{j = 1}^K {y_j}{\rm{log}}({s_j}), $

where s_j is the predicted probability of class j and y_j is the ground truth of class j. The final function is recorded and tested on independent samples to verify the accuracy. In all the experiments, the testing set is independent from the training set. This approach can go beyond the limitations of the conventional optical sensors. By analyzing the spatial distribution of photons, it transforms a widely-used complementary metal-oxide-semiconductor imaging device (CMOS) into a multi-functional sensor that can measure multiple light properties as well as the incident direction (Figs. 1(d) and 1(e)). Although the initial training process may take hours for our case, once the final function is found, it takes an average of 0.05 second to process the scattered photon profile and determine its physical value.

Next, how this method can be applied in inverse sensing is demonstrated. Specifically, to measure the incident direction of the laser beam (Fig. 1(d)), the CMOS is fixed to maintain a static field of view. We then vary the incident angle of the laser beam, and the profile of the scattered photon is changed correspondingly. At every incident angle, 25 images are taken, which make up the training data set for that incident angle. The entire training data set consists of all the data at different incident angles. Our inverse sensing method is applied to analyze the entire training data set. To demonstrate its capability, the range of the incident angle is set to rotate in the horizontal and vertical directions. The experimental conditions and accuracy after each training epoch are described with more details summarized in the supplementary material. The accuracy of the DL approach can reach ~100% for this experimental condition, which means that our method can achieve the same accuracy as the instruments used to collect the dataset. In addition, it is able to detect small variation of the incident angle down to 1 degree. This accurate detection can be applied to passive laser radar mechanisms to detect the location of the signal source. As the scattered photon analysis does not have to intercept the light beam, it no longer requires to directly capture the incident light and greatly extends the detection range by scattered photon, which is useful for various applications based on light detections, including self-driving cars, and industrial monitoring.

This method can also evolve and upgrade its functions by learning other data of interests. It is important to measure the intensity of the laser for many applications. Our inverse sensing method is applied to measure the training data of intensity. The collection of the training data set is similar to the previous experiments, except that the incident angle is kept constant and the intensity is tuned. In our experiment, the incident light source is a 532 nm continuous wave (CW) laser. The incident power increases from 0 to 0.15 W with the increment of 0.01 W. The accuracy of the experimental results is presented in Fig. 1(e). The peak value of 0.99 indicates that the parameters set in epoch 8 generate a function with 99% prediction accuracy. It should be noted that the measurement range and resolution depend on the training data. Theoretically, there is no limitation on the measurement range or resolution and this method can be as accurate as the best scientific instruments used for the calibration and collection of the training dataset. In addition, due to the large quantity of information contained in the scattered photon profile, it can even go beyond the capability of the conventional scientific instruments.

With our approach, the scattered photon transfers from noises to resources of useful information, which brings us many unique advantages. Compared to conventional methods which only measure signals in the straight light path, our method can extend the vision beyond the sight of view. To verify this concept, we design the experiment to collect the training data set which contains only the scattered photon without any incident light beam in the image (Fig. 3(a)). In this situation, the characterization relies on the scattered photon reflected by the surrounding objects. Our inverse sensing method is applied and the results in terms of accuracy after training are presented in Fig. 3(b). In our experiments, the accuracy can be as high as 99%, indicating a reliable measurement for these "out-of-sight" situations. Furthermore, for conventional detectors, the optical signals are treated as scalar values. The amplitudes of multiple signals are added together. The inverse sensing method is based on the spatial distribution of photons. It is more similar to a vector, which also contains the direction information of the light. This advantage makes it possible to distinguish multiple signals and monitor them simultaneously (Figs. 3(c) and 3(d)). In this experiment, two light beams present in the same field of view. While there is overlapping of the scattered photon at certain locations, there are also areas that are dominated by the scattered photon from one laser beam. This case is challenging for most convention optical sensors such as the power meters as the second laser beam contributes noises. Our method can successfully distinguish two laser beams and the intensity measurement can be as accurate as 100% in our experimental conditions. While we only demonstrate the simultaneous monitoring of two laser beams with one single sensor, this approach can be further developed to monitor a larger number of independent remote sensing signals, which shows great potentials for the integrated photonics and optical information devices.

Furthermore, as this approach is not required to intercept the light beam in the optical path, remote monitoring is also possible (Fig. 3(e)). In this experiment, the laser beam has the diameter of 2.25 mm. We place the CMOS 5 meters away from the laser beam. The laser parameters are the same as those in the experiment of Fig. 1(e) and the training data set is collected to measure the laser intensity from 0 to 0.15 W. In spite of the distance between the CMOS and the laser signal, the accuracy of measurement can still reach 97%, which is similar to the previous experimental results. These results demonstrate its capability to conduct remote characterization, showing the potentials for large area monitoring and high energy non-contact applications for the space technology. For all these applications, the stability of our methods needs to be verified. Therefore, we repeat the measurement of the intensity in the room-light illumination condition. As DL based mechanism is well-known for its strong capability to select the critical information in an image³², our method can eliminate the noises from other light sources and detect the intensity of the laser beam with an accuracy of 99%, which is the same as the experimental results previously. The high noise-resistance is superior to the conventional approaches and is a widely demanded feature in our optical sensing, as discussed in the following sections.

This inverse sensing method can be applied to highly challenging applications. The first application aims to achieve detection that "goes through the wall". Illustrated by Figs. 4(a) to 4(c), one object, a horse, is completely hidden behind a bulk obstruction that prevents the detection by conventional means. With this inverse sensing method, one laser beam is irradiated near the obstruction and its scattered photon illuminates the hidden horse. By detecting and analyzing its scattered photon profile in the environment, it is able to characterize the property of the hidden object. To better study its capability, 4 different types of objects are selected and put behind the obstruction. Those objects are of different shapes, dimensions, and reflective properties. This inverse sensing can successfully detect and identify every object. The accuracy can reach 100% in our experimental conditions (The accuracy after every training epoch is presented in the supplementary materials). Compared to other methods to achieve similar purposes, our approach is "automatic", requires much less computational powers, and is not limited to certain environments^{3, 33}. Due to the impressive learning ability of the DL methods, this approach has potentials to more general situations with more intensive training. With the similar development strategy for the self-driving car to build the database, it is highly promising to launch a completely new sensing method that provides new vision without any blind area.

For the second application, we show two examples of applying this inverse sensing method to measure the light signals of different energy levels. Firstly, this inverse sensing method is applied to detect the intensity of the laser pulse of one femtosecond pulsed laser (maximum average power 3.6 W, 100 fs, 800 nm, and 1000 Hz repetition rate). In this experiment, when the output power is set as 20% of the max power, the peak power of one single pulse can reach 7.2 GW, which can damage most materials. Our inverse sensing is applied to study the intensity at the focal spot in the process of laser ablation of steel. The scattered photon profiles at different laser intensities make the training sets for the DL training. More details about the experimental procedure can be found in the supplementary materials. After the training, the accuracy of the intensity characterization goes up to 100% in our experimental conditions (Fig. 4(d)). With the training results, we can dynamically monitor the average power of the laser pulse at every second in the fs laser micro-fabrication. The results are presented in Fig. 4(e), which show the increase of the peak power from zero to GW level. For every data point, it only takes ~0.05 seconds for the DL analysis. As the entire method is based on the profile of scattered photon, it does not saturate even at high laser intensity. Furthermore, the detector will not get damaged by the laser pulse. We achieve the dynamic monitoring at the focal point of one femtosecond laser ablation (Fig. 4(e)). It can provide rich insights on the in-situ light matter interaction and there are huge potentials for other high energy applications, such as laser weapons and advanced manufacturing.

Secondly, this inverse sensing method is also applied to measure extremely weak signal of the fluorescence from bio-dyes as presented in Figs. 4(f) to 4(h). Rhodamine B molecules are uniformly spreaded on a glass substrate. Such samples are patterned with a femtosecond laser. The intensity of the laser is carefully controlled so that it is sufficient to ablate the Rhodamine B molecules but lower than the damage threshold of the glass substrate. After the fabrication, Rhodamine B islands are created which have similar scales as the cells of living organisms (Fig. 4(h), details can be found in the supplementary materials). When we pump the sample with a 532 nm CW laser, the CMOS on the microscope captures the image of the fluorescence emission. The DL is applied to build the function of its emission intensity with respect to different pumping conditions. After the training, it is used to detect the emission intensity of the Rhodamine B island (Figs. 4(f)-4(h), the details of the training process and its results are presented in the supplementary materials). The emission intensities of every Rhodamine B islands in the picture can be successfully detected by the DL. It should be noted that the emission of the Rhodamine B islands is as low as 0.42 nW. The characterization of the intensity can be useful for the cell labeling as well as molecule tracing³⁴. With the same pumping light intensity, this light profile can also reveal the concentration of the dye molecules by comparing its emission light intensity, which is helpful to provide a new source of information in the fluorescence-based microscope for biological applications³⁵.

In summary, we studied the inverse sensing method based on the CNN DL for optical detection in a specific environment. The accuracy can go up to 100% in our experimental conditions, which means this method is as accurate as the instruments to collect the dataset. This new concept opens a gate to a completely new design strategy in many fields including advanced manufacturing³⁶, data security³⁷, self-driving car sensing³⁸, space technology, and biological characterization³⁹. The ability to characterize the light beam remotely offers a fast and automated way to monitor high power laser and decode optical signals without an interception, which are very useful for in situ ultra-fast pulsed laser research and decryption of optical signals in an undetected manner. Furthermore, the reconstruction of the light beam leads to passive laser detection systems, which can reduce the blind area and extend the observation "out-of-view". As this methodology can be flexibly integrated without adding any components into the light path, its compact feature is also much favorable for future applications, including the integrated photonics^40-42.

We thank H. Gao, H. Y. Yu, and Y. Zhou for the helpful discussion.

L. W. Chen and Y. M. Yin contributed equally to this work. L. W. Chen, Y. M. Yin and Y. Li conceived the design, and carried out the design and simulation. Y. M. Yin wrote the code. L. W. Chen, Y. M. Yin and Y. Li conceived and performed the measurements. L. W. Chen, and M. H. Hong discussed the results and co-wrote the paper. All authors commented on the manuscript.

The authors declare no competing financial interests.