The action of implementing computational photography for targets out of the field of view (FOV) such as behind the barrier by a reflective relay surface is defined as non-line-of-sight (NLOS) imaging. NLOS technology has a promising future in the fields of medicine, road safety, and scientific research due to its ability to expand the human's FOV in scenarios where a direct view is impossible to obtain by devices or human eyes. The present NLOS technology mostly includes transient imaging, range-gated imaging, and passive pattern imaging, and they are mostly dedicated to Lambert reflector relay surfaces. However, the materials in common scenarios are usually non-Lambertian reflectors, whose scattering characteristics are random and diverse, with different degradation characteristics of NLOS imaging results. The traditional methods always have complex system structures, slow imaging speeds, and high costs. We propose a new NLOS method based on the time of flight (TOF) camera, which requires lower maintenance costs, indicates higher availability than the transient imaging and range-gated imaging, and can also realize 3D reconstruction compared with the passive pattern. The NLOS 3D imaging can be achieved for the relay surface with non-Lambertian scattering characteristics by the proposed method to provide theoretical references and an experimental basis for the application.

The NLOS images collected by the reflective relay surface suffer from serious degradation of which the process is directly determined by the optical scattering characteristics of the relay surface. The following theoretical hypothesis is provided by analyzing the propagation method of the target optical signal (Fig. 1). The degraded images are formed from convolution between the clear image and the relay surface with scattering characteristics. As a result, reconstruction for clear images can be realized through the deconvolution of degraded images if scattering characteristics of the relay surface are obtained. Depth imaging can be achieved for the TOF camera by calculating the phase differences between the incoherently modulated illumination and the returned light signal from targets. The phase difference calculation relies on the signal intensity, and the relay surface scatters the optical signal and degrades the depth image. Therefore, the implementation method of the reconstruction algorithm is as follows. First, scattering characteristics of the relay surface are obtained with the genetic algorithm (GA) through clear and degraded images respectively of the given target. Second, clear images are reconstructed through deconvolution for degraded images of other targets based on the above scattering characteristics. Finally, the pixel value of the depth image for the TOF camera is the floating number which is different from common 2D images. Thus, most of the traditional imaging method evaluation is unsuitable for depth evaluation. A kind of difference image is employed for subjective evaluation and mean square error (MSE) for objective evaluation.

We put forward an NLOS depth imaging method based on the TOF camera. The scattering characteristics of the relay surface should be solved first to realize the reconstruction through deconvolution. However, the scattering characteristics of the relay surface are completely unknown in the NLOS mathematical model. For any of two different materials (Fig. 5), several different scales of matrices are adopted to express them in the algorithm, while the GA calculates matrix value by the fitness function [Eq. (8)]. The results are approximate numerical values of scattering characteristics of the relay surface (Fig. 7), and the 2D matrix with minimum fitness is set to be the deconvolution kernel. Then reconstruction is conducted for degraded NLOS depth images of the unknown target by Lucy-Richardson (LR) deconvolution. The experimental results reveal the effectiveness and feasibility of the algorithm both from subjective and objective evaluation (Figs. 8 and 9), but the results are not always convergent without complete expression of scattering characteristics in the 2D matrix. Although the solving process takes a long time, the calculated matrix of scattering characteristics can be applied. However, the GA-LR algorithm provides a low-cost NLOS 3D imaging technique, which can be directly applied to practical scenarios if the scattering characteristics database of common materials is established in advance. Therefore, the proposed method both provides theoretical guidance and an experimental basis for NLOS imaging, and also provides solutions to practical problems, with economic significance.

Based on a TOF camera, we propose a 3D imaging method for NLOS targets. By analyzing the optical phenomena, we put forward a hypothesis that the degraded 3D image results from the clear 3D image convolution with some unknown scattering process, where the unknown scattering process is caused by the relay surface. Therefore, the GA is adopted to solve the approximate optical model of the relay surface, and then the LR deconvolution algorithm is to perform the 3D reconstruction of NLOS targets. The experimental results show that the NLOS targets are well reconstructed from both subjective and objective aspects. In practical applications, after the approximation model of the relay surface is determined, this algorithm can quickly capture the NLOS target and perform 3D imaging when a TOF camera is utilized, which indicates high application significance. In the future, deep neural networks may be leveraged to fit the degradation process to realize reconstruction.