Infrared and Laser Engineering, Volume. 51, Issue 3, 20210471(2022)

Research advances on non-line-of-sight three-dimensional imaging lidar

Chenfei Jin1, Xiaorui Tian1, Meng Tang1, Feng Wang2, Jie Yang1, Kai Qiao1, Xiaojie Shi1, and Siqi Zhang1
Author Affiliations
  • 1School of Physics, Harbin Institute of Technology, Harbin 150001, China
  • 2Tianjin Jinhang Institute of Technical Physics, Tianjin 300308, China
  • show less
    Figures & Tables(23)
    Basic configuration of the non-line-of-sight imaging lidar
    Experimental results of non-line-of-sight imaging with glass window[10-11]. (a) Diagram of light path; (b) Experimental scenario; (c) Imaging results of a person; (d) Imaging results of license plate
    Experimental results of non-line-of-sight imaging with different mirror surfaces[12]. (a)-(d) Position map of the detector, light source, target and the intermediate reflective surface and target respectively; (e)-(h) Experimental results of mirrors with different materials
    Experimental results of non-line-of-sight imaging with glass under different background lights[15]. (a) Schematic diagram of the experimental scene; (b1)-(b5) Experimental results under different conditions
    Experimental results of non-line-of-sight imaging using streak camera[1-2]. (a) Schematic of system principle; (b) Acquired raw data; (c) Reconstructed image; (d) Photo of hardware prototype
    Experimental results of non-line-of-sight imaging using ICCD[17]. (a)-(c) Results of point lighting experiment; (d)-(f) Results of surface lighting, from left to right are top view, side view, 3D panorama
    Experimental results of non-line-of-sight imaging using PMD[18]. (a) Scene; (b) System; (c) Lighting source; (d) Target; (e) Intensity image; (f) Depth image
    Experimental results of non-line-of-sight imaging using single pixel of SPAD [21]. (a) Experimental setup; (b) Reconstructed image
    Results of non-line-of-sight target detection using SPAD array [22] . (a) Experimental setup; (b) Diagram of target location
    Experimental results of non-line-of-sight target location and velocity measurement using SPAD array[23]. (a) Schematic diagram of target location experiment; (b) Schematic diagram of velocity measurement experiment
    Experimental results of pre-occlusion non-line-of-sight imaging [27]. (a) Experimental configuration; (b) Raw data in the absence of an occluder; (c) Raw data in the presence of the occluder; (d) Reconstructed image in the absence of an occluder; (e) Reconstructed image in the presence of the occluder
    Confocal non-line-of-sight imaging system based on Light Cone Transform (LCT)[28]. (a) Schematic diagram of principle; (b) Experimental setup; (c) Reconstructed image
    Confocal non-line-of-sight imaging experimental results based on frequency-wavenumber migration(f-k Mig)[29]. (a) Experiment scene; (b) Line-of-sight view; (c) Measured data; (d) Reconstructed image
    Experimental results of non-confocal non-line-of-sight imaging experiment result based on phasor-field virtual wave[30]. (a) Imaging system; (b) Target; (c) Reconstructed image
    Experiment results of scanning non-line-of-sight imaging with SPAD linear array[32]. (a) Imaging system, scene; (b) Restored 3D images by different pixels, respectively
    Experiment results of dual scanning non-line-of-sight imaging system[33]. (a)-(g) Experiment scene and system; (h), (i) Imaging results
    Processing results of the fast back projection algorithm[36]. (a) Normal back projection algorithm; (b) Fast back projection algorithm; (c) Target; (d) System
    Processing results of the optimization algorithm considering the direction of the target surface and the occlusion between the targets[41]. (a) Measured data; (b) Linear optimization reconstruction; (c) Improved optimization reconstruction
    Processing results of the LCT algorithm and other algorithms[28]. Processing results of back projection(a), filtered back projection(b), LCT(c), and LCT+ optimization(d)
    Processing results comparison of frequency wave vector migration(f-k Mig) and other algorithms[29]. Processing results of target(a), measurement data(b), filtered back projection(c), LCT(d), f-kMig(e)
    Processing results of the phasor-field virtual wave algorithm[30]. Processing results of confocal LCT (a), filtered back projection (b), virtual wave (c)
    • Table 1. Main performance parameters of a typical non-visual imaging system

      View table
      View in Article

      Table 1. Main performance parameters of a typical non-visual imaging system

      InstitutionsDetectorsLasersImaging performance
      Notes: 1. Tr: Time resolution; 2. Pixel: Pixel elements; 3. St: Sampling time; 4. λ: Wavelength; 5. τ: Pulse width; 6. f: Repetition frequency; 7. Num: Scan points; 8. P: Power; 9. W: Energy; 10. MBW: Modulation bandwidth; 11. GW: Gated width; 12. RS: Reconstruction space; 13. l0: Distance of object to diffuse wall; 14.l1: Distance of object to detector; 15. Size: Object size; 16. Ires: Image resolution; 17. RT: Reconstruction time
      Massachusetts Institute of Technology [1-2]Streak CameraTr: 2 ps Pixel: 672×1 St: 300-1200 s λ: 795 nm τ: 50 fs f: 75 MHz Num: 60 RS: 40 cm×40 cm×40 cm l0: 25 cm Ires: vertical 0.4 mm Horizontal 5-10 mm RT: 2-4 s
      French-German Research Institute of Saint-Louis [16-17]ICCDGW: 2 ns Pixel: 1360×1024 λ: 532 nm τ: 2 ns f: 21 kHz W: 18 μJ Size: 10 in×10 in l0: 2 m Ires: limited by gate width
      The University of British Columbia and University of Bonn [18, 20]PMDMBW: 110-180 MHz Pixel: 120×160 St: 4 min λ: 650 nm τ: 2-3 ns P: 1.5 W RS: 1.5 m×1.5 m×2 m l0: 1.5 m Ires: a few centimeters - more than ten centimeters
      Stanford University [28]Single-pixel SPAD (Confocal LCT) Tr: 27 ps Pixel: 1 St: a few minutes to one hour λ: 670 nm τ: 30.6 ps f: 10 MHz P: 0.11 mW Num: 64×64/32×32 RS: 1 m×1 m×1 m l0: 40-115 cm Ires: 3.1 cm@65 cm
      Stanford University and Carnegie Mellon University [29]Single-pixel SPAD (Confocal f-k) Tr: 70 ps Pixel: 1 St: 0.25 s-180 min λ: 532 nm τ: 35 ps f: 10 MHz P: 1 W Num: 512×512/64× 64/32×32 RS: 2 m×2 m×1.5 m l0: 1 m Ires: a few centimeters - more than ten centimeters
      University of Wisconsin and University of Zaragoza [30]Single-pixel SPAD (Virtual waves) Tr: 30 ps Pixel: 1 St: 24 s-20 min λ: 532 nm τ: 35 ps f: 10 MHz P: 1 W Num: 180×130 RS: 1.8 m×1.3 m×5.1 m l0: 0.5-5.1 m Ires: a few centimeters @0.5 m
      University of Milan and University of Wisconsin[32]Linear-array SPADTr: 50 ps Pixel: 16×1 St: 8 min λ: 532 nm f: 10 MHz P: 1 W Num: 150×150 Size: 40 cm×60 cm l0: 1 m Ires: a few centimeter
      French-German Research Institute of Saint-Louis [22]Planar-array InGaAs SPADTr: 250 ps Pixel: 32×32 St: 340 ms λ: 1.545 μm τ: 500 ps f: 100 kHz W: 3 μJ Size: 25 cm×25 cm l0:1.8 m Ires: approximate location
      Heriot-Watt University [23]Planar-array Si SPAD Tr: 110 ps Pixel: 32×32 St: 3 s λ: 800 nm τ: 10 fs f: 67 MHz W: 10 nJ Size: 30 cm×10 cm×4 cm l1: about 1 m Ires: objects positioning and tracking with centimeter accuracy
    • Table 2. Performance comparison of different NLOS imaging algorithms

      View table
      View in Article

      Table 2. Performance comparison of different NLOS imaging algorithms

      FBPLCTf-kVirtual wavesOptimization (single iteration)
      Runtime${{O} }({N^5})$${{O} }({N^3}\log N)$${{O} }({N^3}\log N)$${{O} }({N^5})$${{O} }({N^5})$
      Memory${{O} }({N^3})$${{O} }({N^3})$${{O} }({N^3})$${{O} }({N^3})$${{O} }({N^5})$
      QualityLowMid-highMid-highMid-highHigh
      ConfigurationConfocal or non-confocal ConfocalConfocal or approximately non-confocal Confocal or non-confocalConfocal or non-confocal
    Tools

    Get Citation

    Copy Citation Text

    Chenfei Jin, Xiaorui Tian, Meng Tang, Feng Wang, Jie Yang, Kai Qiao, Xiaojie Shi, Siqi Zhang. Research advances on non-line-of-sight three-dimensional imaging lidar[J]. Infrared and Laser Engineering, 2022, 51(3): 20210471

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Lasers & Laser optics

    Received: Jul. 12, 2021

    Accepted: --

    Published Online: Apr. 8, 2022

    The Author Email:

    DOI:10.3788/IRLA20210471

    Topics