Infrared and Laser Engineering, Volume. 49, Issue 3, 0303001(2020)

Has 3D finally come of age? ——An introduction to 3D structured-light sensor

Chao Zuo1,2, Xiaolei Zhang3, Yan Hu1,2,3, Wei Yin1,2,3, Detong Shen3, Jinxin Zhong1,2,3, Jing Zheng3, and Qian Chen2
Author Affiliations
  • 1Smart Computational Imaging Laboratory (SCILab), School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
  • 2Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing 210094, China
  • 3Nanjing Zircon optoelectronic technology co. LTD, Nanjing 210014, China
  • show less
    Figures & Tables(68)
    Representative techniques for 3D optical sensing
    Measurement results of Stanford rabbits by photometric stereo method. (a) Stanford rabbit model; (b) Normal map; (c) Reconstruction map
    Schematic diagram of stereo vision
    Schematic diagram of time-of-flight method[24]
    Schematic of laser scanning
    Schematic diagram of defocus recovery method[35]
    Measurement results of the defocus recovery method[36]
    Schematic diagram of structured light projection method[38]
    Fringe order ambiguity in the wrapping phase of isolated objects and discontinuity surfaces
    Nintendo Wii with motion sensing controller[99]
    Microsoft Xbox360 with 3D motion sensor Kinect[103]
    Components of Kinect and the projected speckle pattern
    Apple iPhone X with 3D structured light sensor[106]
    Face-ID 3D face recognition technology of iPhone X[107]
    Alipay and WeChat face payment terminals[110]
    Ubiquitous 3D face payment terminals
    Schematic diagram of binocular stereo matching
    Difficulty in binocular matching of texture-less objects
    Speckle projection marks each point in space uniquely
    A module of an infrared structured-light 3D sensor[118]
    Three semiconductor lasers models (a) VCSEL; (b) DFB EEL; (c) Fabry–Pérot EEL
    Emitted patterns comparison of VCSEL, LED, and EEL
    (a) Schematic diagram of DOE diffraction; (b) Schematic diagram of an EEL dot projector
    (a) Schematic diagram of DOE duplicator; (b) Schematic diagram of a VCSEL dot projector
    Lens wafer and a WLO lens
    Projected patterns based on the non-formal codification (a) slot line constraint[43]; (b) Gray constraint[129]
    Projected patterns based on De Bruijn sequences[51] (a) De Bruijn sequences; (b) Projected patterns
    Projected patterns based on M-arrays[51] (a) M-array; (b) Projected patterns
    Projected patterns of commercial speckle-based structured light sensors. (a) Coding method based on global uniqueness; (b) Coding method based on Sudoku; (c) Coding method based on M-array
    Image correlation method based on local window
    Principle and process of one-dimensional matching in binocular stereo vision system. (a) Basic principle; (b) Matching process
    Schematic diagram of matching algorithm based on reference image
    Schematic diagram of cost aggregation algorithm[134]. (a) Cross construction; (b) Cost aggregation
    Schematic diagram of SGM[136]. (a) Minimum cost path; (b) 16 paths from all directions
    Effect of local windows with different sizes on the calculation results of disparity maps
    Schematic diagram of triangulation measurement model based on binocular vision system
    Pinhole model of the camera
    Architecture of 3D structured light sensor
    Hardware architecture of 3D sensor based on customized ASIC chip[143]
    Hardware architecture of 3D sensor based on FPGA[144]
    Global biometric industry market share
    3D face recognition using infrared ray can solve the influence of ambient illumination[146]
    Face recognition based on 3D data[148]
    3D finger tracker Leap Motion
    Statistics of global VR/AR industry investment in 2013-2017 (Unit: USD 100 million)
    3D camera space chaser N500[154]
    Similarity of speckle is influenced by the 3D surface
    Tradeoff between global uniqueness and spatial resolution of speckle pattern
    Relationship between baseline distance and measurement accuracy
    Ideal high-precision 3D face data and real 3D face data obtained by iPhone X
    HD 3D mask fools face recognition systems[155]
    World's first commercially mobile phone OPPO R17 Pro with ToF technology[156]
    Error comparison between ToF and structured-light at different measurement depths
    Real-time 3D measurement system at 120 Hz for complex dynamic scene[158]
    Quad-camera 3D measurement system based on SPU. (a) Appearance of the system; (b) Top view of the internal structure of the system; (c) System measurement scenario
    Results of the David model after registration. (a) Point cloud results; (b) Triangulation results
    Schematic diagram of a panoramic 3D structured light measurement system based on plane mirror reflection
    3D measurement results of a Voltaire model. (a) Full-surface 3D measurement results of a Voltaire model; (b)-(d) Corresponding results of (a) from three different views
    Highlight intensity removal with polarizers[205]
    Principle of fringe analysis method based on deep learning[221]
    Basic structure of a digital projector based on Digital Light Processing (DLP) technology and its core component DMD
    3D measurement and tracking a bullet fired from a toy gun. (a) Camera images at different time points; (b) Corresponding 3D reconstructions; (c) 3D reconstruction of the muzzle region (corresponding to the boxed region shown in (b)) as well as the bullet at three different points of time over the course of flight (7.5 ms, 12.6 ms, and 17.7 ms). The insets show the horizontal (x–z) and vertical (y-z) profiles crossing the body center of the flying bullet at 17.7 ms; (d) 3D point cloud of the scene at the last moment (135 ms), with the colored line showing the 130 ms long bullet trajectory. The inset plots the bullet velocity as a function of time
    Systems and results of 5D hyperspectral imaging and high speed thermal imaging[244, 245]. (a) 5D hyperspectral imaging system; (b) High speed thermal imaging system; (c) 5D hyperspectral imaging results: the measurement of water absorption by a citrus plant; (d) High-speed thermal imaging results: the measurement of a basketball player at different times
    Fast 3D face scanning system FaceScan
    Feature points for 3D facial recognition
    Future world of the real “3D”
    • Table 1. Matching function based on cross-correlation criteria

      View table
      View in Article

      Table 1. Matching function based on cross-correlation criteria

      CC correlation criterionDefinition
      Cross-correlation (CC)
      Normalized cross-correlation (NCC)
      Zero-normalized cross-correlation (ZNCC)
    • Table 2. Matching function based on SSD-correlation criteria

      View table
      View in Article

      Table 2. Matching function based on SSD-correlation criteria

      SSD correlation criterionDefinition
      Sum of squared differences (SSD)
      Normalized sum of squared differences (NSSD)
      Zero-normalized sum of squared differences (ZNSSD)
    Tools

    Get Citation

    Copy Citation Text

    Chao Zuo, Xiaolei Zhang, Yan Hu, Wei Yin, Detong Shen, Jinxin Zhong, Jing Zheng, Qian Chen. Has 3D finally come of age? ——An introduction to 3D structured-light sensor[J]. Infrared and Laser Engineering, 2020, 49(3): 0303001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Dec. 25, 2019

    Accepted: --

    Published Online: Apr. 22, 2020

    The Author Email:

    DOI:10.3788/IRLA202049.0303001

    Topics