Laser & Optoelectronics Progress, Volume. 60, Issue 3, 0312010(2023)

A Review of Position and Orientation Visual Measurement Methods and Applications

Zhenzhong Wei*, Guangkun Feng, Danya Zhou, Yueming Ma, Mingkun Liu, Qifeng Luo, and Tengda Huang
Author Affiliations
  • Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
  • show less
    Figures & Tables(53)
    Perspective projection imaging model of a camera
    Position and orientation monocular visual measurement model
    3D reconstruction model of stereo visual measurement system
    Position and orientation measurement based on 3D reconstruction of stereo visual measurement system
    Development of classical object pose measurement methods based on point feature
    Comparesion of different PnP algorithms [16]. (a) Ordinary-3D point configuration; (b) quasi-singular point configuration; (c) planar point configuration
    Development of deep learning object pose measurement methods based on point feature
    Deep learning based visual pose measurement using point feature. (a) PVNet [22] based on sparse keypoints; (b) CDPN [23] based on dense points; (c) GDR-Net [27] based on points-guided direct regression
    Space geometry diagram of the PnL problem[34]
    Development of PnL methods
    Process of pose iterative method based on contour features [45]. (a) RGB image; (b) extracted contour features; (c) 2D contour projections corresponding to initial pose (red) and optimized pose (green); (d) model projection corresponding to optimized pose
    Development of pose iterative methods based on edge and contour feature
    Time line of region-based pose measurement method
    Local area correspondence of target surface[50]
    Estimation of foreground and background probability based on different shape of areas. (a) Local circular area[57]; (b) banded area[61]
    Development of object pose measurement methods based on multiple features
    Object pose measurement results under strong occlusion [65]
    Pose measurement method under hybrid representations [68]
    Aircraft pose estimation approach with keypoints and structures[21]
    Development of object pose measurement methods based on 3D reconstruction
    Multi-view pose measurement methods. (a) Multi-view 3D keypoints estimation[71]; (b) multi-view 3D bounding box estimation[72]
    Disparities estimation method. (a) Semiglobal matching (SGM) stereo method[74]; (b) GC-Net: geometry and context for deep stereo regression[75]
    Multi-source fusion pose measurement algorithm based on vision/IMU [83]
    Multi-source fusion pose measurement algorithm based on GPS/vision [87]
    Multi-source fusion pose measurement algorithm based on IMU/GPS/vision/UWB[89]
    FFB6D algorithm framework using RGB-D image as input [90]
    Coordinate system setting in simulation experiment[94]
    Model of aircraft landing process[95]
    Experiment of aircraft landing guidance system[97]. (a) Schematic diagram of the experiment; (b) experimental setup
    Trinocular visual system of Madrid University of Technology [99]
    Stereo vision guidance system in the test of UAV autonomous landing[100]
    Visual measurement and recovery system of Portuguese Naval ship-based UAV [101]
    Monocular pose measurement system applied to aircraft test task[68] .(a) Monocular pose measurement system model; (b) real-time pose results
    SMART-OLEV docking with the client satellite[103]
    Visual measurement of muzzle vibration[104]
    Image sequence of embedded missile separation under different attack angles of carrier aircraft[105]. (a) Attack angles is 0°; (b) attack angle is 2°; (c) attack angle is 3°
    Test of projectile penetrating ice target [106]. (a) Imaging with normal high speed camera; (b) imaging with high speed infrared camera
    High speed camera and position and attitude interpretation [107]. (a) Before hitting the target; (b) the hitting moment; (c) 0.015 s after; (d) 0.0875 s after
    Aircraft cabin assembly in airbus assembly line[108]
    TriDAR on board Space Shuttle Discovery[109]
    Cabin automatic docking platform[111]
    Automatic assembly equipment for low pressure turbine shaft of aero-engine[112]
    High precision micro-vision automatic assembly system[113]
    Human action recognition[123]
    Human pose tracking[126]
    Human pose estimation in the 2022 Beijing Winter Olympics. (a) 3DAT athlete tracking system[130]; (b) athlete scoring system[131]
    Pilot pose estimation[134]
    • Table 1. Comparison on object pose measurement methods based on point feature

      View table

      Table 1. Comparison on object pose measurement methods based on point feature

      MethodYearAccuracySpeedFeasibility
      SURF102006GoodCommonScale and rotation invariance
      ORB112011CommonGood(10 times faster than SURF)Suitable for applications requiring speed priority
      BB8182017CommonCommonThe deviation between corner and target area leads to limted measurement accuracy
      PVNet222019GoodGoodSuitable for severe occlusion or truncation
      CDPN232019BetterBetterBetter robustness and local occlusion resistance
      GDR-Net272021BetterBetterEnsuring the real-time performance,accuracy and robustness
    • Table 2. Comparison of PnL methods on VGG dataset

      View table

      Table 2. Comparison of PnL methods on VGG dataset

      MethodModel-HouseCorridorMerton-College-Merton-College-ⅡMerton-College-ⅢUniversity-LibraryWadham-College
      Number of images101133335
      LPnL-Bar-LS40Δθ /(°)0.41350.11780.02410.02610.06520.36420.1526
      Δt /m0.04030.04400.00990.01490.02330.16320.0909
      AlgLS42Δθ /(°)0.42200.19833.620055.80373.74951.883860.0517
      Δt /m0.03840.08881.150414.18791.36830.95199.8801
      RPnL43Δθ /(°)0.55210.36521.08700.32491.75282.97310.4200
      Δt /m0.06310.11500.32150.16600.91211.56130.1909
      ASPnL40Δθ /(°)0.22650.09110.11410.15151.55843.66620.4227
      Δt /m0.01620.02980.03140.06000.55711.66830.1955
      SRPnL44Δθ /(°)0.2258158.95200.43810.115136.40344.18480.0880
      Δt /m0.016017.55700.10640.04953.93982.06320.0407
      EPnL34Δθ /(°)0.22650.09690.03060.01700.05040.08710.0808
      Δt /m0.01620.02520.00970.01230.01470.03430.0375
    • Table 3. Comparison of pose measurement methods based on region-based feature

      View table

      Table 3. Comparison of pose measurement methods based on region-based feature

      MethodYearAccuracySpeedFeasibility
      Schmaltz et al.532007CommonCommonGood application effect in weak texture,just for asymmetric targets
      Brox et al.542012BetterCommonBetter application effect for occluded or symmetric targets,better robustness
      Tjaden et al.572017BetterGoodBetter application effect for occluded or symmetric targets,good robustness and better real-time performance
      Zhong et al.592020BetterBetterBetter robustness and more suitable for symmetric targets
      Hodaň et al.502021GoodCommonBetter suitable for occluded or symmetric targets,better robustness
    • Table 4. Comparison on object pose measurement methods based on multiple feature

      View table

      Table 4. Comparison on object pose measurement methods based on multiple feature

      MethodYearAccuracySpeedFeasibility
      Choi et al.632010CommonGoodNeed to know the 3D coordinates of each keypoint
      Choi et al.642011GoodCommonNeed to know the 3D coordinates of each keypoint
      Pauwels et al.652013CommonGoodDifficult to extend to the articulated scenario
      Tuzel et al.662014GoodCommonIdentifies and ranks important features on 3D object
      Liu et al.582021GoodBetterSuitable for application
      Fan et al.212021GoodCommonNeed the 3D aircraft model
      Hu et al.672019CommonGoodSuitable for severe occlusion
      Zhong et al.622020GoodCommon

      Still fail for heavy occlusions

      No efficient 3D object detection module for pose initialization

      Song et al.682020GoodBetterLimited by image resolution
    • Table 5. Comparison on object pose measurement methods based on 3D reconstruction

      View table

      Table 5. Comparison on object pose measurement methods based on 3D reconstruction

      MethodYearAccuracySpeedFeasibility
      SGM et al.742007CommonGoodPoor application effect in weak texture,transparent,reflective objects scenes
      FPFH et al.772009BetterBetterThe reduction of complexity compared to PFH ensures the real-time performance
      SHOT et al.782014GoodGoodBetter robustness,efficiency and description for point clouds with noise,clutter and uneven density
      GC-Net et al.752017BetterBetterMore suitable for multi-target tracking in occluded scenes
    • Table 6. Multi-source data fusion pose measurement method

      View table

      Table 6. Multi-source data fusion pose measurement method

      ClassificationCombinationCharacteristicApplication scenario
      Multi-sensor data fusionVisual system+inertial navigation systemInertial navigation system has low cost and good stability,but there are accumulated errors. Visual measurement has good autonomy and high precision in close rangeFlight process of aircraft,automatic pilot and so on
      Visual system+LiDARLiDAR is used to acquire depth information,and the visual system is used to acquire other information in the sceneSLAM
      Visual system+other sensorsGPS is used for positioning,and UWB is used to obtain position information,etc.,which can assist the visual system to obtain more accurate position and poseFlight process of aircraft,human-computer interaction and so on
      Multi-dimensional feature fusion2D information+deep informationCommon features(points,lines,regions,etc.)can be acquired in two-dimensional images,and depth information can make up for the deficiency of two-dimensional information and obtain more abundant object featuresPosture measurement of static objects
    Tools

    Get Citation

    Copy Citation Text

    Zhenzhong Wei, Guangkun Feng, Danya Zhou, Yueming Ma, Mingkun Liu, Qifeng Luo, Tengda Huang. A Review of Position and Orientation Visual Measurement Methods and Applications[J]. Laser & Optoelectronics Progress, 2023, 60(3): 0312010

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Instrumentation, Measurement and Metrology

    Received: Dec. 28, 2022

    Accepted: Jan. 25, 2023

    Published Online: Mar. 3, 2023

    The Author Email: Wei Zhenzhong (zhenzhongwei@buaa.edu.cn)

    DOI:10.3788/LOP223420

    Topics