Acta Optica Sinica, Volume. 42, Issue 14, 1415002(2022)
Visual SLAM Method Based on Optical Flow and Instance Segmentation for Dynamic Scenes
Figures & Tables(24)
Fig. 3. Dynamic region detection of optical flow field direction. (a) Visualization of vector direction of original optical flow field; (b) visualization of vector direction of optical flow field after transformation; (c) edge of optical flow field vector direction; (d) instantaneous centre of velocity
Fig. 4. Velocity instantaneous centre positions. (a) Occupying two quadrants; (b) occupying single quadrant; (c) occupying four quadrants; (d) occupying two quadrants and their boundaries; (e) occupying single quadrant and its boundaries
Fig. 5. Dynamic region mask in optical flow direction. (a) Edge of optical flow field vector direction; (b) result of edge open operation; (c) dynamic region mask
Fig. 6. Flow chart of dynamic region detection algorithm based on optical flow direction
Fig. 9. Dynamic region detection based on optical flow field amplitude. (a) Original optical flow; (b) optical flow of camera motion; (c) optical flow of dynamic objects; (d) dynamic region mask
Fig. 10. Flow chart of dynamic region detection algorithm based on optical flow field amplitude
Fig. 13. Trajectory and error of TUM. (a) (b) Trajectory and error of ORB-SLAM2; (c) (d) trajectory and error of proposed algorithm
Fig. 14. Trajectory and error of TUM. (a) (b) Trajectory and error of ORB-SLAM2; (c) (d) trajectory and error of proposed algorithm
Fig. 15. Trajectory and error of KITTI. (a) Sequence 01 of KITTI; (b) sequence 09 of KITTI
Table 1. ATE for TUM
View tableTable 1. ATE for TUM
Sequence ORB-SLAM2 Ours RMSE Median Mean SD RMSE Median Mean SD fr3/s/hs 0.0234 0.0170 0.0189 0.0138 0.0162 0.0128 0.0141 0.0079 fr3/s/rpy 0.0223 0.0122 0.0169 0.0146 0.0180 0.0109 0.0142 0.0112 fr3/s/static 0.0095 0.0075 0.0083 0.0046 0.0057 0.0043 0.0049 0.0030 fr3/s/xyz 0.0089 0.0071 0.0077 0.0044 0.0082 0.0060 0.0070 0.0044 fr3/w/hs 0.5812 0.4899 0.4909 0.3112 0.0186 0.0149 0.0163 0.0090 fr3/w/rpy 0.8416 0.6353 0.7147 0.4445 0.0320 0.0232 0.0261 0.0184 fr3/w/static 0.3844 0.3309 0.3590 0.1374 0.0078 0.0063 0.0069 0.0037 fr3/w/xyz 0.6867 0.5446 0.6209 0.2932 0.0145 0.0103 0.0121 0.0081
Table 2. Relative translation error of TUM
View tableTable 2. Relative translation error of TUM
Sequence ORB-SLAM2 Ours RMSE Median Mean SD RMSE Median Mean SD fr3/s/hs 0.0081 0.0058 0.0068 0.0045 0.0128 0.0090 0.0106 0.0071 fr3/s/rpy 0.0132 0.0075 0.0098 0.0089 0.0117 0.0075 0.0091 0.0073 fr3/s/static 0.0055 0.0041 0.0047 0.0028 0.0052 0.0040 0.0045 0.0026 fr3/s/xyz 0.0083 0.0067 0.0072 0.0042 0.0081 0.0064 0.0070 0.0041 fr3/w/hs 0.0242 0.0133 0.0186 0.0155 0.0135 0.0091 0.0110 0.0079 fr3/w/rpy 0.0715 0.0166 0.0240 0.0674 0.0182 0.0108 0.0139 0.0117 fr3/w/static 0.0178 0.0076 0.0119 0.0132 0.0063 0.0039 0.0048 0.0040 fr3/w/xyz 0.0372 0.0172 0.0224 0.0298 0.0123 0.0080 0.0099 0.0074
Table 3. Relative rotation error of TUM
View tableTable 3. Relative rotation error of TUM
Sequence ORB-SLAM2 Ours RMSE Median Mean SD RMSE Median Mean SD fr3/s/hs 0.3620 0.2626 0.3103 0.1864 0.4095 0.3064 0.3508 0.2113 fr3/s/rpy 0.4097 0.2947 0.3396 0.2291 0.4036 0.3013 0.3443 0.2108 fr3/s/static 0.1646 0.1204 0.1411 0.0848 0.1638 0.1190 0.1390 0.0867 fr3/s/xyz 0.3152 0.2354 0.2685 0.1651 0.3134 0.2361 0.2669 0.1643 fr3/w/hs 0.6217 0.4168 0.5034 0.3648 0.4281 0.3002 0.3569 0.2363 fr3/w/rpy 1.5127 0.4391 0.5985 1.3892 0.4785 0.3219 0.3858 0.2829 fr3/w/static 0.3719 0.2070 0.2729 0.2526 0.1815 0.1288 0.1520 0.0992 fr3/w/xyz 0.7920 0.4216 0.5210 0.5965 0.3902 0.2354 0.2794 0.2723
Table 4. Improvement of ATE for TUM
View tableTable 4. Improvement of ATE for TUM
Sequence RMSE Median Mean SD fr3/s/hs 30.8 24.4 25.1 42.9 fr3/s/rpy 19.1 10.9 16.0 23.3 fr3/s/static 39.9 42.5 41.3 35.3 fr3/s/xyz 7.7 15.0 10.3 0 fr3/w/hs 96.8 97.0 96.7 97.1 fr3/w/rpy 96.2 96.3 96.3 95.9 fr3/w/static 98.0 98.1 98.1 97.3 fr3/w/xyz 97.9 98.1 98.1 97.2
Table 5. Improvement of relative translation error for TUM
View tableTable 5. Improvement of relative translation error for TUM
Sequence RMSE Median Mean SD fr3/s/hs -57.4 -55.5 -57.3 -57.5 fr3/s/rpy 11.9 0.4 7.1 18.0 fr3/s/static 5.2 1.1 4.2 8.0 fr3/s/xyz 2.7 4.2 3.1 1.4 fr3/w/hs 44.1 31.6 40.8 49.2 fr3/w/rpy 74.6 34.8 42.0 82.7 fr3/w/static 64.7 48.3 59.8 69.3 fr3/w/xyz 66.9 53.6 55.9 75.1
Table 6. Improvement of relative rotation error for TUM
View tableTable 6. Improvement of relative rotation error for TUM
Sequence RMSE Median Mean SD fr3/s/hs -13.1 -16.7 -13.1 -13.3 fr3/s/rpy 1.5 -2.2 -1.4 8.0 fr3/s/static 0.5 1.2 1.5 -2.3 fr3/s/xyz 0.6 -0.3 0.6 0.5 fr3/w/hs 31.1 28.0 29.1 35.2 fr3/w/rpy 68.4 26.7 35.5 79.6 fr3/w/static 51.2 37.8 44.3 60.7 fr3/w/xyz 50.7 44.2 46.4 54.3
Table 7. Comparison of RMSE of absolute trajectory for TUM
View tableTable 7. Comparison of RMSE of absolute trajectory for TUM
Sequence DS-SLAM Dyna-SLAM Detect-SLAM SOF-SLAM Ours fr3/s/hs — 0.0170 0.0231 — 0.0162 fr3/s/rpy — — — — 0.0180 fr3/s/static 0.0065 — — 0.0100 0.0057 fr3/s/xyz — 0.0150 0.0201 — 0.0082 fr3/w/hs 0.0303 0.0250 0.0514 0.0290 0.0186 fr3/w/rpy 0.4442 0.0400 0.2959 0.0270 0.0320 fr3/w/static 0.0081 0.0060 — 0.0070 0.0078 fr3/w/xyz 0.0247 0.0150 0.0241 0.0180 0.0145
Table 8. Comparison of test results on KITTI dataset
View tableTable 8. Comparison of test results on KITTI dataset
Sequence ORB-SLAM2 DynaSLAM VDO-SLAM Ours RMSE of ATE /m RPEt /（m·frame-1） RPEr /［（°）·frame-1］ RMSE of
ATE /m
RPEt /（m·frame-1） RPEr /［（°）·frame-1］ RMSE of
ATE /m
RPEt /（m·frame-1） RPEr /［（°）·frame-1］ RMSE of
ATE /m
RPEt /（m·frame-1） RPEr /［（°）·frame-1］ 00 1.30 0.04 0.06 1.40 0.04 0.06 — 0.05 0.05 1.24 0.02 0.06 01 10.40 0.05 0.04 9.40 0.05 0.04 — 0.12 0.04 9.03 0.05 0.04 02 5.70 0.04 0.03 6.70 0.04 0.03 — 0.04 0.02 5.35 0.02 0.05 03 0.60 0.07 0.04 0.60 0.07 0.04 — 0.09 0.04 0.59 0.02 0.04 04 0.20 0.07 0.06 0.20 0.07 0.06 — 0.11 0.05 0.17 0.02 0.03 05 0.80 0.06 0.03 0.80 0.06 0.03 — 0.10 0.02 0.77 0.01 0.04 06 0.80 0.02 0.04 0.80 0.02 0.04 — 0.02 0.05 0.72 0.02 0.03 07 0.50 0.05 0.07 0.50 0.05 0.07 — — — 0.46 0.01 0.04 08 3.60 0.08 0.04 3.50 0.08 0.04 — — — 3.19 0.03 0.05 09 3.20 0.06 0.05 1.60 0.06 0.05 — — — 1.57 0.02 0.05 10 1.00 0.07 0.04 1.20 0.07 0.04 — — — 0.96 0.01 0.05
Table 9. Running time of each module
View tableTable 9. Running time of each module
Module Time MaskFlownet-S 27.6 YOLACT++ 23.2 Optical flow field direction mask 8.1 Optical flow field amplitude mask Depth image inpainting 73.2 Optical flow of static scene 2.1 K-means clustering 10.8
Tools
Get Citation
Copy Citation Text
Chen Xu, Yijun Zhou, Chen Luo. Visual SLAM Method Based on Optical Flow and Instance Segmentation for Dynamic Scenes[J]. Acta Optica Sinica, 2022, 42(14): 1415002
Paper Information
Category: Machine Vision
Received: Nov. 30, 2021
Accepted: Jan. 24, 2022
Published Online: Jul. 15, 2022
The Author Email: Luo Chen (chenluo@seu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20