Opto-Electronic Engineering, Volume. 50, Issue 4, 220246(2023)
STransMNet: a stereo matching method with swin transformer fusion
Figures & Tables(11)
Fig. 2. (a) The network structure of STransMNet; (b) The structure of extractor
Fig. 3. Euclidean distance between the pixel features on the left image. (a) There is a feature differentiation loss; (b) No a feature differentiation loss
Fig. 4. Disparity map estimated by different methods on the Sceneflow datasets
Fig. 5. Disparity map estimated by different methods on the KITTI datasets
Table 1. Ablation study
View tableView in ArticleTable 1. Ablation study
实验 基于Swin Transformer模块 相关运算 特征差异化损失 3 px error / % ↓ EPE ↓ Occ IOU ↑ 第1组 1.68 0.56 0.94 第2组 √ 1.36 0.48 0.96 第3组 √ √ 1.40 0.51 0.95 第4组 √ √ √ 1.03 0.42 0.97
Table 2. Experimental results of different loss weights
View tableView in ArticleTable 2. Experimental results of different loss weights
Ld1,r Ld1,f Lrr Lbe,f Ldiff 3 px error /%↓ EPE ↓ Occ IOU ↑ 0.2 0.2 0.2 0.2 0.2 0.85 0.41 0.97 0.2 0.2 0.2 0.1 0.3 0.93 0.51 0.84 0.3 0.3 0.1 0.1 0.2 0.89 0.43 0.85 0.3 0.4 0.1 0.1 0.1 0.84 0.39 0.96 0.4 0.3 0.1 0.1 0.1 0.87 0.40 0.91
Table 3. Test results Ⅰ of model generalization performance
View tableView in ArticleTable 3. Test results Ⅰ of model generalization performance
模型 MPI Sintel KITTI 3 px error /% ↓ EPE ↓ Occ IOU ↑ 3 px error /% ↓ EPE ↓ Occ IOU ↑ PSMNet[ 10 ]6.81 3.31 N/A 27.79 6.56 N/A AANet[ 11 ]5.91 1.89 N/A 12.42 1.99 N/A STTR-light[ 19 ]5.82 2.95 0.69 7.2 1.56 0.95 STTR[ 19 ]5.75 3.01 0.86 6.74 1.50 0.98 本文算法 5.23 2.78 0.84 6.51 1.44 0.97
Table 4. Test results Ⅱ of model generalization performance
View tableView in ArticleTable 4. Test results Ⅱ of model generalization performance
模型 Middlebury SCARED 3 px error /% ↓ EPE ↓ Occ IOU ↑ 3 px error /% ↓ EPE ↓ Occ IOU ↑ PSMNet[ 10 ]12.96 3.05 N/A OOM OOM N/A AANet[ 11 ]12.80 2.19 N/A 6.39 1.36 N/A STTR-light[ 19 ]5.36 2.05 0.76 3.30 1.19 0.89 STTR[ 19 ]6.19 2.33 0.95 3.69 1.57 0.96 本文算法 6.24 2.12 0.96 3.15 1.26 0.94
Table 5. Comparative experiments
View tableView in ArticleTable 5. Comparative experiments
模型 Sceneflow KITTI 3 px error / % ↓ EPE ↓ Occ IOU ↑ 3 px error / % ↓ EPE ↓ Occ IOU ↑ PSMNet[ 10 ]3.94 1.11 N/A 1.25 0.57 N/A AANet[ 11 ]3.89 0.82 N/A 1.93 0.64 N/A STTR-light[ 19 ]1.84 0.56 0.98 1.68 0.56 0.94 STTR[ 19 ]1.43 0.48 0.91 1.12 0.44 0.97 本文算法 1.03 0.42 0.97 0.84 0.39 0.96
Table 6. Comparison of model operation efficiency
View tableView in ArticleTable 6. Comparison of model operation efficiency
模型 Params ↓ / M FLOPs ↓ / G Memory ↓ / G Runtime ↓ / s PSMNet[ 10 ]5.22 613.90 4.08 0.63 AANet[ 11 ]3.68 119.64 1.63 0.09 STTR-light[ 19 ]2.33 110.21 0.43 0.65 STTR[ 19 ]2.51 510.93 1.23 0.67 本文算法 27.85 136.11 2.90 0.73
Tools
Get Citation
Copy Citation Text
Gaoping Wang, Xun Li, Xuefang Jia, Zhewen Li, Wenjie Wang. STransMNet: a stereo matching method with swin transformer fusion[J]. Opto-Electronic Engineering, 2023, 50(4): 220246
Paper Information
Category: Article
Received: Oct. 8, 2022
Accepted: Jan. 19, 2023
Published Online: Jun. 15, 2023
The Author Email: Li Xun (lixun@xpu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20