Acta Optica Sinica, Volume. 43, Issue 15, 1510003(2023)
Research Progress in Fundamental Architecture of Deep Learning-Based Single Object Tracking Method
Figures & Tables(9)
Fig. 1. Two basic architectures of single object tracking methods. (a) Siamese network-based two-stream tracking method; (b) transformer-based one-stream tracking method
Fig. 3. Feature fusion methods in single object tracking methods. (a) Naïve cross correlation-based fusion; (b) depth-wise cross convolution-based fusion; (c) pixel-wise correlation-based fusion; (d) cross-attention-based fusion; (e) concatenation-based fusion
Fig. 4. Tracking head in single object tracking methods. (a) Anchor-based head; (b) anchor-free head; (c) corner-based head
Table 1. Structural composition of representative deep learning-based single object tracking methods
View tableTable 1. Structural composition of representative deep learning-based single object tracking methods
Tracker Publication Feature extraction Feature fusion Tracking head SiamFC［ 2 ］ECCVW 2016 AlexNet Naïve cross correlation — SiamRPN［ 39 ］CVPR 2018 AlexNet Up-channel cross correlation Anchor-based SiamRPN++［ 34 ］CVPR 2019 ResNet Depth-wise cross convolution Anchor-based SiamFC++［ 7 ］AAAI 2020 InceptionV3 Depth-wise cross convolution Anchor-free SiamCAR［ 51 ］CVPR 2020 ResNet Depth-wise cross convolution Anchor-free PCDHV［ 41 ］ACCV 2021 InceptionV3 Pixel-wise cross convolution Corner-based TransT［ 12 ］CVPR 2021 ResNet50 Cross-attention-based fusion Anchor-free STARK［ 44 ］ICCV 2021 ResNet Concatenation-based fusion Corner-based SwinTrack［ 28 ］NeurIPS 2022 Swin transformer Concatenation-based fusion Anchor-free MixFormer［ 53 ］CVPR 2022 One-stage feature extraction and fusion Corner-based SimTrack［ 54 ］ECCV 2022 One-stage feature extraction and fusion Corner-based OSTrack［ 14 ］ECCV 2022 One-stage feature extraction and fusion Anchor-free
Table 2. Common single object tracking datasets
View tableTable 2. Common single object tracking datasets
Dataset Sequence Attribute Average duration /s Minimum frame Maximum frame VOT-2018［ 15 ］60 12 355.90 41 1500 UAV123［ 61 ］123 12 30.48 109 3085 GOT-10k［ 16 ］Total：9935
Training：9335
Validating：180
Testing：420
6 15.00 51 920 LaSOT［ 17 ］Total：1400
Training：1120
Testing：280
14 83.57 1000 11397 TrackingNet［ 18 ］Total：30643
Training：30132
Testing：511
15 16.70 96 2368 TNL2K［ 62 ］Total：2000
Training：1300
Testing：700
17 20.74 21 18488
Table 3. Video properties of common single object tracking datasets
View tableTable 3. Video properties of common single object tracking datasets
No. Attr Description GOT-10k UAV123 LaSOT TrackingNet TNL2K 1 POC Partial occlusion √ √ √ √ √ 2 FOC Full occlusion √ √ √ √ 3 SV Scale variation √ √ √ √ √ 4 ARC Aspect ratio change √ √ √ √ √ 5 FM Fast motion √ √ √ √ √ 6 IV Illumination variation √ √ √ √ √ 7 LR Low resolution √ √ √ √ √ 8 OV Out-of-view √ √ √ √ 9 CM Camera motion √ √ √ √ 10 BC Background clutter √ √ √ √ 11 VC Viewpoint change √ √ √ 12 SOB Similar object √ 13 DEF Deformation √ √ √ 14 MB Motion blur √ √ √ 15 IPR In-plane rotation √ √ √ 16 OPR Out-of-plane rotation √ √ 17 AS Influence of adversarial samples √ 18 TC Two targets with similar intensity cross each other √ 19 MS Video contain both color and thermal images √
Table 4. Evaluation metrics of single object tracking methods
View tableTable 4. Evaluation metrics of single object tracking methods
Category Principle Evaluation metric Applicable dataset S Intersection over union（IoU）between tracking results and groundtruths AO（average overlap） GOT-10k SR（success rate） GOT-10k Success plot LaSOT AUC（area under the curve） LaSOT，TrackingNet P Pixel distance between centers of tracking results and groundtruths Precision plot LaSOT，TrackingNet Precision LaSOT，TrackingNet Normalized precision LaSOT，TrackingNet
Table 5. Performance comparison of single object tracking methods on GOT-10K, LaSOT, and TrackingNet datasets
View tableTable 5. Performance comparison of single object tracking methods on GOT-10K, LaSOT, and TrackingNet datasets
Type Tracker Publication GOT-10k［ 16 ］LaSOT［ 17 ］TrackingNet［ 18 ］AO SR50 SR75 AUC Pnorm P AUC Pnorm P Two-stream SiamFC［ 2 ］ECCVW 2016 39.2 42.6 13.5 — — — 57.1 65.4 53.3 SiamRPN［ 29 ］CVPR 2018 48.1 58.1 27.0 — — — — — — SiamRPN++［ 34 ］CVPR 2019 51.7 61.6 32.5 49.6 56.9 49.1 73.3 80.0 69.4 SiamBAN［ 49 ］CVPR 2020 — — — 51.4 59.8 52.1 — — — CGACD［ 52 ］CVPR 2020 — — — 51.8 62.6 — 71.1 80.0 69.3 SiamCAR［ 51 ］CVPR 2020 56.9 67.0 41.5 50.7 60.0 51.0 — — — SiamAttn［ 48 ］CVPR 2020 — — — 56.0 64.8 — 75.2 81.7 — SiamFC++［ 7 ］AAAI 2020 59.5 69.5 47.9 54.4 62.3 54.7 75.4 80.0 70.5 Ocean［ 63 ］ECCV 2020 61.1 72.1 47.3 56.0 65.1 56.6 — — — TransT［ 12 ］CVPR 2021 67.7 76.8 60.9 64.9 73.8 69.0 81.4 86.7 80.3 STMTrack［ 42 ］CVPR 2021 64.2 73.7 57.5 60.6 69.3 63.3 80.3 85.1 76.7 AutoMatch［ 46 ］ICCV 2021 65.2 76.6 54.3 58.2 — 59.9 76.0 — 72.6 STARK［ 44 ］ICCV 2021 68.8 78.1 64.1 67.1 77.0 — 82.0 86.9 — SparseTT［ 37 ］IJCAI 2022 69.3 79.1 63.8 66.0 74.8 70.1 81.7 86.6 79.5 CsWinTT［ 45 ］CVPR 2022 69.4 78.9 65.4 66.2 75.2 70.9 81.9 86.7 79.5 SwinTrack［ 28 ］NeurIPS 2022 72.4 80.5 67.8 71.3 — 76.5 84.0 — 82.8 One-stream SBT［ 13 ］CVPR 2022 70.4 80.8 64.7 66.7 — 71.1 — — — MixFormer［ 53 ］CVPR 2022 70.7 80.0 67.8 70.1 79.9 76.3 83.9 88.9 83.1 OsTrack［ 14 ］ECCV 2022 73.7 83.2 70.8 71.1 81.1 77.6 83.9 88.5 83.2 SimTrack［ 54 ］ECCV 2022 69.8 78.8 66.0 70.5 79.7 — 83.4 — 87.4 CTTrack［ 57 ］AAAI 2023 72.8 81.3 71.5 69.8 79.7 76.2 84.9 89.1 83.5
Tools
Get Citation
Copy Citation Text
Tingfa Xu, Ying Wang, Guokai Shi, Tianhao Li, Jianan Li. Research Progress in Fundamental Architecture of Deep Learning-Based Single Object Tracking Method[J]. Acta Optica Sinica, 2023, 43(15): 1510003
Paper Information
Category: Image Processing
Received: Mar. 29, 2023
Accepted: Jun. 15, 2023
Published Online: Aug. 15, 2023
The Author Email: Xu Tingfa (ciom_xtf1@bit.edu.cn), Li Jianan (lijianan@bit.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20