Cross-Modal Optical Information Interaction and Template Dynamic Update for RGBT Target Tracking Method

Jianming Chen; Dingjian Li; Xiangjin Zeng; Zhenbo Ren; Jianglei Di; Yuwen Qin

doi:10.3788/AOS231907

Acta Optica Sinica, Volume. 44, Issue 7, 0715001(2024)

Cross-Modal Optical Information Interaction and Template Dynamic Update for RGBT Target Tracking Method

Jianming Chen^1,2, Dingjian Li¹, Xiangjin Zeng^1,2, Zhenbo Ren³, Jianglei Di^1、*, and Yuwen Qin^1,2、**

¹Key Laboratory of Photonic Technology for Integrated Sensing and Communication, Ministry of Education, Guangdong Provincial Key Laboratory of Information Photonics Technology, School of Information Engineering of Guangdong University of Technology, Institute of Advanced Photonics Technology, Guangzhou 510006, Guangdong , China

²Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, Guangdong , China

³Key Laboratory of Light-Field Manipulation and Information Acquisition, Ministry of Industry and Information Technology, Shaanxi Key Laboratory of Photonics Technology for Information, School of Physical Science and Technology, Northwestern Polytechnical University, Xi an 710129, Shaanxi , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(14)

Fig. 1. Overall network structure of SiamCTU

Download full size

Fig. 2. Model structure of target tracking method based on Siamese network

Download full size

Fig. 3. Residual structure of ResNet

Download full size

Fig. 4. Schematic diagram of cross correlation operations

Download full size

Fig. 5. Structure diagram of feature interaction module

Download full size

Fig. 6. Comparison of SiamCTU with 10 advanced trackers on GTOT dataset. (a) PR; (b) SR

Download full size

Fig. 7. Comparison of SiamCTU with 10 advanced trackers on RGBT234 dataset. (a) PR; (b) SR

Download full size

Fig. 8. Experimental results on RGBT234 dataset based on 12 challenge attributes. (a1)(a2) Background clutter; (b1)(b2) camera moving; (c1)(c2) deformation; (d1)(d2) fast moving; (e1)(e2) heavy occlution; (f1)(f2) low illumination; (g1)(g2) low resolution; (h1)(h2) motion blur; (i1)(i2) no occlution; (j1)(j2) partial occlution; (k1)(k2) scale variation; (l1)(l2) thermal crossover

Download full size

Fig. 9. Comparison of SiamCTU with 7 advanced trackers on LasHeR dataset. (a) PR; (b) SR

Download full size

Fig. 10. Visual tracking comparison results on GTOT and RGBT234 datasets. (a) LightOcc sequences; (b) cycling sequences; (c) night2 sequences; (d) baketballwaliking sequences

Download full size

Table 1. PR and SR of the SiamCTU compared with the 10 most advanced target trackers based on challenge attributes in the GTOT dataset (represented by PR/SR)

View table

Table 1. PR and SR of the SiamCTU compared with the 10 most advanced target trackers based on challenge attributes in the GTOT dataset (represented by PR/SR)

Publisher information	Trackers	OCC	LSV	FM	LI	TC	SO	DEF	All
ECCV 2020	CAT^［16］	89.9/69.2	85.0/67.9	83.9/65.4	89.2/72.3	89.9/71.0	94.7/69.9	92.5/75.5	88.9/71.7
IEEE TIV 2020	FANet^［31］	86.4/70.3	81.6/68.2	80.1/65.3	89.9/73.5	90.4/72.3	94.3/70.8	94.6/77.6	89.1/72.8
CVPR 2020	CMPP^［30］	94.7/71.7	91.2/69.9	91.7/68.6	92.4/74.3	93.8/72.9	98.0/72.5	94.6/78.8	92.6/73.8
IJCV 2021	ADRNet^［32］	88.5/69.6	86.1/70.6	83.4/67.1	92.2/75.9	91.1/73.6	94.7/72.1	94.5/77.5	90.4/73.9
IEEE TIP 2022	M5L^［33］	87.1/66.6	91.0/70.2	89.4/68.5	91.7/73.0	89.2/69.5	96.0/70.2	92.2/74.6	89.6/71.0
AAAI 2022	APFNet^［17］	90.3/71.3	87.7/71.2	86.5/68.4	91.4/74.8	90.4/71.6	94.3/71.3	94.6/78.0	90.5/73.7
IEEE TCSVT 2022	SiamCDA^［19］	82.2/69.4	91.5/74.8	86.6/72.0	92.4/76.4	82.6/68.5	87.4/69.1	87.9/72.7	87.7/73.2
CVPR 2022	HMFT^［13］	88.5/72.0	89.3/75.2	85.8/74.1	94.6/76.7	89.6/73.0	92.8/71.3	94.4/74.7	91.3/74.9
IEEE TITS 2023	DFNet^［34］	88.7/68.9	84.2/69.7	81.4/64.4	89.6/73.3	88.6/71.5	94.3/71.3	92.8/74.8	88.1/71.9
Inf Fusion 2023	DFAT^［35］	86.3/68.7	92.4/75.0	89.1/74.0	92.2/74.1	89.1/70.7	94.4/71.9	91.9/73.5	89.3/72.3
	Our	91.5/72.1	94.2/75.9	90.2/73.3	95.8/76.6	92.7/74.0	92.8/71.7	94.1/74.8	94.0/75.6

Table 2. Experimental results of tracking efficiency comparison based on GTOT dataset
View table
Table 2. Experimental results of tracking efficiency comparison based on GTOT dataset
Metric CMPP APFNet SiamCDA CAT M5L Ours
FPS 1.3 1.4 37.0 20.0 9.7 30.0
PR/SR 92.6/73.8 90.5/73.7 87.7/73.2 88.9/71.7 89.6/71.0 94.0/75.6

Table 3. Results of ablation experiments on 3 baseline datasets (PR/SR)

View table

Table 3. Results of ablation experiments on 3 baseline datasets (PR/SR)

Method	GTOT	RGBT234	LasHeR
Baseline	88.2/72.6	75.3/53.9	54.8/42.6
Baseline+CB	90.7/72.4	75.4/54.1	55.4/43.5
Baseline+CB+FIM	91.8/74.7	78.5/56.6	57.2/44.6
Baseline+CB+TU	91.9/73.1	76.1/54.0	57.1/44.4
Baseline+CB+FIM+TU（ours）	94.0/75.6	79.2/57.1	58.5/46.0

Table 4. Experimental results of updating weights with different templates on GTOT dataset
View table
Table 4. Experimental results of updating weights with different templates on GTOT dataset
Metric λ=0 λ=0.5 λ=0.6 λ=0.7 λ=0.8 λ=0.9 λ=1
PR/SR 71.0/58.2 86.9/72.6 91.2/73.6 94.0/75.6 91.7/74.6 91.8/74.5 91.8/74.7

Tools

Get Citation

Copy Citation Text

Jianming Chen, Dingjian Li, Xiangjin Zeng, Zhenbo Ren, Jianglei Di, Yuwen Qin. Cross-Modal Optical Information Interaction and Template Dynamic Update for RGBT Target Tracking Method[J]. Acta Optica Sinica, 2024, 44(7): 0715001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Machine Vision

Received: Dec. 11, 2023

Accepted: Jan. 25, 2024

Published Online: Apr. 11, 2024

The Author Email: Di Jianglei (jiangleidi@gdut.edu.cn), Qin Yuwen (qinyw@gdut.edu.cn)

DOI:10.3788/AOS231907

CSTR:32393.14.AOS231907

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. PR and SR of the SiamCTU compared with the 10 most advanced target trackers based on challenge attributes in the GTOT dataset (represented by PR/SR)

Table 1. PR and SR of the SiamCTU compared with the 10 most advanced target trackers based on challenge attributes in the GTOT dataset (represented by PR/SR)

Table 2. Experimental results of tracking efficiency comparison based on GTOT dataset

Table 2. Experimental results of tracking efficiency comparison based on GTOT dataset

Table 3. Results of ablation experiments on 3 baseline datasets (PR/SR)

Table 3. Results of ablation experiments on 3 baseline datasets (PR/SR)

Table 4. Experimental results of updating weights with different templates on GTOT dataset

Table 4. Experimental results of updating weights with different templates on GTOT dataset