Optical Instruments, Volume. 44, Issue 4, 16(2022)
A dual-branch network for action recognition
Figures & Tables(12)
Table 1. Model architecture of slow-fast network
View tableView in ArticleTable 1. Model architecture of slow-fast network
Stage Slow pathway Fast pathway Output sizes T × S2 Input data — — 64×2242 Data layer Stride 16, 12 Stride 2, 12 Slow : 4×2242 Fast: 32×2242 Conv1 1×72, 64 Stride 1, 22 5×72, 8 Stride 1, 22 Slow: 4×1122 Fast: 32×1122 Pool1 1×32 max Stride 1, 22 1×32 max Stride 1, 22 Slow: 4×562 Fast: 32×562 Res1 x 3 1×12, 64 1×32, 64 1×12, 256 3×12, 8 1×32, 8 1×12, 32 Slow :4×562 Fast :32×562 Res2 x 4 1×12, 128 1×32, 128 1×12, 512 3×12, 16 1×32, 16 1×12, 64 Slow: 4×282 Fast: 32×282 Res3 x 6 3×12, 256 1×32, 256 1×12, 1024 3×12, 32 1×32, 32 1×12, 128 Slow: 4×142 Fast: 32×142 Res4 x 3 3×12, 512 1×32, 512 1×12, 2048 3×12, 64 1×32, 64 1×12, 256 Slow: 4×72 Fast: 32×72 GAP GAP 1×2304
Table 2. Comparison of parameters and accuracies between different representations
View tableView in ArticleTable 2. Comparison of parameters and accuracies between different representations
Input Parameter amount/106 Accuracy/% CS CV J 0.89 89.1 94.9 B 0.89 87.4 94.9 2-stream 1.78 90.5 96.0 J + B (proposed) 0.89 90.6 96.0
Table 3. Accuracy comparison between different number of stacked GCN blocks
View tableView in ArticleTable 3. Accuracy comparison between different number of stacked GCN blocks
a Accuracy/% CS CV 2 73.0 80.0 3 86.5 90.3 4 90.6 96.0 5 90.6 95.9
Table 4. Accuracy change caused by the value of
a View tableView in ArticleTable 4. Accuracy change caused by the value of
a a Accuracy/% CS CV 1 94.9 98.1 1/2 95.4 98.6 1/3 95.6 99.0 1/4 95.3 99.0 1/5 95.3 98.7
Table 5. Accuracy comparison of dual branch network
View tableView in ArticleTable 5. Accuracy comparison of dual branch network
Branch Accuracy/% CS CV Skeleton 90.2 94.7 RGB 82.3 90.1 Skeleton+RGB 95.6 99.0
Table 6. Comparison of the accuracy and parameters on NTU-RGBD60 dataset
View tableView in ArticleTable 6. Comparison of the accuracy and parameters on NTU-RGBD60 dataset
Algorithm Parameter amount/106 Accuracy/% CS CV AS-GCN[ 12 ]7.40 86.8 94.2 GR-GCN[ 11 ]— 84.8 92.4 2s-AGCN[ 8 ]6.92 88.5 95.1 AGC-LSTM[ 14 ]22.81 89.2 95.0 2s-SDGCN[ 18 ]— 89.6 95.7 SGN[ 15 ]0.69 89.0 94.5 DGNN[ 24 ]8.16 89.9 96.1 Shift-GCN(2s)[ 10 ]1.48 89.7 96.0 Shift-GCN(4s)[ 10 ]2.94 90.7 96.5 MS-G3D(Joint)[ 6 ]3.20 89.4 95.0 MS-G3D(2s)[ 6 ]6.40 91.5 96.2 MST-GCN[ 1 ]2.03 91.7 95.7 Hierarchical Action 46.8 95.3 98.3 Skeleton 2.85 90.6 96.0 Skeleton + RGB 33.7 95.6 99.0
Table 7. Comparison of the accuracy on NTU-RGBD120 dataset
View tableView in ArticleTable 7. Comparison of the accuracy on NTU-RGBD120 dataset
Algorithm Accuracy/% c-sub c-set ST-LSTM[ 25 ]55.7 57.9 GCA-LSTM[ 26 ]58.3 59.2 Pose Evolution Map[ 27 ]64.6 66.9 2s-AGCN[ 8 ]82.5 84.9 Shift-GCN[ 10 ]85.9 87.6 MS-G3D[ 6 ]86.9 88.4 SGN[ 15 ]79.2 81.5 MST-GCN[ 1 ]88.5 87.8 Hierarchical Action 93.7 94.5 Skeleton 84.5 85.6 Skeleton + RGB 94.7 95.2
Table 8. Comparison of the accuracy on Kinetics 400 skeleton dataset
View tableView in ArticleTable 8. Comparison of the accuracy on Kinetics 400 skeleton dataset
Algorithm Accuracy/% Top 1 Top 5 ST-GCN[ 13 ]30.7 52.8 AS-GCN[ 12 ]34.8 56.5 2s-AGCN[ 8 ]36.1 58.7 DGNN[ 25 ]36.9 59.6 MS-AAGCN[ 7 ]37.8 61.0 MS-G3D[ 6 ]38.0 60.9 Slow-fast[ 5 ]75.6 92.1 Skeleton 37.6 60.1 Skeleton+RGB 78.1 93.3
Tools
Get Citation
Copy Citation Text
Xiaofei QIN, Rui CAI, Meng CHEN, Wenqi ZHANG, Changxiang HE, Xuedian ZHANG. A dual-branch network for action recognition[J]. Optical Instruments, 2022, 44(4): 16
Paper Information
Category: APPLICATION TECHNOLOGY
Received: Dec. 21, 2021
Accepted: --
Published Online: Oct. 19, 2022
The Author Email:
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20