Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition

Qinghui Li; Aihua Li; Tao Wang; Zhigao Cui

doi:10.3788/AOS201838.0615002

Acta Optica Sinica, Volume. 38, Issue 6, 0615002(2018)

Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition

Qinghui Li^*, Aihua Li, Tao Wang, and Zhigao Cui

Author Affiliations

Academy of Operational Support, Rocket Force Engineering University, Xi’an, Shaanxi 710025, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(8)

Fig. 1. (a) Original video frames; (b) optical flow images; (c) sequential optical flow images

Download full size

Fig. 2. Double-stream network framework fusing appearance information and motion information

Download full size

Fig. 3. Results of different subsequence lengths

Download full size

Table 1. Recognition accuracy of HMDB51%
View table
Table 1. Recognition accuracy of HMDB51%
Method Split 1 Split 2 Split 3 Average
SI 49.1 50.6 49.6 49.8
SOF 55.2 53.4 54.7 54.4
DI 50.7 52.5 53.1 52.1
SOFI 57.8 58.4 57.2 57.8
SOFI+DI 58.1 58.9 58.4 58.5
SOFI+SI 63.3 61.8 62.5 62.5

Table 2. Recognition accuracy of UCF101%
View table
Table 2. Recognition accuracy of UCF101%
Method Split 1 Split 2 Split 3 Average
SI 81.4 80.5 81.9 81.3
SOF 79.3 81.5 79.5 80.1
DI 83.4 83.9 82.6 83.3
SOFI 85.8 86.1 85.2 85.7
SOFI+DI 87.7 86.9 87.3 87.3
SOFI+SI 89.6 90.9 90.3 90.3

Table 3. Recognition accuracy of different convolutional networks%

View table

Table 3. Recognition accuracy of different convolutional networks%

Network	HMDB51	UCF101
Spatial stream	41.6	81.2
Temporal stream	54.3	75.6
Original double-stream	59.4	88.0
Appearance stream	43.4	82.3
Motion stream	55.4	79.1
ST-ResNet	65.6	92.7
A&STM stream	64.9	90.1
LTM stream	57.8	81.7
Proposed double-stream	72.6	94.8

Table 4. TOP10 categories of accuracy improvement%

View table

Table 4. TOP10 categories of accuracy improvement%

HMDB51		UCF101
Action category	Increment	Action category	Increment
Cartwheel	35.6	IceDancing	24.6
Climb_stairs	32.3	Hammering	22.3
Swing_baseball	31.7	FloorGymnastics	17.8
Hit	30.0	JumpRope	17.2
Handstand	29.6	Fencing	16.4
Smoke	26.5	BrushingTeeth	13.9
Drink	24.3	Skiing	12.8
Draw_sword	19.8	Nunchucks	11.6
Shoot_ball	17.4	CricketBowling	11.2
Wave	16.9	HighJump	10.7

Table 5. Comparison of recognition accuracy for different methods%
View table
Table 5. Comparison of recognition accuracy for different methods%
Method HMDB51 UCF101
Shallow IDT+FV^[15] 57.2 84.8
IDT+HSV^[16] 61.1 87.9
MoFAP^[17] 61.7 88.3
Deep CNN-hid6+IDT^[18] - 89.6
TDD+IDT^[19] 65.9 91.5
TSN^[8] 71.0 94.0
I3D+Double-stream^[20] 66.4 93.4
SOFI+Double-stream 72.6 94.8

Tools

Get Citation

Copy Citation Text

Qinghui Li, Aihua Li, Tao Wang, Zhigao Cui. Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition[J]. Acta Optica Sinica, 2018, 38(6): 0615002

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites