Acta Optica Sinica, Volume. 38, Issue 6, 0615002(2018)
Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition
In order to effectively utilize the long-term temporal information of video for improving the accuracy of action recognition, a new recognition approach is proposed based on the sequential optical flow image and double-stream convolutional neural networks. Firstly, the Rank support vector machine (SVM) algorithm is used to compress the continuous optical flow frames into a single sequential optical flow image to realize the modeling of the long-term temporal structure of video. Secondly, we design a double-stream convolutional networks containing appearance and short-term motion stream and long-term motion stream. It takes the stacked RGB frames and the sequential optical flow images as input to extract the appearance and short-time motion information and the long-time motion information of the video. Finally, the linear SVM is adopted to integrate C3D descriptor and VGG descriptor for action recognition. The experimental results on HMDB51 and UCF101 datasets show that the proposed approach improves the action recognition accuracy effectively by using the spatial information and the temporal motion information.
Get Citation
Copy Citation Text
Qinghui Li, Aihua Li, Tao Wang, Zhigao Cui. Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition[J]. Acta Optica Sinica, 2018, 38(6): 0615002
Category: Machine Vision
Received: Nov. 27, 2017
Accepted: --
Published Online: Jul. 9, 2018
The Author Email: Li Qinghui (lqhui1212@126.com)