Journal of Optoelectronics · Laser, Volume. 33, Issue 2, 149(2022)
Recognition of students′ online classroom action based on spatio-temporal graph convolutional network
In order to effectively identify students′ online classroom action,a human skeleton action recognition model integrating global attention mechanism and spatiotemporal convolution network is proposed.Firstly,a global attention module is added between the spatial graph convolutional network and the temporal convolutional network of the Spatiotemporal graph convolutional neural network,and the spatial feature map output by the spatial graph convolutional network is used as the input of the attention module;Secondly,average pooling and maximum pooling operations according to the time dimension are introduced to increase the ability of the model to learn global feature information.Finally,three spatiotemporal graph convolutional neural networks and class activation map (CAM) added to the attention mechanism are used to construct a rich activation map convolutional network with stronger ability to recognize occlusion data (RA-GCNv2-A) model,and realize student online classroom action recognition function through transfer learning.Experimental verification is performed on the NTU-RGB+D and NTU-RGB+D120 two datasets.Compared with the RA-GCNv2 model,the recognition accuracy on the NTU-RGB+D dataset is increased by 1.3% (cross-subject,CS),1.2% (cross-view,CV),the recognition accuracy on the NTU-RGB+D120 dataset is increased by 1.6% (cross-subject,CSub),1.4% (cross-setup,CSet) respectively.The experimental results show that the proposed method is an effective way to recognize students′ online classroom action.
Get Citation
Copy Citation Text
HU Jinlin, QI Yongfeng, WANG Jiaying. Recognition of students′ online classroom action based on spatio-temporal graph convolutional network[J]. Journal of Optoelectronics · Laser, 2022, 33(2): 149
Received: Jun. 3, 2021
Accepted: --
Published Online: Oct. 9, 2024
The Author Email: QI Yongfeng (qiyf@nwnu.edu.cn)