Journal of Optoelectronics · Laser, Volume. 33, Issue 2, 149(2022)

Recognition of students′ online classroom action based on spatio-temporal graph convolutional network

HU Jinlin, QI Yongfeng*, and WANG Jiaying
Author Affiliations
  • [in Chinese]
  • show less

    In order to effectively identify students′ online classroom action,a human skeleton action recognition model integrating global attention mechanism and spatiotemporal convolution network is proposed.Firstly,a global attention module is added between the spatial graph convolutional network and the temporal convolutional network of the Spatiotemporal graph convolutional neural network,and the spatial feature map output by the spatial graph convolutional network is used as the input of the attention module;Secondly,average pooling and maximum pooling operations according to the time dimension are introduced to increase the ability of the model to learn global feature information.Finally,three spatiotemporal graph convolutional neural networks and class activation map (CAM) added to the attention mechanism are used to construct a rich activation map convolutional network with stronger ability to recognize occlusion data (RA-GCNv2-A) model,and realize student online classroom action recognition function through transfer learning.Experimental verification is performed on the NTU-RGB+D and NTU-RGB+D120 two datasets.Compared with the RA-GCNv2 model,the recognition accuracy on the NTU-RGB+D dataset is increased by 1.3% (cross-subject,CS),1.2% (cross-view,CV),the recognition accuracy on the NTU-RGB+D120 dataset is increased by 1.6% (cross-subject,CSub),1.4% (cross-setup,CSet) respectively.The experimental results show that the proposed method is an effective way to recognize students′ online classroom action.

    Tools

    Get Citation

    Copy Citation Text

    HU Jinlin, QI Yongfeng, WANG Jiaying. Recognition of students′ online classroom action based on spatio-temporal graph convolutional network[J]. Journal of Optoelectronics · Laser, 2022, 33(2): 149

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Received: Jun. 3, 2021

    Accepted: --

    Published Online: Oct. 9, 2024

    The Author Email: QI Yongfeng (qiyf@nwnu.edu.cn)

    DOI:10.16136/j.joel.2022.02.0384

    Topics