Aiming at the problem of low recognition speed and recognition rate of convolution neural network and graph convolution network in campus violence recognition, this paper proposes a lightweight graph convolution network combined with multi-information flow data fusion and spatio-temporal attention mechanism. The human skeleton is taken as the research object. Firstly, the multi-information flow data related to joint points and skeleton are fused to improve the operation speed by reducing the number of network parameters. Secondly, the spatio-temporal attention module based on nonlocal operation is constructed to focus on the most action discriminant nodes, and the recognition accuracy is improved by reducing redundant information. Then, the spatio-temporal feature extraction module is constructed to obtain the spatio-temporal correlation information of the concerned nodes. Finally, action recognition is realized by Softmax layer. The experimental results show that the recognition accuracy of boxing, kicking, falling, pushing, earlighting and kneeling in campus security scene is 94.5%, 97.0%, 98.5%, 95.0%, 94.5% and 95.5%, respectively, and the maximum recognition speed is 20.6 fps. Compared with the two benchmark networks on UCF101 dataset, the recognition speed and accuracy are improved, which verifies the universality of the method for other actions. Therefore, it can meet the real-time and reliability requirements of typical campus violence identification.