The traditional Convolutional Neural Network (CNN) algorithm-based abnormal crowd behavior detection methods adopt two-dimensional convolution kernel to extract image features, which causes such methods impossible to accurately capture the dynamic features of video streams in time series. To solve this problem, a detection method based on the combination of improved C3D network and random forest (RF) algorithm is proposed. Firstly, the C3D network with temporal feature capture capability is used to extract the Histogram of Oriented Gradient (HOG) feature of the video stream, which is used as the input of the three-dimensional convolution kernel to realize the extraction of the temporal and spatial features of the video. Secondly, a Random Forest (RF) classifier is used to replace the softmax fully connected layer to avoid cumbersome gradient calculation operations during the training process, and to reduce the requirement on the sample size of the training data set. Finally, calculation results of the examples based on the benchmark data set show that the proposed improved C3D-RF scheme maintains an accuracy rate of over 90% for the detection of abnormal crowd behaviors. Compared with traditional C3D network, Support Vector Data Description model (SVDD), Convolutional Auto-Encode (CAE) and other machine learning classifiers, the training time of the proposed scheme is shortened by more than 15.34%.