In order to detect abnormal behaviors in surveillance video accurately and efficiently, a weak supervised abnormal behavior detection method based on improved yolov3 is proposed. Firstly, the multi-scale fusion method is used to improve the YOLOv3 network, and the improved yolov3 is used to complete the target detection in the video, which improves the computational efficiency and the universality of the method. Then, the large-scale optical flow histogram descriptor (LSOFH) is proposed to describe the target behavior by using the optical flow which can effectively capture the motion information, so as to better extract the abnormal behavior features. Finally, the least squares support vector machine (LSSVM) is trained to identify abnormal behaviors in surveillance video. Based on MATLAB simulation platform, the proposed method is verified by experiments. The results show that, compared with other methods, the proposed method performs best on the UCSD data set, UMN data set and subway exit data set, that is, the area under the curve (AUC) is the largest, the equal error rate (EER) is the lowest, and the detection rate is the highest. It has a good application prospect.