Aiming at the problems that moving object detection in video sequence is easily interfered by environmental noise and the object contour is difficult to extract, this paper proposes a multi-moving object detection algorithm based on the improved model of edge multi-channel gradient. First, the Canny operator is used to obtain the edge information of the object in the video sequence, and a multi-channel gradient model of time, space and color is established on the object edge according to the constant characteristics of human visual color; Then the model is used to obtain the motion state description information of the object edge pixel and achieve the separation of the background edge and the edge of the moving object; Finally, the discontinuity edge pixels are associated with the motion state of their neighboring points to connect the discontinuity edges of the object, which achieve the complete extraction of the contour of the moving object; The connected contour is morphologically processed to segment the object. Experimental results show that, compared with similar algorithms, the algorithm has superior real-time performance, accuracy, and robustness in moving object detection.