Sigma-Delta background estimate is an effective moving object detection algorithm, which is able to achieve a good detection effect with a smaller amount of calculation. However, the algorithm can't clear the target image in the initial background model for the reason that there is no background model initialization. To solve this problem, it is proposed to initialize the background model based on method of mean value, which can clear the target image and get a clean background model. In order to improve the adaptability, the algorithm adopts a way to update the background model based on the idea of random update of Visual Background extractor (ViBe). In the aspect of foreground detection, a new Sigma-Delta algorithm based on RGB color space is raised to reduce the noise and improve the quality of detection. The experimental result demonstrates that the improved algorithm can obtain a clean background model and is adaptable to complex scenes. The images detected by the algorithm have less noise points and better detection performance.