Abstract：In view of the "ghost" problem caused by the interference of foreground pixels in the model initialization of Visual Background Extractor(ViBe) algorithm, and facing the problem of updating strategies for complex contextual environments, the initial optimization of the ViBe algorithm model using the image pixel mean as a reference is proposed. At the same time, an adaptive updating strategy varied with the complexity of the background model is proposed. Firstly, the background model is initialized by using the similarity between neighborhood pixels and continuous frame background pixels. Then whether the background model is stable is determined by calculating the variance of each pixel between the samples, in order to establish the adaptive update strategy. Lastly, the moving target is extracted. The validation of CDnet2014 data set shows that this algorithm can effectively improve the phenomenon of"ghost". It not only improves the robustness of the background model in the complex environment, but also improves the objective evaluation indexes.