Aiming at the problems that the traditional visual background extractor (ViBe) algorithm cannot reflect the scene changes in time and has poor adaptability to dynamic scenes, an improved ViBe algorithm was proposed by using randomly selected background samples and 24 neighborhood method to obtain the initial background, which could accelerate the "ghost" ablation. The average adaptive threshold calculation method was adopted to improve algorithm adaptability to external dynamic environment and illumination changes in combination with OTSU method and uniformity measurement method, which retained effective pixels to the greatest extent. In the update phase, the adaptive update factor was introduced, which could effectively reduce the misjudgment probability, so as to enhance algorithm robustness. Finally, the target was more complete through morphological processing and filtering. The standard dataset video was applied to test and compare the improved algorithm. Compared with kernel density estimation (KDE) algorithm, Gaussian mixed model (GMM) algorithm and traditional ViBe algorithm, the indexes of the improved algorithm were greatly improved. The accuracy is improved by 30.44%, 40.72% and 20.95%, respectively and the percentage of wrong classifications is reduced by 43.28%, 40.59% and 29.43%, respectively.