A moving object detection algorithm based on Learning Vector Quantization (LVQ) is presented. By training samples, the threshold vector of extracting the moving objects has the self-adaptive ability. The input vector includes components of YCbCr color space and direction feature of Gray Level Co-occurrence Matrix (GLCM). These two features are integrated to the algorithm, which has the efficiency of inhibiting the disturbance of background brightness variation. Experiment results indicate that the moving objects can be extracted correctly by using the algorithm, even if the complex background has an acute brightness variation.