In order to adapt the correlation filter model to the change of the target appearance and improve the robustness and real-time performance of the correlation filter algorithm for visual tracking, an adaptive learning rate adjustment method for real-time tracking of a single-layer convolution filter is proposed, which is based on the relationship of the correlation filter response value, mean frame difference and the object movement displacement. This method selects the convolution features of a single convolution layer to train the correlation filter classifier that is used to predict the position of object, reducing the convolution feature dimension and improving the speed of visual tracking. Meanwhile it uses the fast-scale prediction method to estimate the object's scale, and adopts a sparse model update strategy. Besides, the Peak-to-Sidelobe Ratio (PSR) of convolutional response is used to estimate the credibility of the predicted location. The apparent change of the object is evaluated by combining the mean frame difference and the object movement displacement. And the learning rate of the correlation filter model update can be adjusted by these two terms adaptively, so that the change characteristics of the object can be quickly learned. The accuracy of visual tracking is improved by this method. The method is tested on the standard OTB-100 dataset. The results show that the average distance accuracy is 90.1%, which is better than the state-of-the-art algorithms in the experiment. And the average success rate is 79.2%, which is only smaller than the continuous convolution tracking algorithm(CCOT). But the average speed is 31.8 frames per second, nearly 30 times of the CCOT.