Correlation filters have demonsrtated excellent performance in real-time and accurate tracking of video signals in recent years, which have attracted the attention of many scholars. However, when the appearance of complex scenes changes greatly, the tracking effect of the filter is easy to be unstable, and the ability to estimate scale changes needs improvement. The model update process easily introduces untrustworthy samples, which is not conducive to track the target accurately. Therefore, a combination of a color histogram model and correlation filtering-discrimination model is used to track objects accurately, and then scale samples are extracted from the predicted position to train the kernel scale correlator. Considering the problem of interference information getting introduced in the model update process, the model is updated when the response confidence attains a certain threshold. The proposed algorithm in the tracking data set (OTB-2015) performed well with a success rate and a precision score of 0.694 and 0.794, respectively.