For the correlation filtering tracking algorithm is not robust enough and cannot adapt to scale changes due to the boundary effect, an improved correlation filtering tracking algorithm based on double model was proposed. The target tracking consisted of position prediction and scale prediction. In the position prediction stage, the samples were enhanced to make them more consistent with the actual scene. Then, the solution was obtained using the alternating direction method of multipliers, and the estimated target position was achieved. For scale prediction, a multiscale pyramid was constructed to train the scale filter, and then the target scale was acquired. The final tracking result was determined by both the target position and scale. Finally, an occlusion criterion was introduced to determine whether the model is updated or not. Compared with the classical correlation filtering tracking algorithm, the proposed algorithm boosts the tracking success rate by 18% and tracking accuracy by 11%. The algorithm can track the target stably even when the target is occluded and its scale changes.