A tracking algorithm is proposed based on correlation filter fusing with keypoint matching. The object is tracked and verified by multiple support-vector-based correlation filters, respectively. Meanwhile, a database containing the keypoints of the target and the background is established and updated in real-time. After the validation of a tracking failure, the global keypoints are classified by utilizing the keypoint matching method and the target keypoints are analyzed according to these classified results. Thus the redetection results are obtained. The experimental results show that the proposed method has a better accuracy and robustness than those by the existing methods in the complex tracking scenes of motion blur, deformation, object occlusion, disappearance and so on.