An algorithm based on SIFT and compressive features is proposed to develop effective and efficient appearance models for robust object tracking due to factors such as pose variation, illumination change, occlusion, and motion blur. The algorithm describes the target and background with compressive features which labeled as positive and negative specimens sampling from frames. The tracking task is formulated as a binary classification via a SVM classifier with online update in the compressed domain. In new frame, utilize the classifier to obtain the target’s position. Meanwhile, introduce SIFT to solve the target size change, so as to achieve adaptive template size. The proposed tracking algorithm performs favorably against state-of-the-art algorithms on challenging sequences in terms of efficiency, accuracy and robustness.