In order to robustly track the multi-degree-of-freedom moving objects in video sequences at a complex background, a tracking algorithm for multi-degree-of-freedom moving objects was proposed based on the particle filter principle. Firstly, the similarity of a target model and a candidate model was taken as the structural basis of observation by using mean shift algorithm. Then, based on the kernel-color histogram, the center position of the object and the covariance matrix that described the shape of the object were updated to adjust kernel-bandwidth and modify the size of tracking window,then to implement the tracking for multi-degree-of-freedom moving objects. In particle filter, the number of particles is to be 100, the variance of coordinate components is 5 in the covariance matrix,and the variance of scale and angle components is 0.1. Tracking experiments for various objects in different scenarios show that the proposed algorithm can track multi-degree-of-freedom moving objects steadily, and can adapt to the change of scales and angles for objects.