To overcome the problem that the common subtractive clustering object locating method could not acquire video object’s scale and orientation parameters, an improved subtractive clustering object locating algorithm, that is called the scale and direction adaptive locating of video moving objects with subtractive clustering algorithm, is proposed. The proposed method first uses common subtractive clustering locating method to acquire video object position and moving object number of each frame. Then, the proposed algorithm uses fuzzy C-means clustering algorithm to further cluster the moving object foreground pixel samples. Finally, matrix analysis theory for calculating eigenvalues and eigenvectors of covariance matrix is applied to compute the object scale and orientation parameters. Experiment results show that the proposed object locating algorithm could obtain much more reasonable video object locating results.