In image matching, higher resolution of an image means higher matching accuracy and there are more features that can be extracted, but the matching time is longer.Thus, the image matching algorithm based on SIFT is improved in this paper.Rough matching is performed on the feature points of the second group of Gaussian pyramids.The improved RANSAC based on GMS is used to eliminate the mismatched points and obtain the affine matrix, and the matching area is roughly determined.During the process of accurate matching, the remaining feature points in the same matching area are filtered, and the matching pairs are filtered by using the previous affine matrix, and the least squares algorithm is used to eliminate the mismatched points again and quickly solve the final affine matrix.The experimental results show that this method greatly shortens the matching time and improves real-time performance under the condition of guaranteeing the sub-pixel accuracy.