Aiming at the problem of large number of image feature mismatches, an oriented FAST and rotated BRIEF (ORB) image feature point matching algorithm based on sparse optical flow method was proposed. Firstly, the feature points were violently matched to obtain the initial matching point set. Then the sparse optical flow method was used to calculate the feature point motion vector, and the two-dimensional coordinate position of the feature point in the image to be matched was estimated, and the feature points far from the estimated position were eliminated. Finally, the random sampling consistency algorithm was used to perform geometric verification to further optimize the matching result, so as to eliminate the effect of mismatching. The experimental results show that compared with the ORB operator, SIFT operator and SURF operator, the accuracy of this algorithm is increased by 21.6% on average, and the accuracy of random sample consensus FAST and rotated BRIEF (RANSAC-ORB) algorithm is increased by 2% on average; moreover, the algorithm has good versatility for image illumination transformation, perspective transformation, fuzzy transformation, rotation and scaling transformations and illumination variations.