In visual positioning system, the accuracy of image matching directly affects the accuracy of the whole positioning system. In this paper, an image matching algorithm based on multi-level FAST (MFAST) and random sampling consistency (RANSAC) algorithm with optimized sampling is proposed for solving the problem of high mismatch rate in image matching. First, the MFAST algorithm is used to extract the corner points, and the speeded up robust feature (SURF) algorithm is used to determine the main direction to generate feature descriptors. Then, in the framework based on RANSAC algorithm, improved weighted K-nearest neighbor (PTM-DWKNN) classification method is utilized to calculate the best model parameters by selecting a new sample set, thereby eliminating the mismatch points. Simulation results confirm the superiority of the proposed method in comparison with the classic ones in real-world scenarios. The proposed algorithm can effectively eliminate mismatched points, improve the matching accuracy of the image, and meet the real-time requirements.