This paper proposes an Attention-HardNet feature-matching algorithm in the sub-window scale space to protect the scale space’s edge and corner information and improve the reliability of the feature-matching algorithm. First, a sub-window box filter was used to construct the scale space for fully retaining the scale-space image’s edge and corner information. Second, the FAST algorithm was used to extract the scale-space feature points for increasing the speed of feature-point extraction, and the circular non-maximum suppression algorithm was used to suppress the scale-space feature points. It was optimized to improve the accuracy. Then, the SENet attention mechanism was added to the HardNet feature-extraction network to form the Attention-HardNet network, which extracted more robust 128-dimensional floating-point feature descriptors. Finally, L2 distance was used to measure the similarity of different descriptors. The image’s feature-point matching was complete. On the Oxford dataset, tests on the matching algorithm’s anti-scale, compression and illumination performances show that when compared with the commonly used matching algorithms, the matching accuracy rate of the proposed algorithm has been greatly improved. Compared with the deep learning methods such as L2net and HardNet, the matching accuracy rate of the proposed algorithm is increased by ~3%, and the speed is increased by ~10%.