An image matching method based on phase feature and pyramid algorithm is proposed to improve the matching performance of heterogeneous images. Firstly, the image is processed by pyramid operation, from top matching pair to bottom matching pair based on FM phase feature. The region of interest in the image is selected. Then the image block extracted is traversed by each pixel in the region. The FM phase feature is matched with the corresponding foreign image. The point is got with the largest phase feature matching coefficient in the region of interest as the translation. Entering the lower pyramid for processing, the more accurate parameters are searched near the initial translation parameters. After traversing all pyramid levels, the final translation, magnification coefficient and rotation parameters are obtained. Through several sets of simulation experiments of infrared-visible, SAR-visible matching pairs and Gauss white noise interference experiments, it is found that the proposed method has better matching success rate, matching stability and robustness than several classical methods and the algorithms proposed in recent years.