A multi-source image matching method named PCMM-Net was proposed to address the problem of unmatched keypoints resulting from the different imaging mechanisms of visible and infrared images. Firstly， a U-Net model with a policy gradient mechanism was introduced as the baseline model to extract keypoints from the images. This foundational model transformed pixel values into normalized probabilities， serving to filter out low-texture areas. This process enabled the network to focus on and learn keypoints that were both reliable and repeatable. Then， to address the radiance discrepancies between visible images and infrared images， a pseudo-twin network was employed to extract similar features from local image patches. Finally， a fusion layer was proposed to integrate similar features and features from keypoint detectors， generating descriptors suitable for multi-source image matching. The proposed algorithm was validated for matching performance on the VEDAI near-infrared dataset and the MTV thermal infrared dataset. Experimental results demonstrate that the proposed algorithm achieves average matching accuracies of 97.77% and 95.88% on the VEDAI and MTV datasets， respectively. Compared to the DALF algorithm， the average matching accuracies are improved by 2.26% and 14% on VEDAI and MTV datasets. Experimental results show that the algorithm has better matching effect and improves the accuracy of matching.