A fast image matching algorithm based on a multiscale dense structure feature has been proposed for matching multi-sensor images. In this method, the Gabor filter is employed for generating the structure response of the image. Then, the multiscale structure feature is combined on the basis of the major orientation response. Subsequently, fast Fourier transform is employed to calculate the convolution for each feature component image in the frequency domain. Finally, the similarity between images is estimated based on the sum of the convolutions, and the position with maximum similarity is outputted as the matching result. The proposed algorithm can effectively adapt to non-linear intensity variation between a multi-sensor image and noise distortion. In the experiments, a dataset consisting of optical, infrared, and synthetic aperture radar images was used for evaluating the proposed algorithm and other existing algorithms. The results indicate that the average error matching rate of the proposed algorithm is the lowest among the investigated algorithms and it has a distinct advantage in terms of computational performance.