In order to effectively integrate the spectral saliency information of infrared and visible light images and improve the visual contrast of the fused images, a fusion method of infrared and visible light images based on weighted visual saliency and maximum gradient singular value is proposed in this paper. Firstly, the new algorithm uses the rolling guidance shearlet transform as a multi-scale analysis tool to obtain the approximate layer components and multi-directional detail layer components of the image. Secondly, for the approximate layer components that reflect the energy characteristics of the image subject, visual saliency weighted fusion is used as its fusion rule. This method uses the saliency weighted coefficient matrix to guide the effective fusion of spectral saliency information in the image, and improves the visual observation of the fused image. In addition, the principle of maximum gradient singular value is used to guide the fusion of detail layer components. This method can restore the gradient features hidden in the two source images to the fused image to a great extent, so that the fused image has clearer edge details. In order to verify the effectiveness of this algorithm, we have adopted five groups of independent fusion experiments. The final experimental results show that this algorithm has higher contrast and richer edge details. Compared with the existing typical methods, the objective parameters such as AVG, IE, Q^E, SF, SD and SCD are improved by 16.4%, 3.9%, 11.8%, 17.1%, 21.4% and 10.1%, respectively, so it has better visual effect.