For the fusion of infrared and visible images, it is easy to produce problems such as missing detail information and suppressing less noise. In this paper, an improved fusion algorithm is proposed by applying the characteristics of a rolling guidance filter, which preserves edge and local brightness. First, the input images are decomposed into base and detail layers by mean filtering. Second, the saliency maps of the input images are obtained by combining the rolling guidance and Gaussian filters. The weight maps are then optimized by guided filters of different scales. The optimized maps are used to instruct the fusion of the base and detail layers. Finally, the fused image is reconstructed by combining the merged sub-images. The method of this paper is superior on the six indicators, such as nonlinear correlation information entropy and phase consistency, compared to the classical methods such as non-subsampled contourlet transform(NSCT), image fusion with guided filtering(GFF), and two-scale image fusion based on visual saliency(TSIFVS).