In remote sensing image fusion, these different object regions have different demands on the spatial and spectral resolution. A new adaptive fusion method based on saliency analysis for the remote sensing image is proposed. The multi-scale spectral residual (MSR) analysis model is introduced to divide the remote sensing image into the salient regions with the rich texture and edge information and the non- salient regions with the less texture and edge information. Two different fusion algorithms are applied to the two kinds of areas. The window mean intensity hue saturation (IHS) transform can be used in salient regions with the rich texture and edge information, such as urban areas and roads, to retain more texture and edge information while the wavelet transform can be used in non-salient areas, such as farmlands and mountains, to retain more spectral information. The experimental results show that the new algorithm can obtain the salient regions as urban areas and roads with more texture and edge information and non- salient areas as farmlands and mountains with more spectral information. The new method has more theory and application value in further research of remote sensing image fusion.