To overcome the shortcoming of traditional image fusion method based on multi-scale transform, a novel adaptive image fusion algorithm based on shift-invariant shearlet transform (SIST) is proposed. Firstly, the SIST is utilized to decompose the source images, and the low frequency sub-band coefficients and directional bandpass sub-band coefficients are obtained. Secondly, for the low frequency sub-band coefficients, the singular value decomposition method in the gradient domain is used to estimate the local structure information of image, and a variable weights fusion scheme based on the sigmoid function and the extracted features is presented, while for the directional bandpass sub-band coefficients, a scheme based on the Sum-modified-Laplacian (SML) combined with the weighted average scheme is presented. Finally, the fused image is obtained by performing the inverse SIST on the combined coefficients. The experimental results show that the proposed approach can significantly outperform the conventional image fusion methods in terms of both objective evaluation criteria and visual quality.