In order to improve the fusion quality of multispectral image and panchromatic image, a remote sensing image fusion algorithm was proposed based on Non-subsampled Dual-tree Complex Contourlet Transform(NSDTCT) and sparse representation. Firstly, the Intensity-Hue-Saturation(IHS) transform was applied to the multispectral image. Then, the histogram matching and smoothing filter-based intensity modulation were used to handle intensity component and panchromatic image. Secondly, the NSDTCT was employed to decompose the new intensity component and panchromatic image, and the low frequency coefficients and high frequency coefficients were obtained. For the low frequency coefficients, a fusion method based on sparse representation was presented, and the fused coefficients were obtained by combining spatial frequency with l1-norm maximum. For the high frequency coefficients, the sum-modified Laplacian was used for the external input of Pulse Coupled Neural Network(PCNN), and a fusion method based on the theory of improved PCNN was presented. Finally, the fused image was obtained by inverse NSDTCT and inverse IHS transform. The experimental results show that the proposed algorithm can improve the spatial resolution and maintain the spectral characteristics simultaneously, and outperforms other classical fusion algorithms in terms of both the visual quality and objective evaluation.