Aiming at insufficient details information extraction and higher complexity algorithm when the infrared and visible light image fusion is processed, an infrared and visible image fusion algorithm based on the Non-subsampled Shearlet Transform (NSST) and the improved Non-negative Matrix Factorization (NMF) is proposed. Making use of NSST to decompose source images on multi-direction and multi-scale sparse, low-frequency components and high frequency components are obtained. The fusion method of the improved NMF is adopted in the low frequency subband. The fusion rule for the combination of the Laplace energy and visual sensitivity coefficient is used to the high frequency components. Finally, the fusion image is obtained after executing the NSST inverse transformation. Experimental results show that the fusion method can not only guarantee the definition of the fused image, but also shorten the running time of the algorithm.