To solve the problems of difficulty of feature fusion and low efficiency of joint recognition in color image and depth image(RGB-D), a new algorithm of RGB-D scene image fusion is proposed based on K singular value decomposition (KSVD) and maximum correlation minimum redundancy atoms (mRMR) principle. Firstly, the features of the sparse KSVD image and the dictionary atoms corresponding to the sparse coefficients are used as the parameters of feature fusion to fully express the whole information of image. Secondly, the mRMR principle of mutual information is used to determine the characteristic atom combination which has minimum dimensions and minimum correlation among different dimensions. Finally, the sparse coefficients are fused by the maximization principle to obtain the effective information fusion between two images. Experimental results show that the proposed algorithm has advantages over principal component analysis-K singular value decomposition and non-subsampled contour transform-K singular value decomposition fusion algorithms in terms of information entropy, mutual information and edge preservation, which improves recognition accuracy and success rate of the image targets effectively.