Natural color fusion of low-light visible images and infrared images can significantly improve abilities of human vision for situation perceiving and targets detecting in low-light environment. Sample-based color fusion is a fast, effective and real-time natural color fusion algorithm. In view of the problems of existing algorithms in construction of color look-up table and utilization of grayscale information, we propose a new color fusion algorithm of dual-band images based on CbCr look-up table. We obtain the mapping f(Y1,Y2)→(Cb,Cr) between luminance and chromaticity by using the back propagation neural network to nonlinearly fit the two-dimensional luminance vector (Y1,Y2) and the two-dimensional chromaticity vector (Cb,Cr) of image simples, and construct the CbCr look-up table based on the mapping. When color fusing, the chromaticity Cb,Cr of fused image are obtained by the CbCr look-up table and the input luminance Y1,Y2 of dual-band grayscale images. The luminance YF of fused image is obtained by the image fusion of luminance Y1,Y2 based on two-layer Laplacian pyramid transformation. The luminance Y1,Y2 are calibrated to diminish color mapping errors owing to environmental changes. The experimental results show that the fused images based on proposed algorithm have natural color, rich details and are more conducive to (hot) targets detection. The dual-band fusion results obtained by the proposed algorithm are almost as good as or even better than the fusion results by Toet method in definition, colorfulness, and mapping accuracy.