In order to improve the quality of low-light night vision images, a color transfer algorithm based on image segmentation and local brightness adjustment was proposed. The simple linear iterative clustering was combined with K-means clustering to segment the low-light image, and the color component of matching reference image was transmitted to the sub-region of target image by using the uniformity for the brightness of each sub-region and reference image in the YCbCr color space. The contrast value in the texture feature of the target image was taken as the coefficient to adjust the brightness value of the sub-region of the target image, perform the color space conversion and display the color transfer results. A low-light image imaging system was built, and the low-light image segmentation and color transfer were completed. The results show that the improved segmentation algorithm separates different scenes in the image, and the peak signal-to-noise mean of the obtained color low-light image reaches 12.048 dB, which is 2.63 dB higher than the Welsh algorithm.