In consideration of the hardware resources and improving computing speeds, a dual-channel infrared and visible image fusion system with nature color was designed based on a Digital Signal Processor(DSP) to registrate dual-channel images by combining hardware and software. A false color image fusion algorithm was used to fuse the images in a YCbCr color space, then the system chose a reference image to transfer color to the fusion result. Furthermore, a color lookup table based on statistical properties of image was used to solve the complexity computation problem in color transfer. The real-time fusion and natural colorization of infrared and visible images were realized by this system. An experiment was performed on the wareplatform. The result shows that the color-transferred image has natural color perception to human eyes, and can highlight the targets effectively with clear background details. The fusion speed of this system is up to 25 frame/s, which impletments the fusion of infrared and visible images and the natural colorization in real time and satisfies the requirements of real-time image processing. The system makes up the disadvantages of single image sensor, improves the ability to detect potential targets.