With the development of image fusion technology, various fusion methods are proposed. However, in many cases, the final fused image is observed by human eyes. As a result, fused image quality assessment based on human visual system is particularly important. In order to obtain the objective quality evaluation by simulating human eye perception, a novel method is proposed. First, the source images and the fused image are transformed into CIE L*a*b* color space respectively. Then in the frequency domain these images are filtered by the contrast sensitivity function. The richness of details of the fused image can be predicted by calculating the chromatism of it. With the increasement of the chromatism value, the information details also increase. While the correlation coefficient is evaluated at the same time by calculating the chromatism among the source images and the fused image. The higher the correlation coefficient, the better the fusion algorithm. With these two constrained parameters, a judgement can be given to different evaluation algorithms. Experiments show that the results of the proposed method are consistent with the human eye observation.