A new color image assessment method is proposed to solve the problem of neglecting color information and a poor consistent behavior with the human visual system in traditional image quality assessment methods. The human eye sensitive image structure is enhanced and full color information is used in this method. Three important parts are taken into account in this structure, which are detail information, luminance information and color information. Quaternion is taken as a tool to perform the task. A quaternion matrix is constructed to evaluate color image quality. Then singular value decomposition is performed on the quaternion matrix. Max singular value is used to describe image structure information. Numerical results are obtained by using distortion map. 982 images in LIVE database including five types of distortion are used to investigate the behaviors of the proposed method. The nonlinearity property of the proposed method in the cross-distortion experiment is that the Root Mean Square Error(RMSE) value is 9.176,and Spearman Rank Order Correlation Coefficient(SROCC) value is 0.929 6, however, those of from Structural Similarity Index( SSIM) method are 9.299 and 0.925 6, respectively. The results show that proposed method is more consistent than the traditional methods because of the considering of more image properties and using a quaternion matrix.