Aiming at the problems of strong background noise and indistinct detail contour features in polarization fusion image of dim and weak targets， considering to the difference of gray level between Gaussian background noise and target contour in degree of polarization image， a denoising algorithm based on noise template threshold matching is proposed. Analyzing the gray correlation between the sub-image of pure noise and the sub-image of target contour in the angle of polarization image， a denoising algorithm based on Region Gray Correlation Matching （RGCM） is proposed， compared with the traditional denoising methods， the RGCM algorithm can effectively remove a large amount of background noise and retain the target contour features better. The experimental results of polarization image fusion further prove that the algorithm proposed in this paper can not only improve the contrast of dim and weak targets but also significantly improve the image quality in polarization fusion enhancement image.