The imaging process of polarization images in the natural environment is easily affected by noise, which not only causes the acquired relevant polarimetric parameters to deviate from their real values but also affects the further processing of subsequent polarization information. Due to nonlinear operations, polarimetric parameters such as the degree of polarization (DoP) and the angle of polarization (AoP) are easily distorted by noise, especially in photon-starved environments. Therefore, effective denoising is crucial to polarimetric imaging. The denoising method based on deep learning can significantly remove the influence of noise on polarization images. However, the performance of current supervised algorithms is highly dependent on the labeled dataset, and high-quality polarization labels are difficult to obtain in practical applications, which limits the application of the existing methods. Therefore, this paper proposes a polarization image denoising method based on unsupervised learning. This method breaks the restriction that supervised learning-based deep learning requires strictly paired images and uses unpaired polarization images to train a polarization-specialized cycle generative adversarial network (CycleGan). The method in this paper are of great significance to the application of polarimetric imaging in complex noise environments.

In the proposed CycleGan structure, the discriminator for the input domain is removed, and two discriminators for polarimetric parameters are added. In the structure of generators, the residual dense block (RDB) is introduced to extract abundant local features via densely connected convolutional layers, and PatchGANs are adopted for discriminators, which can work on arbitrarily sized images and grow the receptive field after each convolution layer. In addition, a batch normalization (BN) layer and a ReLU layer are added right after each convolutional layer to accelerate network training. Furthermore, a cycle consistency loss is maintained to keep the consistency between input and output, and two cycle gradient losses are introduced for the degree of linear polarization (DoLP) and AoP to preserve the variations of polarization information. With the help of the designed network structure and the polarization-based loss function, the network trained by unpaired polarization images can statistically learn the mapping between noisy and clean images.

Experiments show that the network can effectively suppress the noise of polarization images in different indoor and outdoor environments and recover DoLP and AoP. The ablation experiment proves the effectiveness of additional polarization discriminators. With two discriminators, the network accurately recovers both DoLP and AoP images (Fig. 3) and achieves the highest PSNR/SSIM value among different network structures (Table 1). Compared with other methods, the unsupervised method has the best performance in terms of intensity, DoLP, and AoP images (Fig. 4). The average PSNR and SSIM of indoor images illustrate that the method has advantages in the reconstruction of DoLP images (Table 2). Several groups of experiments on different materials, including resin, fabric, wood, and plastic, are conducted to verify the universality of the proposed method. The denoised results reveal that the proposed method can suppress the noise of these materials for polarization information (Fig. 5). Finally, experiments with outdoor noise polarization images are carried out to verify the robustness of the method. Compared to the supervised method, the unsupervised method does not see dramatical performance degradation when applied to different environments (Fig. 6), which is important for the application of polarization imaging in realistic environments.

This paper proposes a polarization image denoising method based on unsupervised learning. On the basis of the CycleGan model, a structure of generative adversarial network suitable for polarization image denoising is designed. Through an unsupervised training network with unpaired images, a denoising network model that can effectively remove the noise of polarization images and restore polarization information is obtained. Experiments with indoor images are conducted to test the method, and qualitative and quantitative evaluations are given. The experimental results show that this method can achieve the same performance as the supervised learning method in indoor image denoising and can effectively restore polarization information, especially in DoLP image restoration. Furthermore, the polarization images of different materials are tested. The results reveal that this method has good generalization and can effectively recover the polarization information of different materials. In addition, the outdoor images are also tested, and a qualitative evaluation is presented. The experimental results suggest that this method can effectively remove the noise of indoor and outdoor images and restore real polarization information when indoor images are used as the training set. The models and methods proposed in this study can be extended to other applications. For example, they can be used to study polarization image denoising and polarization information recovery in extreme environments (e.g., night, low light).