Advanced Imaging, Volume. 1, Issue 2, 022001(2024)
Review of polarimetric image denoising Author Presentation , Editors' Pick
Figures & Tables(29)
![The whole framework figure. (a) Outline of the review. It consists of four parts, including noise models, generic methods, denoising methods tailored for polarimetric images, and challenges and potential directions. (b) Timeline and classification of polarimetric image denoising works in recent years, including PCA-based[34], BM3D-DoFP[35], PBM3D[32], KSVD-based[36], Adaptive BM3D[37], BM3D-CPFA[33], IPLNet[38], TL-based, PDRDN[39], DnCNN-based[40], MDU-Net[41], ColorPolorNet[42], CARDN[43], 3DCNN[44], PJNDNMNet[45], Pol2Pol[116], and ViT-based[46] methods.](/Images/icon/loading.gif){kind=icon}
Fig. 1. The whole framework figure. (a) Outline of the review. It consists of four parts, including noise models, generic methods, denoising methods tailored for polarimetric images, and challenges and potential directions. (b) Timeline and classification of polarimetric image denoising works in recent years, including PCA-based[34], BM3D-DoFP[35], PBM3D[32], KSVD-based[36], Adaptive BM3D[37], BM3D-CPFA[33], IPLNet[38], TL-based, PDRDN[39], DnCNN-based[40], MDU-Net[41], ColorPolorNet[42], CARDN[43], 3DCNN[44], PJNDNMNet[45], Pol2Pol[116], and ViT-based[46] methods.
Fig. 3. Sensitivity of different polarization parameters. (a) Relationship between mean error rates and the standard deviation. (b) The computed Stokes parameters, DoLP, and AoP, with different variances.
Fig. 4. Denoising performance of the PCA-based method. (a) Denoising results for the simulated noisy image; (b) denoising results for the real noisy image; (c) quantitative analysis results.
Fig. 5. Denoising performance of the KSVD-based method for Gaussian noise. (a) Intensity (
Fig. 7. Denoising performance for Gaussian noise. (a) Polarization components after applying PBM3D methods (
Fig. 8. Synergy between polarimetric imaging and deep learning techniques.
Fig. 9. Learning-based denoising for grayscale polarimetric images[39].
Fig. 10. Learning-based denoising for grayscale polarimetric images[43]: (a) network architecture; (b) channel attention residual dense block; (c) restored results comparison.
Fig. 11. Transfer-learning-based denoising for polarimetric images[46]: (a) the pipeline of the transfer-learning-based method; (b) visual comparison for denoising performance.
Fig. 12. Self-supervised-based denoising for polarimetric images[116]: (a) the pipeline of Pol2Pol; (b) the detail of the polarization generator; (c) the pipeline of supervised learning; (d) the pipeline of the Noise2Noise method; (e) visual comparisons of different denoising methods.
Fig. 13. Learning-based denoising for color polarimetric images[43]: (a) network architecture of ColorPolarNet; (b) restored result comparison of IPLNet and ColorPolarNet.
Fig. 14. 3D CNN-based denoising method[43]: (a) network architecture; (b) comparison of 2D and 3D convolutions; (c) restored results.
Fig. 15. Transformer-based denoising for color polarimetric images[46]: (a) the pipeline of the transformer-based method; (b)–(d) visual comparison of polarimetric color image denoising.
Fig. 16. Learning-based denoising for infrared polarimetric images[46]: (a) the architecture of PJNDNMNet; (b) visual results for infrared polarimetric image denoising.
Fig. 17. DnCNN-based denoising method for integral polarimetric imaging[40]. (a) The structure of the DnCNN. (b) Restored results of the DoLP.
Fig. 18. Deep learning for denoising in Mueller matrix images[41]. (a) Network structure; (b) the denoised images.
Table 1. Summary of Generic Deep Learning Denoising Methods.
View tableView in ArticleTable 1. Summary of Generic Deep Learning Denoising Methods.
Method Category Application Feature DnCNN[ 78 ]CNN Image denoising, super-resolution, and deblocking Feedforward denoising CNNs integrating BN, ReLU, and residual learning RDN[ 86 ,87 ]CNN Image denoising, super-resolution, artifact reduction, and deblurring An effective denoising CNN with advantages of ResNet and DenseNet U-Net-based[ 88 ]CNN Gaussian image denoising and real noisy image denoising A densely connected hierarchical image denoising network based on modified U-Net PSDNet[ 89 ]CNN Synthesized noisy image and real noisy image denoising An effective denoising CNN with parallel and serial structure MWDCNN[ 90 ]CNN Gaussian noisy image and real noisy image denoising A multi-stage image denoising CNN with the wavelet transform CTNet[ 79 ]CNN with Transformer Synthesized noisy image and real noisy image denoising Improving denoising in complex scenes by cross-transformer techniques HWformer[ 91 ]Transformer Synthesized noisy image and real noisy image denoising Long- and short-distance modeling by a heterogeneous window transformer Noise2Noise[ 92 ]CNN Image denoising, Monte Carlo image denoising, and reconstruction of MRI scan Training a blind denoiser by two noisy images from two snapshots SSNet[ 93 ]CNN Image denoising and watermark removal A perceptive self-supervised learning network for noisy image watermark removal PSLNet[ 94 ]CNN Image denoising and watermark removal Self-supervised learning network training a denoising CNN Net[ 95 ]CNN Image denoising and dehazing Unsupervised unified image dehazing and denoising network for a single image
Table 2. Comparison of KSVD, PCA, and BM3D Image Denoising Methods.
View tableView in ArticleTable 2. Comparison of KSVD, PCA, and BM3D Image Denoising Methods.
Method Characteristic Advantage Disadvantage KSVD Dictionary learning-based approach; utilizes sparse representations Effective for various types of noise; adaptable to different image structures Requires training data; sensitive to parameter settings PCA Reduces dimensionality; identifies and retains major patterns Good for Gaussian noise; simple implementation Limited performance with complex noise; loss of fine details BM3D Groups similar image blocks; applies 3D transformation and filtering Excellent at preserving details and textures; effective for various noise types High computational complexity; requires tuning of multiple parameters
Table 3. PSNR Comparisons of Different Polarization Information.
View tableView in ArticleTable 3. PSNR Comparisons of Different Polarization Information.
Method DoLP AoP Noisy input (dB) 18.61 18.58 18.58 4.71 3.84 K-SVD-based (dB) 28.97 30.99 33.00 22.58 4.24
Table 4. Average PSNRs (dB)/SSIMs of Different Methods.
View tableView in ArticleTable 4. Average PSNRs (dB)/SSIMs of Different Methods.
Method DoLP AoP Input 22.24/0.602 10.70/0.123 9.64/0.054 BM3D 25.93/0.890 17.52/0.766 13.27/0.159 PDRDN 32.53/0.967 23.89/0.808 16.34/0.224 CARDN 31.65/0.923 27.04/0.843 20.27/0.249
Table 5. Average PSNRs (dB)/SSIMs of Pol2Pol and Pol2GT.
View tableView in ArticleTable 5. Average PSNRs (dB)/SSIMs of Pol2Pol and Pol2GT.
Method DoLP AoP Input 23.50/0.707 15.54/0.122 12.51/0.112 Pol2Pol 27.72/0.874 23.34/0.720 16.00/0.325 Pol2GT 27.95/0.880 23.61/0.723 16.11/0.328
Table 6. Average PSNRs (dB) and SSIMs of Different Color Polarimetric Image Denoising Methods.
View tableView in ArticleTable 6. Average PSNRs (dB) and SSIMs of Different Color Polarimetric Image Denoising Methods.
Method DoLP AoP Input 20.24/0.605 14.37/0.180 8.42/0.096 CBM3D 25.45/0.805 17.87/0.419 10.33/0.157 IPLNet 27.54/0.841 24.33/0.558 14.57/0.277 ColorPolarNet 33.24/0.937 24.12/0.513 14.94/0.286 3DCNN-based 33.44/0.948 25.94/0.706 17.18/0.310
Table 7. Summary of Learning-Based Denoising Methods for Typical Polarimetric Images.
View tableView in ArticleTable 7. Summary of Learning-Based Denoising Methods for Typical Polarimetric Images.
Method Category Application Highlight PDRDN[ 39 ]Supervised Grayscale Modified RDN for polarimetric image denoising CARDN[ 43 ]Supervised Grayscale Channel attention and adaptive polarization loss TL[ 114 ]Supervised Grayscale Recovering polarization information with a small-scale polarimetric dataset Pol2Pol[ 116 ]Self-supervised Grayscale Conducing paired images by the Stokes relationship IPLNet[ 38 ]Supervised Color Decomposing color polarization images into 12 channels by three sub-networks ColorPolarNet[ 42 ]Supervised Color Decomposing color polarization images into intensity network and polarization network with the Stokes vector 3DCNN[ 44 ]Supervised Color Processing high dimensions of color polarization by 3D convolution ViT-based[ 46 ]Supervised Color Exploiting polarization correlation by the transformer block
Table 8. Quantitative Analysis Results of the DnCNN-Based Method in Different Noise Levels.
View tableView in ArticleTable 8. Quantitative Analysis Results of the DnCNN-Based Method in Different Noise Levels.
Noise Level (Photons/pixel) Noisy polarimetric images Recovered polarimetric images using DnCNN 2D DoLP image (SNR) (dB) 3D DoLP image (SNR) (dB) 2D DoLP image (SNR) (dB) 3D DoLP image (SNR) (dB) 0–4 0.06 0.23 0.51 2.79 4–8 0.11 1.24 0.92 6.74 8–12 0.70 3.32 3.80 10.01 12–16 2.23 6.13 11.70 21.63 16–20 2.91 7.69 15.78 23.23
Table 9. Quantitative Denoised Results of MDU-Net.
View tableView in ArticleTable 9. Quantitative Denoised Results of MDU-Net.
Mueller matrix images D/ / Method MRMSE MPSNR (dB) MSSIM RMSE PSNR (dB) SSIM Noisy 6.589 30.07 0.6875 5.616/3.405/5.891 31.81/36.03/29.30 68.42/69.11/84.05 MDU-Net 2.654 38.12 0.8968 2.298/1.217/2.831 39.40/45.15/35.19 90.30/94.26/95.88
Table 10. Summary of Deep Learning Denoising Methods for Extensive Polarimetric Images.
View tableView in ArticleTable 10. Summary of Deep Learning Denoising Methods for Extensive Polarimetric Images.
Method Category Application Highlight PJNDMNet[ 45 ]Supervised LWIR polarization image Generating training data based on the polarization measurement redundancy error DnCNN based[ 40 ]Supervised Polarization 3D integral image Reconstructing 3D DoLP images with the help of DnCNN MDU-Net[ 41 ]Supervised Mueller matrix image Polarimetry basis parameter recovery based on modified U-Net
Table 11. Summary of Some Representative Methods in Polarimetric Image Denoising.
View tableView in ArticleTable 11. Summary of Some Representative Methods in Polarimetric Image Denoising.
Method Noise type Feature Key word PCA-based[ 34 ]AWGN and realistic Noise suppression and DoLP enhancement for polarization images Keeping the most relevant parts of polarimetric images in the PCA domain KSVD-based[ 36 ]AWGN Denoising polarization images Obtaining the optimized representation of polarization images via dictionary learning and sparse coding PBM3D[ 32 ]AWGN and realistic Reducing noise of polarimetric images and enhancing , DoLP, and AoP information Applying BM3D to a chosen polarization space with the optimal denoising transformation matrix BM3D-DoFP[ 35 ]AWGN Noise suppression and DoLP enhancement for polarization images Extension of BM3D to polarimetric images using a superpixel patch Adaptive BM3D[ 37 ]Speckle Suppressing speckle noise of polarization images BM3D with an adaptive thresholding technique based on the smoothness of a polarimetric image patch BM3D-CPFA[ 33 ]AWGN and realistic Color polarization image denoising and color DoLP reconstruction Applying BM3D according to the correlation between different polarization and color channels PDRDN[ 39 ]Realistic Reducing noise of polarimetric images and recovering , DoLP, and AoP information Residual dense network designed for polarimetric image denoising CARDN[ 43 ]Realistic Reducing noise of polarimetric images and recovering , DoLP, and AoP information A flexible model with a channel attention block and adaptive polarization loss IPLNet[ 38 ]Realistic Color polarimetric image denoising and , DoLP, and AoP restoration A denoising network with intensity polarization, two sub-nets, and one-to-three three sub-branches ColorPolarNet[ 42 ]Realistic Color polarimetric image denoising and , DoLP, and AoP restoration A two-step parallel multitask CNN with a loss function based on Stokes parameters 3DCNN-based[ 44 ]Realistic Denoising color polarimetric images and restoring color , DoLP, and AoP information An entirely 3D denoising network with 3D convolution and color polarization loss ViT-based[ 46 ]Realistic Color polarimetric image denoising and , DoLP, and AoP restoration A vision transformer model with hybrid attention mechanisms and Stokes parameter loss PJDNDMNet[ 45 ]Synthetic and realistic Reducing noise of LWIR polarization images and reconstructing infrared , DoLP, and AoP information A three-stage progressive CNN guided by mixed noise level estimation DnCNN-based[ 40 ]AWGN Suppressing noise of polarimetric 3D integral images and restoring , DoLP, and AoP information A physical model trained DnCNN with passive 3D polarimetric integral imaging in low-light conditions MDU-Net[ 41 ]Realistic Suppressing noise of Mueller matrix images and restoring diattenuation, depolarization, and linear retardance A modified U-Net network incorporating channel attention and Mueller matrix loss Pol2Pol[ 116 ]AWGN and realistic Reducing noise of polarimetric images and recovering , DoLP, and AoP information A self-supervised model trained only with one-shot noisy polarimetric images TL-based[ 114 ]Realistic Reducing noise of polarimetric images and recovering , DoLP, and AoP information Small-scale polarimetric dataset trained network by fine-tuning a pre-trained color image denoising model
Tools
Get Citation
Copy Citation Text
Hedong Liu, Xiaobo Li, Zihan Wang, Yizhao Huang, Jingsheng Zhai, Haofeng Hu, "Review of polarimetric image denoising," Adv. Imaging 1, 022001 (2024)
Paper Information
Category: Review Article
Received: Apr. 15, 2024
Accepted: Sep. 4, 2024
Published Online: Oct. 11, 2024
The Author Email: Haofeng Hu (haofeng_hu@tju.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20