Infrared and Laser Engineering, Volume. 53, Issue 3, 20240057(2024)
Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)
Figures & Tables(14)
Fig. 1. General polarization imaging system (Blue optical components represent polarizers, and gray optical components represent waveplates)
Fig. 2. Degradation mechanism of polarization images in complex environments
Fig. 3. The general workflow of polarization imaging technology in complex environments based on deep learning
Fig. 4. Polarization imaging process in the water scattering environment[19]
Fig. 5. (a) PDN network structure; (b) PDN network restoration effect[53]
Fig. 6. (a1)-(b1) Original image captured directly in the turbid underwater scene; (a2)-(b2) 3D reconstruction result of the restoration image[56]
Fig. 7. (a) The structure of the U2R-pGAN; (b) Restoration results (The first row shows the light intensity images and the second row displays the restoration images)[60]
Fig. 8. (a) The proposed network structure; (b) Foggy image synthesis process; (c) Distant scene restoration effect[49]
Fig. 9. Noise transfer when calculating polarization parameters (The red rectangle indicates the enlarged area)
Fig. 10. (a) The structure of PDRDN; (b) The structure of residual dense block; (c) Comparison between polarization parameter restoration effect and ground truth; (d) Restoration effects of different materials[71]
Fig. 11. Model test performance under optically captured polarization images in the low light environment (8 photons/pixel)[72]
Fig. 12. (a) The flowchart of Pol2Pol method; (b) The workflow of polarization generator; (c) Comparation on restoration effects of different materials[79]
Fig. 13. Restoration effects of different materials based on channel attention mechanism method[80]
Table 1. Summary of representative work on deep learning polarization imaging in the complex environments
View tableView in ArticleTable 1. Summary of representative work on deep learning polarization imaging in the complex environments
Task Reference Training method Characteristics of representative work Descattering [ 53 ]Supervised/data-driven Proposing for the first time using residual dense network to achieve underwater polarization images descattering [ 55 ]Supervised/data-driven Descattering with high turbidity water [ 56 ]Supervised/data-driven Achieving high-precision three-dimensional imaging of targets in the turbid water [ 57 ]Supervised/data-driven Using Monte Carlo simulation algorithm to simulate the polarization information degradation process [ 60 ]Unsupervised/data-driven Training does not require paired datasets [ 62 ]Self-supervised/data-driven Using physical degradation model to generate pseudo-label data to drive network training [ 63 ]Self-supervised/data-driven Using the Stokes-based descattering model to replace the network backpropagation process [ 49 ]Supervised/physical model embedded Embedding physical degradation model in the network and the key parameters of the model are fitted [ 66 ]Supervised/physical model embedded Integrating physical model parameters to reduce the degree of freedom of network fitting [ 67 ]Supervised/prior knowledge guidance Using the spectral information of the light intensity image to input the gating network to assign sub-network weights Denoising [ 71 ]Supervised/data-driven Proposing for the first time low-light environment polarization images denoising [ 72 ]Supervised/data-driven Proposing three-dimensional polarization images denoising with low illumination and partial occlusion [ 74 ]Supervised/data-driven Denoising with color polarization images [ 76 ]Supervised/data-driven Using transfer learning method to achieve small sample data polarization images denoising [ 77 ]Unsupervised/data-driven Training does not require paired datasets [ 79 ]Self-supervised/data-driven Using the polarization generator to implement self-supervised training to achieve the polarization images denoising effect of supervised training [ 80 ]Supervised/data-driven Exploring the underlying principles of polarization images denoising from the perspective of feature map screening by attention mechanism [ 81 ]Supervised/data-driven Analyzing the reasons for maintaining the relationship between polarization information during network training from the perspective of feature maps
Tools
Get Citation
Copy Citation Text
Haofeng Hu, Yizhao Huang, Zhen Zhu, Qianwen Ma, Jingsheng Zhai, Xiaobo Li. Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)[J]. Infrared and Laser Engineering, 2024, 53(3): 20240057
Paper Information
Category:
Received: Jan. 31, 2024
Accepted: --
Published Online: Jun. 21, 2024
The Author Email: Li Xiaobo (lixiaobo@tju.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20