Polarization, a fundamental physical property of light, reveals many intrinsic characteristics of the media light traverses during its propagation. Harnessing polarization thus provides a very valuable additional dimension of information in computational imaging.

Data-driven polarization imaging nowadays stands out as a novel interdisciplinary research area, where the complementary strengths of data-driven approaches and physical models effectively enhance information interpretation and imaging performance. The review details how in the future deeper integration with existing physical models may optimize network training results, improve the interpretability of neural networks and provide a research foundation for developing comprehensive synthetic datasets. Deep learning architectures such as semi-supervised learning, unsupervised learning, transfer learning, multi-task learning and federated learning have also been shown to play a significant role in reducing dependence on datasets, further improving existing imaging outcomes and holding significant implications for expanding into new applications. Finally, the review details how leveraging advanced manufacturing technologies like metasurfaces and meta-lenses as novel optoelectronic devices will allow for precise manipulation of light in specific polarization states. These new capabilities enable optimized acquisition, separation and interpretation of polarized light, potentially enhancing the sensitivity and accuracy of polarization imaging. It is clear that data-driven polarization imaging holds substantial potential for future applications.

Traditional polarization imaging is often limited in accuracy or resolution. Complementing polarization-sensitive acquisition with machine learning schemes provides an interesting solution, leveraging its excellence at non-linear expression and information extraction based on large datasets. This helps bridge the gap between theory and practical implementation. While data-driven based polarimetric imaging develops from the mid-2010s, but in the past years, the combination of deep learning and polarization imaging has become a very active field, and researchers have used deep learning to explore the potential for processing and analyzing polarimetric data in more diverse domains. Today, the synergy between deep learning and polarimetric imaging continues to evolve with advancements in models, algorithms and applications. In recent work published in Opto-Electronic Science, Kui Yang and his colleagues at Xidian University provide a comprehensive overview of the research progress in data-driven polarization imaging, focusing on trends, applications, information utilization, and they also discuss the potential future development¹.

Data-driven polarimetric imaging aims at compensating for the defects of physical model relying on a singular information. It broadens the application fields and a better use of polarization information. The utilization of polarization information has been extended from direct polarization data acquisition to preprocessed polarization features. Furthermore, physical insight is crucial during network training. Incorporating physical models is playing an increasingly integral role in guiding and planning the design and training of neural networks. Based on the polarimetric information fused into the network, other physical properties of light have also been introduced into network training, expanding the application domain from the realm of image processing to more high-level tasks such as classification or segmentation.

Data-driven polarization imaging technologies have gradually found applications in descattering imaging², denoising³, demosaicing⁴, dynamic range enhancement⁵, reflection removal⁶, low-light imaging⁷, 3D reconstruction shape⁸, semantic segmentation⁹, camouflage object detection¹⁰, classification¹¹, pathological diagnosis¹², to cite just a few, as illustrated in Fig. 1. In a nutshell, NNs’ high-order non-linear representation enables higher-dimensional information extraction from complex imaging media and scenes. It significantly improves the interpretation and reconstruction of physical properties, even in challenging low signal-to-noise ratio environments, such as natural settings, scattering media, noise, ambient light interference, low dynamics, and biological tissues. As an additional degree of freedom, polarization expands the application scope of intensity-based deep learning algorithms from target segmentation, authentication, camouflage target identification, medical diagnosis to domains including 3D reconstruction and physical information transformation. It is a powerful tool for improving imaging quality, enhancing target interpretation accuracy, and driving the progress of emerging application areas.