Ultra-wide FOV meta-camera with transformer-neural-network color imaging methodology

Yan Liu; Wen-Dong Li; Kun-Yuan Xin; Ze-Ming Chen; Zun-Yi Chen; Rui Chen; Xiao-Dong Chen; Fu-Li Zhao; Wei-Shi Zheng; Jian-Wen Dong

doi:10.1117/1.AP.6.5.056001

Advanced Photonics, Volume. 6, Issue 5, 056001(2024)

Ultra-wide FOV meta-camera with transformer-neural-network color imaging methodology

Yan Liu^1、†, Wen-Dong Li², Kun-Yuan Xin¹, Ze-Ming Chen¹, Zun-Yi Chen¹, Rui Chen¹, Xiao-Dong Chen¹, Fu-Li Zhao¹, Wei-Shi Zheng^2、*, and Jian-Wen Dong^1、*

Author Affiliations

¹Sun Yat-sen University, School of Physics, State Key Laboratory of Optoelectronic Materials and Technologies, Guangzhou, China

²Sun Yat-sen University, School of Computer Science and Engineering, Guangzhou, China

show less

References(54)

[1] J. Wu et al. An integrated imaging sensor for aberration-corrected 3D photography. Nature, 612, 62-71(2022).

[2] K. Kim et al. Biologically inspired ultrathin arrayed camera for high-contrast and high-resolution imaging. Light Sci. Appl., 9, 28(2020).

[3] Z.-Y. Hu et al. Miniature optoelectronic compound eye camera. Nat. Commun., 13, 5634(2022).

[4] Y. Zhou et al. Flat optics for image differentiation. Nat. Photonics, 14, 316-323(2020).

[5] M. K. Chen et al. Principles, functions, and applications of optical meta-lens. Adv. Opt. Mater., 9, 2001414(2021).

[6] A. Arbabi, A. Faraon. Advances in optical metalenses. Nat. Photonics, 17, 16-25(2023).

[7] M. Pan et al. Dielectric metalens for miniaturized imaging systems: progress and challenges. Light Sci. Appl., 11, 195-226(2022).

[8] B. B. Xu et al. Metalens-integrated compact imaging devices for wide-field microscopy. Adv. Photonics, 2, 066004(2020).

[9] X. Luo et al. Recent advances of wide-angle metalenses: principle, design, and applications. Nanophotonics, 11, 1-20(2022).

[10] F. Yang et al. Wide field-of-view metalens: a tutorial. Adv. Photonics, 5, 033001(2023).

[11] A. Arbabi et al. Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission. Nat. Nanotechnol., 10, 937-943(2015).

[12] A. Arbabi et al. Subwavelength-thick lenses with high numerical apertures and large efficiency based on high-contrast transmit arrays. Nat. Commun., 6, 7069(2015).

[13] M. Khorasaninejad et al. Metalenses at visible wavelengths: diffraction-limited focusing and subwavelength resolution imaging. Science, 352, 1190(2016).

[14] Z.-B. Fan et al. Silicon nitride metalenses for close-to-one numerical aperture and wide-angle visible imaging. Phys. Rev. Appl., 10, 014005(2018).

[15] H. Liang et al. Ultrahigh numerical aperture metalens at visible wavelengths. Nano Lett., 18, 4460-4466(2018).

[16] S. Shrestha et al. Broadband achromatic dielectric metalenses. Light Sci. Appl., 7, 85(2018).

[17] S. Wang et al. A broadband achromatic metalens in the visible. Nat. Nanotechnol., 13, 227-232(2018).

[18] Z.-B. Fan et al. A broadband achromatic metalens array for integral imaging in the visible. Light Sci. Appl., 8, 67(2019).

[19] R. J. Lin et al. Achromatic metalens array for full-colour light-field imaging. Nat. Nanotechnol., 14, 227-231(2019).

[20] H. Li et al. Bandpass-filter-integrated multiwavelength achromatic metalens. Photonics Res., 9, 1384-1390(2021).

[21] Z. Li et al. Meta-optics achieves RGB-achromatic focusing for virtual reality. Sci. Adv., 7, eabe4458(2021).

[22] W. Feng et al. RGB achromatic metalens doublet for digital imaging. Nano Lett., 22, 3969-3975(2022).

[23] A. Arbabi et al. Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations. Nat. Commun., 7, 13682(2016).

[24] B. Groever, W. T. Chen, F. Capasso. Meta-lens doublet in the visible region. Nano Lett., 17, 4902-4907(2017).

[25] Y. Liu et al. Meta-objective with sub-micrometer resolution for microendoscopes. Photonics Res., 9, 106-115(2021).

[26] M. Y. Shalaginov et al. Single-element diffraction-limited fisheye metalens. Nano Lett., 20, 7429-7437(2020).

[27] F. Zhang et al. Extreme-angle silicon infrared optics enabled by streamlined surfaces. Adv. Mater., 33, 2008157(2021).

[28] A. Martins et al. On metalenses with arbitrarily wide field of view. ACS Photonics, 7, 2073-2079(2020).

[29] J. Chen et al. Planar wide-angle-imaging camera enabled by metalens array. Optica, 9, 431-437(2022).

[30] S. Colburn, A. Zhan, A. Majumdar. Metasurface optics for full-color computational imaging. Sci. Adv., 4, eaar2114(2018).

[31] E. E. Fenimore. Coded aperture imaging: the modulation transfer function for uniformly redundant arrays. Appl. Opt., 19, 2465-2471(1980).

[32] S. R. Gottesman, E. E. Fenimore. New family of binary arrays for coded aperture imaging. Appl. Opt., 28, 4344-4352(1989).

[33] E. Tseng et al. Neural nano-optics for high-quality thin lens imaging. Nat. Commun., 12, 6493(2021).

[34] Q. Fan et al. Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field. Nat. Commun., 13, 2130(2022).

[35] Y. Peng et al. The diffractive achromat full spectrum computational imaging with diffractive optics. ACM Trans. Graphics, 35, 31(2016).

[36] V. Sitzmann et al. End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graphics, 37, 114(2018).

[37] Y. Peng et al. Learned large field-of-view imaging with thin-plate optics. ACM Trans. Graphics, 38, 219(2019).

[38] L. Yann, B. Yoshua, H. Geoffrey. Deep learning. Nature, 521, 436-444(2015).

[39] J. Jumper et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583-589(2021).

[40] T. Brown et al. Language models are few-shot learners, 1877-1901(2020).

[41] S. Pinilla et al. Miniature color camera via flat hybrid meta-optics. Sci. Adv., 9, eadg7297(2023).

[42] S. Molesky et al. Inverse design in nanophotonics. Nat. Photonics, 12, 659-670(2018).

[43] W. Ma et al. Deep learning for the design of photonic structures. Nat. Photonics, 15, 77-90(2021).

[44] F. Wang et al. Phase imaging with an untrained neural network. Light Sci. Appl., 9, 77(2020).

[45] V. Liu, S. Fan. S⁴: a free electromagnetic solver for layered periodic structures. Comput. Phys. Commun., 183, 2233-2244(2012).

[46] A. Vaswani et al. Attention is all you need, 6000-6010(2017).

[47] O. Ronneberger, P. Fischer, T. Brox. U-Net: convolutional networks for biomedical image segmentation, 234-241(2015).

[48] C. Tan et al. A survey on deep transfer learning, 270-279(2018).

[49] Z. Wang et al. Uformer: a general u-shaped transformer for image restoration, 17683-17693(2022).

[50] D. Chen, J. A. Tachella, M. E. Davies. Robust equivariant imaging: a fully unsupervised framework for learning to image from noisy and partial measurements, 5647-5656(2022).

[51] J. Xiong et al. Dynamic brain spectrum acquired by a real-time ultraspectral imaging chip with reconfigurable metasurfaces. Optica, 9, 461-468(2022).

[52] L. Zhu et al. Large field-of-view non-invasive imaging through scattering layers using fluctuating random illumination. Nat. Commun., 13, 1447(2022).

[53] Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell., 22, 1330-1334(2000).

[54] D. J. Jobson, Z. U. Rahman, G. A. Woodell. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process., 6, 965-976(1997).

Tools

Get Citation

Copy Citation Text

Yan Liu, Wen-Dong Li, Kun-Yuan Xin, Ze-Ming Chen, Zun-Yi Chen, Rui Chen, Xiao-Dong Chen, Fu-Li Zhao, Wei-Shi Zheng, Jian-Wen Dong, "Ultra-wide FOV meta-camera with transformer-neural-network color imaging methodology," Adv. Photon. 6, 056001 (2024)

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites