Optics and Precision Engineering, Volume. 30, Issue 21, 2827(2022)

End-to-end co-design of optics and image processing and its applications

Xiong DUN, Jian ZHANG, Shiqi FENG, Hong LUO, Zhanshan WANG, and Xinbin CHENG*
Author Affiliations
  • Institute of Precision Optical Engineering, MOE Key Laboratory of Advanced Micro-Structured Materials, Shanghai Frontiers Science Center of Digital Optics, Shanghai Professional Technical Service Platform for Full-Spectrum and High-Performance Optical Thin Film Devices and Applications, School of Physics Science and Engineering, Tongji University, Shanghai200092, China
  • show less
    References(34)

    [1] [1] 1左超, 陈钱. 计算光学成像:何来,何处,何去,何从?[J]. 红外与激光工程, 2022, 51(2): 158-338. doi: 10.3788/IRLA20220110ZUOCH, CHENQ. Computational optical imaging: an overview[J]. Infrared and Laser Engineering, 2022, 51(2): 158-338. (in Chinese). doi: 10.3788/IRLA20220110

    [2] LEVOY M. Light fields and computational imaging[J]. Computer, 39, 46-55(2006).

    [3] SUNG H Y, CHANG C W. Method for designing computational optical imaging system[P].

    [4] MAIT J N, EULISS G W, ATHALE R A. Computational imaging[J]. Advances in Optics and Photonics, 10, 409-483(2018).

    [5] SITZMANN V, DIAMOND S, PENG Y F et al. End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging[J]. ACM Transactions on Graphics, 37, 114(2018).

    [6] IKOMA H, WETZSTEIN G et al. Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging[J]. Optica, 7, 913-922(2020).

    [7] PENG Y F, SUN Q L et al. Learned large field-of-view imaging with thin-plate optics[J]. ACM Transactions on Graphics, 38, 219(2019).

    [8] JEON D S, BAEK S H, YI S et al. Compact snapshot hyperspectral imaging with diffracted rotation[J]. ACM Transactions on Graphics, 38, 117(2019).

    [9] SUN Q L, ZHANG J et al. End-to-end learned, optically coded super-resolution SPAD camera[J]. ACM Transactions on Graphics, 39, 9(2020).

    [10] TSENG E, COLBURN S, WHITEHEAD J et al. Neural nano-optics for high-quality thin lens imaging[J]. Nature Communications, 12, 6493(2021).

    [11] LÉVÊQUE O, KULCSÁR C, LEE A et al. Co-designed annular binary phase masks for depth-of-field extension in single-molecule localization microscopy[J]. Optics Express, 28, 32426-32446(2020).

    [12] JIN L B, TANG Y B, WU Y C et al. Deep learning extended depth-of-field microscope for fast and slide-free histology[J]. Proceedings of the National Academy of Sciences of the United States of America, 117, 33051-33060(2020).

    [13] LIU Y K, ZHANG C Y, KOU T D et al. End-to-end computational optics with a singlet lens for large depth-of-field imaging[J]. Optics Express, 29, 28530-28548(2021).

    [14] BANERJI S, MEEM M, MAJUMDER A et al. Extreme-depth-of-focus imaging with a flat lens[J]. Optica, 7, 214-217(2020).

    [15] LUO Y, MENGU D, YARDIMCI N T et al. Design of task-specific optical systems using broadband diffractive neural networks[J]. Light: Science & Applications, 8, 112(2019).

    [16] SONG H Y, MA Y G, HAN Y B et al. Deep-learned broadband encoding stochastic filters for computational spectroscopic instruments[J]. Advanced Theory and Simulations, 4, 2000299(2021).

    [17] CHANG J L, WETZSTEIN G. Deep optics for monocular depth estimation and 3D object detection[C], 10192-10201(2019).

    [18] NEHME E, FREEDMAN D, GORDON R et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning[J]. Nature Methods, 17, 734-740(2020).

    [20] SUN Q L, TSENG E, FU Q et al. Learning rank-1 diffractive optics for single-shot high dynamic range imaging[C], 1383-1393(2020).

    [21] MARTEL J N P, MÜLLER L K, CAREY S J et al. Neural sensors: learning pixel exposures for HDR imaging and video compressive sensing with programmable sensors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 1642-1653(2020).

    [22] SUN Q L, WANG C L, FU Q et al. End-to-end complex lens design with differentiate ray tracing[J]. ACM Transactions on Graphics, 40, 71(2021).

    [23] VOLLMER M, MÖLLMANN K P[M]. Infrared Thermal Imaging: Fundamentals, Research and Applications(2017).

    [24] YANNY K, MONAKHOVA K, SHUAI R W et al. Deep learning for fast spatially varying deconvolution[J]. Optica, 9, 96-99(2022).

    [25] MONAKHOVA K, YURTSEVER J et al. Learned reconstructions for practical mask-based lensless imaging[J]. Optics Express, 27, 28075-28090(2019).

    [26] BANERJI S, MEEM M, MAJUMDER A et al. Imaging with flat optics: metalenses or diffractive lenses?[J]. Optica, 6, 805-810(2019).

    [27] MEEM M, BANERJI S, PIES C et al. Large-area, high-numerical-aperture multi-level diffractive lens via inverse design: errata[J]. Optica, 7, 252-253(2020).

    [28] WANG P, MOHAMMAD N, MENON R. Chromatic-aberration-corrected diffractive lenses for ultra-broadband focusing[J]. Scientific Reports, 6, 21545(2016).

    [29] TSENG M L, HSIAO H, CHU C H et al. Metalenses: advances and applications[J]. Advanced Optical Materials, 6, 1800554(2018).

    [30] AIETA F, KATS M A, GENEVET P et al. Applied optics. Multiwavelength achromatic metasurfaces by dispersive phase compensation[J]. Science, 347, 1342-1345(2015).

    [31] KHORASANINEJAD M, CHEN W T, DEVLIN R C et al. Metalenses at visible wavelengths: Diffraction-limited focusing and subwavelength resolution imaging[J]. Science, 352, 1190-1194(2016).

    [32] KHORASANINEJAD M, ZHU A Y, ROQUES-CARMES C et al. Polarization-insensitive metalenses at visible wavelengths[J]. Nano Letters, 16, 7229-7234(2016).

    [33] PHAN T, SELL D, WANG E W et al. High-efficiency, large-area, topology-optimized metasurfaces[J]. Light: Science & Applications, 8, 48(2019).

    [34] QI B Y, CHEN W et al. All-day thin-lens computational imaging with scene-specific learning recovery[J]. Applied Optics, 61, 1097-1105(2022).

    Tools

    Get Citation

    Copy Citation Text

    Xiong DUN, Jian ZHANG, Shiqi FENG, Hong LUO, Zhanshan WANG, Xinbin CHENG. End-to-end co-design of optics and image processing and its applications[J]. Optics and Precision Engineering, 2022, 30(21): 2827

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Received: Jul. 16, 2022

    Accepted: --

    Published Online: Nov. 28, 2022

    The Author Email: Xinbin CHENG (chengxb@tongji.edu.cn)

    DOI:10.37188/OPE.20223021.2827

    Topics