Laser & Optoelectronics Progress, Volume. 58, Issue 16, 1600001(2021)

Research Progress in the Applications of Convolutional Neural Networks in Optical Information Processing

Jianglei Di1,2、*, Ju Tang1,2, Ji Wu1,2, Kaiqiang Wang1,2, Zhenbo Ren1,2, Mengmeng Zhang1,2, and Jianlin Zhao1,2、**
Author Affiliations
  • 1Key Laboratory of Light Field Manipulation and Information Acquisition, Ministry of Industry and Information Technology, Northwestern Polytechnical University, Xi'an, Shaanxi 710129, China
  • 2Shaanxi Key Laboratory of Optical Information Technology, School of Physical Science and Technology, Northwestern Polytechnical University, Xi'an, Shaanxi 710129, China
  • show less
    Figures & Tables(21)
    Simulation process. (a) Physical model; (b) forward problems fitting by neural network; (c) inverse problems fitting by neural network
    Simulation process. (a) Fully connected structure; (b) convolution operation
    Curve of activation function
    Main structure of network. (a) Backbone; (b) FCN; (c) U-net; (d) GAN
    Detail structure of network. (a) Residual block; (b) multi-scale block; (c) attention block, in which (c1) is channel attention and (c2) is spatial attention; (d) dense connected block
    Flow chart of network training and testing. (a) Training process; (b) testing process
    In-line holographic numerical reconstruction with CNN. (a) CNN is used to suppress “twin image” and autocorrelation artifacts[32]; (b) end-to-end phase reconstruction using eHoloNet [33]
    Off-axis holographic numerical reconstruction with CNN. (a) U-net [34]; (b) Y-Net [36]
    Applications of CNN in holographic reconstruction distance. (a) Regression model[42-44]; (b) classification model[47]
    Fringe patterns analysis with CNN. (a) Method in Ref. [58]; (b) method in Ref. [59]; (c) method in Ref. [60]
    Phase unwrapping with CNN. (a) Method in Ref. [79]; (b) method in Ref. [80]; (c) method in Ref. [84]
    Ghost imaging technology. (a) Computational ghost imaging process; (b) ghost imaging reconstruction using neural network
    Computational ghost imaging with CNN. (a) DRU-Net [99]; (b) DeepGhost [101]; (c) DAttNet [102]
    Fourier ptychographic microscopy system[116]
    Applications of CNN in Fourier ptychographic microscopy. (a) Super-resolution reconstruction of complex amplitude lightfield[115]; (b) aberration-free high resolution image reconstruction with pupil function estimation[116]; (c) LED array position deviation correction to optimize reconstruction quality[117]
    Applications of CNN in super-resolution imaging. (a)Cross modal and super-resolution imaging with GAN[124]; (b) end-to-end lensless microscopic super-resolution imaging [127]; (c) hologram super-resolution optimization[128]
    Applications of CNN in scattering medium classification. (a) Network trained by synthetic data achieves classification in experimental application[143]; (b) speckle pattern classification of face and non-face[144]
    Applications of CNN in scattering medium reconstruction. (a) Image reconstruction of speckle field behind optical fiber with CNN[147]; (b) CNN is used to pre-reconstruct the phase[148]; (c) CNN for image reconstruction in strong scattering media[150]
    Accurate choroidal segmentation using CNN[157]
    Optical fiber reconstruction in optical diffraction tomography using CNN. (a) Internal structure reconstruction of optical fiber with limited angle[166]; (b) internal structure reconstruction of photonic crystal fiber with sparse angle[167]
    • Table 1. Applications of CNN in optical information processing

      View table

      Table 1. Applications of CNN in optical information processing

      Application fieldNetwork structureLoss functionApplication problems
      Digital holographyBackbone, U-net, GANMSE, MAE, cross entropyHolographic reconstruction[30-39]: “twin-image” problem, “end to end” phase recovery, reconstruction of complex amplitude light fieldAuto focusing[42-49]: prediction of holographic reconstruction distanceOthers[50-54]: holographic image denoising, multi wavelength hologram fusion and reconstruction, reconstructed image enhancement
      Fringe analysisBackbone, FCN, U-netMSE, MAE, regularizationPhase demodulation and 3D reconstruction[58-62,67]Fast recognition of fringes[63]Fringe image denoising[64-66]
      Phase unwrappingFCN, U-net, GANMSE, MAE, cross entropy, regularizationPhase unwrapping[79-86]
      Application fieldNetwork structureLoss functionApplication problems
      Ghost imagingBackbone, FCN, U-net, GANMSE, regularizationNoise suppression[94]Blind image reconstruction[95,97]Low sampling imaging[96,101-103,105]Lighting mode optimization[98,102-104]
      Fourier ptychographic microscopyU-net, GANMSE, MAE, regularizationSuper resolution image reconstruction [112-113,115-116]Speed up reconstruction [113,115]Position deviation correction [117]Noise suppression[113-115]
      Super resolution imagingFCN, U-net, GANMSE, MAE, regularizationSuper resolution imaging [120-131]
      Scattering medium imagingBackbone, U-netMSE, MAE, cross entropyTarget classification [142-145]Image reconstruction [146-153]Modal decomposition of multimode fiber [154]
      Optical tomographyBackbone, U-net, GANMSE, cross entropy, regularizationCoherence tomography [156-160]: high precision and fast image segmentation, image enhancementDiffraction tomography[164-167]: noise suppression, Inversion reconstruction
    Tools

    Get Citation

    Copy Citation Text

    Jianglei Di, Ju Tang, Ji Wu, Kaiqiang Wang, Zhenbo Ren, Mengmeng Zhang, Jianlin Zhao. Research Progress in the Applications of Convolutional Neural Networks in Optical Information Processing[J]. Laser & Optoelectronics Progress, 2021, 58(16): 1600001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Reviews

    Received: Apr. 30, 2021

    Accepted: Jun. 10, 2021

    Published Online: Aug. 12, 2021

    The Author Email: Jianglei Di (jiangleidi@nwpu.edu.cn), Jianlin Zhao (jlzhao@nwpu.edu.cn)

    DOI:10.3788/LOP202158.1600001

    Topics