Laser & Optoelectronics Progress, Volume. 61, Issue 16, 1611002(2024)

Advances in Deep Learning for Super-Resolution Microscopy(Invited)

Xinyi Lu1,2, Yu Huang3, Zitong Zhang4, Tianxiao Wu1,2, Hongjun Wu1,2, Yongtao Liu1,2、*, Zhong Fang3、**, Chao Zuo1,2、***, and Qian Chen1,2
Author Affiliations
  • 1Smart Computational Imaging Laboratory, College of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China
  • 2Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China
  • 3School of Mechanical Engineering, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China
  • 4Infection Management Department of Shenzhen Sami Medical Center (Shenzhen Fourth People's Hospital), Shenzhen 518118, Guangdong, China
  • show less
    Figures & Tables(13)
    Framework of deep learning in super-resolution imaging[15-27]
    Schematic diagram of STED[15,16,28]
    Different network framework diagrams and experimental results. (a) MPRNet network architecture[17]; (b) MPRNet reconstruction images of β-tubulin (STAR635P) in U2OS cells[17]; (c) STED-flimGANE network structure diagram[35]; (d) intensity images under extreme conditions of rate depletion[35]; (e) comparison of nuclear hole imaging between low noise SRDAN and other methods[18]; (f) line intensity distribution of core hole images in each algorithm[18]
    Principle of single molecule localization microscopy[19]
    Network and reconstruction results related to improving image reconstruction speed. (a) Deep-STORM reconstruction effect comparison[20]; (b) self-STORM network architecture[42]; (c) self-STORM rebuild effect comparison diagram[42]; (d) DRSN-STORM Network architecture,feature extracting module (FEM), inference module (IM), and reconstruction module (RM)[43]; (e) DRSN-STORM microtubule image reconstruction effect[43]; (f) DECODE reconstruct effect image[46]
    Reduce the network architecture and reconstruction results related to the frame rate of image reconstruction. (a) ANNA-PAM network architecture[47]; (b) ANNA-PAM reconstruction compared with PALM[47]; (c) multi-color single molecule image reconstruction CNN framework[21]; (d) multicolor single molecule network reconstruction effect[21]; (e) DBlink network architecture[49]
    High precision molecular localization related networks and reconstruction results. (a) BGNet network architecture[22]; (b) background estimation of three PSF imaging methods using BGNet[22]; (c) reconstruction results of IEEE ISBI microtubule data set are compared, the top half is binary image, and the bottom half is standardized image[50]; (d) physical simulation feedback flow[51]; (e) tetrapod and learned PSF localization results[51]
    Deep learning extracts additional spectral information from PSF networks and imaging results. (a) smNet network architecture and reconstructed image[52]; (b) color separation and axial positioning architecture and reconstruction effects[23]; (c) image color classification process diagram of optimized phase mask[53]; (d) fluorescent-labeled HeLa cell imaging[53]; (e) color classification image of COS-7 cells[57]
    SIM imaging optical path and schematic diagram. (a) SIM imaging optical path diagram[58]; (b) spectrum extension of SIM[58]; (c) SIM imaging image[58-60]
    Imaging results of deep learning in structured light super-resolution microscopy imaging. (a) U-Net-SIM3 network architecture[24]; (b) reconstruction results of U-Net-SIM5 and scU Net under low light conditions[24]; (c) SIM nanobead imaging using CycleGAN network[63]; (d) caGAN microtubule imaging under low signal-to-noise ratio[64]; (e) schematic diagram of the Fourier attention mechanism principle of DFCAN[65]; (f) DFCAN reconstruction of f-actin cytoskeleton images[65]
    Network architecture and reconstruction result graph for improving the quality of SIM image reconstruction. (a) RED-fairSIM deep learning network architecture diagram[25]; (b) RED-fairSIM imaging results of U2OS osteosarcoma cells[25]; (c) CR-SIM huFIB cell microtubule imaging image[70]; (d) Deep-MSIM microtubule imaging effect[72]
    BioSR data set and PINN network related images. (a) BioSR data set[65]; (b) PINN network architecture[26]; (c) optimization results of nonlinear SIM resolution based on PINN for multiple object types[26]
    Deep learning implements cross modal transformation network structure diagram and result diagram. (a) Deep learning implementation of cross modal transformation network framework for fluorescence microscopy[74]; (b) results of cross modal image conversion from confocal to STED[74]; (c) partial network structure diagram of TCAN network generator[27]; (d) DFCAN mechanism network diagram[27]; (e) TCAN super-resolution imaging results of microtubules[27]
    Tools

    Get Citation

    Copy Citation Text

    Xinyi Lu, Yu Huang, Zitong Zhang, Tianxiao Wu, Hongjun Wu, Yongtao Liu, Zhong Fang, Chao Zuo, Qian Chen. Advances in Deep Learning for Super-Resolution Microscopy(Invited)[J]. Laser & Optoelectronics Progress, 2024, 61(16): 1611002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Imaging Systems

    Received: Jun. 7, 2024

    Accepted: Jul. 2, 2024

    Published Online: Aug. 19, 2024

    The Author Email: Yongtao Liu (Yongtao.Liu@njust.edu.cn), Zhong Fang (fangzhong@njust.edu.cn), Chao Zuo (zuochao@njust.edu.cn)

    DOI:10.3788/LOP241455

    CSTR:32186.14.LOP241455

    Topics