Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Chinese Journal of Lasers, Volume. 51, Issue 21, 2107103(2024)

Advancement in Structured Illumination Microscopy Based on Deep Learning

Xinran Li1, Jiajie Chen1、*, Meiting Wang2, Xiaomin Zheng1, Peng Du1, Yili Zhong1, Xiaoqi Dai1, Junle Qu1, and Yonghong Shao1、**
Author Affiliations
  • 1College of Physics and Optoelectronic Engineering, Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, Shenzhen University, Shenzhen 518060, Guangdong , China
  • 2College of Biomedical Engineering, Department of Medicine, State-Local Joint Engineering Laboratory of Key Technology of Medical Ultrasound, Key Laboratory of Biomedical Information Detection and Ultrasound Imaging and Guangdong Province, Shenzhen University, Shenzhen 518060, Guangdong , China
    • show less
    AbstractGet PDF(in Chinese)
    Figures&Tables (13)
    Equations (0)
    References (82)
    Tools
    Paper Information
    Topics
    References(82)

    [1] Abbe E. Beiträge zur theorie des mikroskops und der mikroskopischen wahrnehmung[J]. Archiv Für Mikroskopische Anatomie, 9, 413-468(1873).

    [2] Wildanger D, Bückers J, Westphal V et al. A STED microscope aligned by design[J]. Optics Express, 17, 16100-16110(2009).

    [3] Hess S T, Girirajan T P K, Mason M D. Ultra-high resolution imaging by fluorescence photoactivation localization microscopy[J]. Biophysical Journal, 91, 4258-4272(2006).

    [4] Rust M J, Bates M, Zhuang X W. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 3, 793-796(2006).

    [5] Gustafsson M G L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy[J]. Journal of Microscopy, 198, 82-87(2000).

    [6] Lin D Y, Qu J L. Recent progress on super-resolution imaging and correlative super-resolution microscopy[J]. Acta Physica Sinica, 66, 148703(2017).

    [7] Gustafsson M G, Agard D A, Sedat J W. Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination[J]. Proceedings of SPIE, 3919, 141-150(2000).

    [8] Gustafsson M G L, Shao L, Carlton P M et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination[J]. Biophysical Journal, 94, 4957-4970(2008).

    [9] Li Z W, Zhang Q R, Chou S W et al. Fast widefield imaging of neuronal structure and function with optical sectioning in vivo[J]. Science Advances, 6, eaaz3870(2020).

    [10] Wang Z J, Zhao T Y, Hao H W et al. High-speed image reconstruction for optically sectioned, super-resolution structured illumination microscopy[J]. Advanced Photonics, 4, 026003(2022).

    [11] Labouesse S, Allain M, Idier J et al. Fluorescence blind structured illumination microscopy: a new reconstruction strategy[C], 3166-3170(2016).

    [12] Kner P, Chhun B B, Griffis E R et al. Super-resolution video microscopy of live cells by structured illumination[J]. Nature Methods, 6, 339-342(2009).

    [13] York A G, Parekh S H, Dalle Nogare D et al. Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy[J]. Nature Methods, 9, 749-754(2012).

    [14] Wen G, Li S M, Wang L B et al. High-fidelity structured illumination microscopy by point-spread-function engineering[J]. Light: Science & Applications, 10, 70(2021).

    [15] Dan D, Lei M, Yao B L et al. DMD-based LED-illumination super-resolution and optical sectioning microscopy[J]. Scientific Reports, 3, 1116(2013).

    [16] Li M Q, Li Y N, Liu W H et al. Structured illumination microscopy using digital micro-mirror device and coherent light source[J]. Applied Physics Letters, 116, 233702(2020).

    [17] Zhao T Y, Hao H W, Wang Z J et al. Multi-color structured illumination microscopy for live cell imaging based on the enhanced image recombination transform algorithm[J]. Biomedical Optics Express, 12, 3474-3484(2021).

    [18] Huang X S, Fan J C, Li L J et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[J]. Nature Biotechnology, 36, 451-459(2018).

    [19] Zhao W S, Zhao S Q, Li L J et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy[J]. Nature Biotechnology, 40, 606-617(2022).

    [20] Wang M T, Wang L, Zheng X M et al. Nonlinear scanning structured illumination microscopy based on nonsinusoidal modulation[J]. Journal of Innovative Optical Health Sciences, 14, 2142002(2021).

    [21] Nehme E, Weiss L E, Michaeli T et al. Deep-STORM: super-resolution single-molecule microscopy by deep learning[J]. Optica, 5, 458-464(2018).

    [22] Ouyang W, Aristov A, Lelek M et al. Deep learning massively accelerates super-resolution localization microscopy[J]. Nature Biotechnology, 36, 460-468(2018).

    [23] Volkmann H. Ernst Abbe and his work[J]. Applied Optics, 5, 1720-1731(1966).

    [24] Wu Y C, Faster Shroff H., sharper, deeper. structured illumination microscopy for biological imaging[J]. Nature Methods, 15, 1011-1019(2018).

    [25] Lu J, Min W, Conchello J A et al. Super-resolution laser scanning microscopy through spatiotemporal modulation[J]. Nano Letters, 9, 3883-3889(2009).

    [26] Zheng X M, Zhou J, Wang L et al. Current challenges and solutions of super-resolution structured illumination microscopy[J]. APL Photonics, 6, 020901(2021).

    [27] Shao L, Kner P, Rego E H et al. Super-resolution 3D microscopy of live whole cells using structured illumination[J]. Nature Methods, 8, 1044-1046(2011).

    [28] Planchon T A, Gao L, Milkie D E et al. Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination[J]. Nature Methods, 8, 417-423(2011).

    [29] Chen B C, Legant W R, Wang K et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution[J]. Science, 346, 1257998(2014).

    [30] Gustafsson M G L. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 102, 13081-13086(2005).

    [31] Hirvonen L, Mandula O, Wicker K et al. Structured illumination microscopy using photoswitchable fluorescent proteins[J]. Proceedings of SPIE, 6861, 68610L(2008).

    [32] Rego E H, Shao L, Macklin J J et al. Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 109, E135-E143(2012).

    [33] Chen T A, Chen L C, Li H et al. Structured illumination super-resolution microscopy technology: review and prospect[J]. Chinese Optics, 11, 307-328(2018).

    [34] Ströhl F, Kaminski C F. Frontiers in structured illumination microscopy[J]. Optica, 3, 667(2016).

    [35] Wang M T, Wu W S, Wang L et al. SHG super-resolution microscopy based on point-scanning structured light[J]. Acta Optica Sinica, 42, 1018001(2022).

    [36] Ingaramo M, York A G, Wawrzusin P et al. Two-photon excitation improves multifocal structured illumination microscopy in thick scattering tissue[J]. Proceedings of the National Academy of Sciences of the United States of America, 111, 5254-5259(2014).

    [37] Urban B E, Yi J, Chen S Y et al. Super-resolution two-photon microscopy via scanning patterned illumination[J]. Physical Review E, 91, 042703(2015).

    [38] Wang L, Zheng X M, Zhou J et al. Improvement in resolution of multiphoton scanning structured illumination microscopy via harmonics[J]. Engineering, 16, 65-72(2022).

    [39] Wang M T, Chen J J, Wu W S et al. Multi-color two-photon scanning structured illumination microscopy imaging of live cells[J]. Journal of Biophotonics, 16, e202300077(2023).

    [40] Zheng X M, Wang L, Wang M T et al. Resolution improvement of structured illumination microscopy using non-diffraction-limited point-scanned fluorescence fringe[J]. Laser & Photonics Reviews, 17, 2200796(2023).

    [41] Tang Y J, Wang L B, Wen G et al. Recent advances in structured illumination microscope super-resolution image reconstruction[J]. Laser & Optoelectronics Progress, 59, 0617009(2022).

    [42] Seo H, Badiei Khuzani M, Vasudevan V et al. Machine learning techniques for biomedical image segmentation: an overview of technical aspects and introduction to state-of-art applications[J]. Medical Physics, 47, e148-e167(2020).

    [43] Schölkopf B, Smola A J. Support vector machines[M]. Learning with kernels: support vector machines, regularization, optimization, and beyond, 187-188(2001).

    [44] Ho T K. A data complexity analysis of comparative advantages of decision forest constructors[J]. Pattern Analysis & Applications, 5, 102-112(2002).

    [45] Zeiler M D, Taylor G W, Fergus R. Adaptive deconvolutional networks for mid and high level feature learning[C], 2018-2025(2011).

    [46] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 640-651(2017).

    [47] LeCun Y, Bottou L, Bengio Y et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 86, 2278-2324(1998).

    [48] Li Y, Qi H Z, Dai J F et al. Fully convolutional instance-aware semantic segmentation[C], 4438-4446(2017).

    [49] Cohen A, Rivlin E, Shimshoni I et al. Memory based active contour algorithm using pixel-level classified images for colon crypt segmentation[J]. Computerized Medical Imaging and Graphics, 43, 150-164(2015).

    [50] Beckers J M. Adaptive optics for astronomy: principles, performance, and applications[J]. Annual Review of Astronomy and Astrophysics, 31, 13-62(1993).

    [51] Noll R J. Zernike polynomials and atmospheric turbulence[J]. Journal of the Optical Society of America, 66, 207-211(1976).

    [52] Paine S W, Fienup J R. Machine learning for improved image-based wavefront sensing[J]. Optics Letters, 43, 1235-1238(2018).

    [53] Jin Y C, Zhang Y Y, Hu L J et al. Machine learning guided rapid focusing with sensor-less aberration corrections[J]. Optics Express, 26, 30162-30171(2018).

    [54] Nishizaki Y, Valdivia M, Horisaki R et al. Deep learning wavefront sensing[J]. Optics Express, 27, 240-251(2019).

    [55] Wang Z J, Zhao T Y, Cai Y N et al. Rapid, artifact-reduced, image reconstruction for super-resolution structured illumination microscopy[J]. The Innovation, 4, 100425(2023).

    [56] Zheng Y, Chen J J, Wu C X et al. Adaptive optics for structured illumination microscopy based on deep learning[J]. Cytometry A, 99, 622-631(2021).

    [57] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[M]. Medical image computing and computer-assisted intervention – MICCAI 2015, 9351, 234-241(2015).

    [58] Xu X Y, Cheng J, Thrall M J et al. Multimodal non-linear optical imaging for label-free differentiation of lung cancerous lesions from normal and desmoplastic tissues[J]. Biomedical Optics Express, 4, 2855-2868(2013).

    [59] Feng X, Qing K, Tustison N J et al. Deep convolutional neural network for segmentation of thoracic organs-at-risk using cropped 3D images[J]. Medical Physics, 46, 2169-2180(2019).

    [60] Orieux F, Sepulveda E, Loriette V et al. Bayesian estimation for optimized structured illumination microscopy[J]. IEEE Transactions on Image Processing, 21, 601-614(2012).

    [61] Lal A, Shan C Y, Zhao K et al. A frequency domain SIM reconstruction algorithm using reduced number of images[J]. IEEE Transactions on Image Processing, 27, 4555-4570(2018).

    [62] Dong S Y, Liao J, Guo K K et al. Resolution doubling with a reduced number of image acquisitions[J]. Biomedical Optics Express, 6, 2946-2952(2015).

    [63] Jin L H, Liu B, Zhao F Q et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed[J]. Nature Communications, 11, 1934(2020).

    [64] Goodfellow I J, Pouget-Abadie J, Mirza M et al. Generative adversarial networks[EB/OL]. https:∥arxiv.org/abs/1406.2661

    [65] Zhu J Y, Park T, Isola P et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C], 2242-2251(2017).

    [66] Mirza M, Simon O. Conditional generative adversarial nets[EB/OL]. https:∥arxiv.org/abs/1411.1784

    [67] Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C], 214-223(2017).

    [68] Ling C, Zhang C L, Wang M Q et al. Fast structured illumination microscopy via deep learning[J]. Photonics Research, 8, 1350-1359(2020).

    [69] Zhang Q N, Chen J W, Li J S et al. Deep learning-based single-shot structured illumination microscopy[J]. Optics and Lasers in Engineering, 155, 107066(2022).

    [70] Chen X, Li B W, Jiang S W et al. Accelerated phase shifting for structured illumination microscopy based on deep learning[J]. IEEE Transactions on Computational Imaging, 7, 700-712(2021).

    [71] Shah Z H, Müller M, Wang T C et al. Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images[J]. Photonics Research, 9, B168-B181(2021).

    [72] Müller M, Mönkemöller V, Hennig S et al. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ[J]. Nature Communications, 7, 10980(2016).

    [73] Cheng X, Li J, Dai Q et al. Fast and lightweight network for single frame structured illumination microscopy super-resolution[J]. IEEE Transactions on Instrumentation and Measurement, 71, 5007711(2022).

    [74] Liao J H, Zhang C S, Xu X C et al. Deep-MSIM: fast image reconstruction with deep learning in multifocal structured illumination microscopy[J]. Advanced Science, 10, 2370185(2023).

    [75] Xypakis E, Gosti G, Giordani T et al. Deep learning for blind structured illumination microscopy[J]. Scientific Reports, 12, 8623(2022).

    [76] Wang H T, Tan J B, Liu C G et al. Wide-field profiling of smooth steep surfaces by structured illumination[J]. Optics Communications, 366, 241-247(2016).

    [77] Xie Z Y, Tang Y, Zhou Y et al. Surface and thickness measurement of transparent thin-film layers utilizing modulation-based structured-illumination microscopy[J]. Optics Express, 26, 2944-2953(2018).

    [78] Yang L T, Liu X L, Chu X X et al. Structured light three-dimensional reconstruction technology based on MultiResHNet[J]. Laser & Optoelectronics Progress, 60, 2015006(2023).

    [79] Feng S J, Chen Q, Gu G H et al. Fringe pattern analysis using deep learning[J]. Advanced Photonics, 1, 025001(2019).

    [80] Qian J M, Feng S J, Li Y X et al. Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry[J]. Optics Letters, 45, 1842-1845(2020).

    [81] Chai C C, Chen C, Liu X J et al. Deep learning based one-shot optically-sectioned structured illumination microscopy for surface measurement[J]. Optics Express, 29, 4010-4021(2021).

    [82] Cheng X L, Tang Y, Yang K J et al. Single-exposure height-recovery structured illumination microscopy based on deep learning[J]. Optics Letters, 47, 3832-3835(2022).

    Tools

    Get Citation

    Copy Citation Text

    Xinran Li, Jiajie Chen, Meiting Wang, Xiaomin Zheng, Peng Du, Yili Zhong, Xiaoqi Dai, Junle Qu, Yonghong Shao. Advancement in Structured Illumination Microscopy Based on Deep Learning[J]. Chinese Journal of Lasers, 2024, 51(21): 2107103

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Category: Biomedical Optical Imaging

    Received: Apr. 29, 2024

    Accepted: Jun. 13, 2024

    Published Online: Oct. 31, 2024

    The Author Email: Chen Jiajie (cjj@szu.edu.cn), Shao Yonghong (shaoyh@szu.edu.cn)

    DOI:10.3788/CJL240817

    CSTR:32183.14.CJL240817

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology