Chinese Journal of Lasers, Volume. 51, Issue 21, 2107103(2024)

Advancement in Structured Illumination Microscopy Based on Deep Learning

Xinran Li1, Jiajie Chen1、*, Meiting Wang2, Xiaomin Zheng1, Peng Du1, Yili Zhong1, Xiaoqi Dai1, Junle Qu1, and Yonghong Shao1、**
Author Affiliations
  • 1College of Physics and Optoelectronic Engineering, Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province, Shenzhen University, Shenzhen 518060, Guangdong , China
  • 2College of Biomedical Engineering, Department of Medicine, State-Local Joint Engineering Laboratory of Key Technology of Medical Ultrasound, Key Laboratory of Biomedical Information Detection and Ultrasound Imaging and Guangdong Province, Shenzhen University, Shenzhen 518060, Guangdong , China
  • show less

    Significance

    Structured illumination microscopy (SIM) is a pivotal technique in super-resolution microscopy as it offers an innovative approach to enhance the spatial resolution exceedingly beyond that achievable by conventional optical microscopes. SIM harnesses the principle of structured illumination, where finely patterned light interacts with the specimen, thereby generating moiré fringes containing high-frequency information that is otherwise unaccessible owing to the diffraction limit.

    Achieving genuine super-resolution via SIM is involves intricate steps, including capturing numerous low-resolution images under an array of varied illumination patterns. Each of these images encapsulates a unique set of moiré patterns, which serve as the foundation for the subsequent computational reconstruction of a high-resolution image. Although effective, this methodology presents some challenges. Biological samples, owing to their inherent irregularities and varying tissue thicknesses, can result in considerable variability in the quality and consistency of the captured moiré patterns. This variability hinders the accurate reconstruction of high-resolution images. Additionally, systematic errors can further complicate the process, thus potentially introducing artifacts or resulting in the loss of crucial details in the final image.

    Furthermore, sample damage due to prolonged light exposure must be considered when acquiring multiple images. Hence, the number of images required must be minimized without compromising the quality of the super-resolution reconstruction. Determining the optimal balance between the number of images and the quality of the final image is key in applying SIM to sensitive biological samples.

    Image-processing algorithms are widely employed to mitigate the effect of excessive image pairs on imaging results. In addition to the classical algorithms, recently developed deep-learning algorithms offer promising solutions. Deep-learning algorithms can extract meaningful information from limited data and efficiently reconstruct images using neural networks. This approach enables high-quality super-resolution images to be acquired faster without necessitating numerous input images. Consequently, in SIM image reconstruction, satisfactory results can be achieved using fewer input images. Furthermore, deep-learning algorithms can effectively manage irregularities and variations in samples. By learning the structure and features of samples, these algorithms can better adapt to different types of samples, thus improving the robustness and accuracy of image reconstruction. This is particularly important when managing complex biological samples, which typically exhibit diversity and variability. Therefore, analyzing and summarizing the applications and effectiveness of deep learning in SIM systems is crucial.

    Progress

    In deep learning, the widely recognized efficient neural network models include the convolutional neural network (CNN), U-Net, and generative adversarial network (GAN). The CNN, which is renowned for its capacity to automatically discern patterns and features within intricate datasets, is particularly suitable for the task mentioned above. By undergoing rigorous training on a substantial corpus of SIM images, the CNN learns to infer missing information that would otherwise require an array of supplementary images to capture. This predictive prowess enables the algorithm to amend the aberrations induced by SIM mode adjustments, thus significantly improving the quality of the reconstructed images. Because of the strategic deployment of skip connections within U-Net, which ingeniously amalgamates information from both the deeper and shallower layers, the network can effectively preserve abundant details and information throughout the upsampling phase. Furthermore, the integration of deconvolution processes not only amplifies the dimensions of the output image but is also pivotal in enhancing U-Net’s exceptional performance and widespread acceptance within the biomedical sector. In the context of SIM reconstruction, harnessing U-Net to extract supplementary insights from available images allows the algorithm to construct high-resolution images from a minimal subset of input images, thereby considerably diminishing the likelihood of specimen damage. By employing U-Net, one can reconstruct a super-resolved image similar to those afforded by classical algorithms using only three captured images. Furthermore, the implementation of GANs has significantly augmented the capabilities of deep-learning algorithms in SIM image processing. GANs comprise two dueling neural networks—a generator and a discriminator—that operate in tandem to fabricate highly realistic images. The generator synthesizes the images, whereas the discriminator assesses their veracity. Similar to U-Net, GANs can reconstruct super-resolved images from three original images. However, GANs can generate data through adversarial learning, and when coupled with other architectures, they can achieve even better results.

    In summary, to enhance performance and generate high-resolution images from a minimal number of original images, various neural network models are synergistically combined. Finally, the application of deep learning in nonstriped and non-super-resolution SIM yields encouraging results, thereby further expanding the possibility of its applicability.

    Conclusions and Prospects

    The integration of deep-learning algorithms into SIM image processing significantly advances the microscopy field. It not only addresses the technical challenges associated with achieving super-resolution but also provides new possibilities for investigating the nanoscale world with unprecedented clarity and detail. As deep-learning algorithms continue to advance, we expect more sophisticated algorithms to emerge and thus transcend the current boundaries of super-resolution microscopy.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Xinran Li, Jiajie Chen, Meiting Wang, Xiaomin Zheng, Peng Du, Yili Zhong, Xiaoqi Dai, Junle Qu, Yonghong Shao. Advancement in Structured Illumination Microscopy Based on Deep Learning[J]. Chinese Journal of Lasers, 2024, 51(21): 2107103

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Biomedical Optical Imaging

    Received: Apr. 29, 2024

    Accepted: Jun. 13, 2024

    Published Online: Oct. 31, 2024

    The Author Email: Chen Jiajie (cjj@szu.edu.cn), Shao Yonghong (shaoyh@szu.edu.cn)

    DOI:10.3788/CJL240817

    CSTR:32183.14.CJL240817

    Topics