Chinese Journal of Lasers, Volume. 50, Issue 3, 0307104(2023)
Advances in Rapid Three-Dimensional Wide Field Microscopy
[1] Wilson T, Masters B R. Confocal microscopy[J]. Applied Optics, 33, 565-566(1994).
[2] Dinsmore A D, Weeks E R, Prasad V et al. Three-dimensional confocal microscopy of colloids[J]. Applied Optics, 40, 4152-4159(2001).
[3] Helmchen F, Denk W. Deep tissue two-photon microscopy[J]. Nature Methods, 2, 932-940(2005).
[4] Denk W, Strickler J H, Webb W W. Two-photon laser scanning fluorescence microscopy[J]. Science, 248, 73-76(1990).
[5] Park J H, Sun W, Cui M. High-resolution in vivo imaging of mouse brain through the intact skull[J]. Proceedings of the National Academy of Sciences of the United States of America, 112, 9236-9241(2015).
[6] Keller P J, Schmidt A D, Wittbrodt J et al. Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy[J]. Science, 322, 1065-1069(2008).
[7] Neil M A A, Juškaitis R, Wilson T. Method of obtaining optical sectioning by using structured light in a conventional microscope[J]. Optics Letters, 22, 1905-1907(1997).
[8] Gustafsson M G L, Shao L, Carlton P M et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination[J]. Biophysical Journal, 94, 4957-4970(2008).
[9] Huisken J, Swoger J, del Bene F et al. Optical sectioning deep inside live embryos by selective plane illumination microscopy[J]. Science, 305, 1007-1009(2004).
[10] Planchon T A, Gao L, Milkie D E et al. Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination[J]. Nature Methods, 8, 417-423(2011).
[11] Chen B C, Legant W R, Wang K et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution[J]. Science, 346, 1257998(2014).
[12] Mahou P, Vermot J, Beaurepaire E et al. Multicolor two-photon light-sheet microscopy[J]. Nature Methods, 11, 600-601(2014).
[13] Nayar S K, Nakagawa Y. Shape from focus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 824-831(1994).
[14] Zuo C, Qian J M, Feng S J et al. Deep learning in optical metrology: a review[J]. Light: Science & Applications, 11, 39(2022).
[15] Wu Y C, Rivenson Y, Wang H D et al. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning[J]. Nature Methods, 16, 1323-1331(2019).
[16] Huang L Z, Chen H L, Luo Y L et al. Recurrent neural network-based volumetric fluorescence microscopy[J]. Light: Science & Applications, 10, 62(2021).
[17] Zhang X Y, Chen Y F, Ning K F et al. Deep learning optical-sectioning method[J]. Optics Express, 26, 30762-30772(2018).
[18] Prevedel R, Yoon Y G, Hoffmann M et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy[J]. Nature Methods, 11, 727-730(2014).
[19] Li H, Guo C, Kim-Holzapfel D et al. Fast, volumetric live-cell imaging using high-resolution light-field microscopy[J]. Biomedical Optics Express, 10, 29-49(2018).
[20] Kim H M, Yoo Y J, Lee J M et al. A wide field-of-view light-field camera with adjustable multiplicity for practical applications[J]. Sensors, 22, 3455(2022).
[21] Wang Z J, Cai Y N, Liang Y S et al. Single shot, three-dimensional fluorescence microscopy with a spatially rotating point spread function[J]. Biomedical Optics Express, 8, 5493-5506(2017).
[22] Wang Z J, Cai Y N, Qian J et al. Hybrid multifocal structured illumination microscopy with enhanced lateral resolution and axial localization capability[J]. Biomedical Optics Express, 11, 3058-3070(2020).
[23] Cao B, Cao H Q, Lin D Y et al. Research progress of double-helix point spread function engineering and its application[J]. Laser & Optoelectronics Progress, 59, 1800001(2022).
[24] Stokseth P A. Properties of a defocused optical system[J]. Journal of the Optical Society of America, 59, 1314-1321(1969).
[25] Jang H S, Yun G, Mutahira H et al. A new focus measure operator for enhancing image focus in 3D shape recovery[J]. Microscopy Research and Technique, 84, 2483-2493(2021).
[26] Zhang Z Q, Liu F, Zhou Z J et al. Roughness measurement of leaf surface based on shape from focus[J]. Plant Methods, 17, 72(2021).
[27] Pertuz S, Garcia M A, Puig D et al. A closed-form focus profile model for conventional digital cameras[J]. International Journal of Computer Vision, 124, 273-286(2017).
[28] Sahay R R, Rajagopalan A N. A model-based approach to shape from focus[C], 243-250(2008).
[29] Huang W, Jing Z L. Evaluation of focus measures in multi-focus image fusion[J]. Pattern Recognition Letters, 28, 493-500(2007).
[30] Wee C Y, Paramesran R. Measure of image sharpness using eigenvalues[J]. Information Sciences, 177, 2533-2552(2007).
[31] Sun Y, Duthaler S, Nelson B J. Autofocusing in computer microscopy: selecting the optimal focus algorithm[J]. Microscopy Research and Technique, 65, 139-149(2004).
[32] Yang G, Nelson B J. Wavelet-based autofocusing and unsupervised segmentation of microscopic images[C], 2143-2148(2003).
[33] Baina J, Dublet J. Automatic focus and iris control for video cameras[C], 232-235(2002).
[34] Shen C H, Chen H H. Robust focus measure for low-contrast images[C], 69-70(2006).
[35] Wang X F, Sun X W, Wang J K et al. Focus measure operator combining cosine transform and Laplacian operator[J]. Laser & Optoelectronics Progress, 58, 2410005(2021).
[36] Pertuz S, Puig D, Garcia M A. Analysis of focus measure operators for shape-from-focus[J]. Pattern Recognition, 46, 1415-1432(2013).
[37] Shang M H, Yu F H. Research on microscopic 3D measurement system based on focus variation[J]. Laser & Optoelectronics Progress, 58, 1600002(2021).
[38] Xiong Y, Shafer S A. Depth from focusing and defocusing[C], 68-73(1993).
[39] Marshall J A, Burbeck C A, Ariely D et al. Occlusion edge blur: a cue to relative visual depth[J]. Journal of the Optical Society of America A, 13, 681-688(1996).
[40] Malik A S, Choi T S. Consideration of illumination effects and optimization of window size for accurate calculation of depth map for 3D shape recovery[J]. Pattern Recognition, 40, 154-170(2007).
[41] Montgomery D C, Runger G C. Applied statistics and probability for engineers[J]. European Journal of Engineering Education, 19, 516-517(1994).
[42] Yan T, Hu Z G, Qian Y H et al. 3D shape reconstruction from multifocus image fusion using a multidirectional modified Laplacian operator[J]. Pattern Recognition, 98, 107065(2020).
[43] Subbarao M, Choi T. Accurate recovery of three-dimensional shape from image focus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17, 266-274(1995).
[44] Choi T S, Yun J. Three-dimensional shape recovery from the focused-image surface[J]. Optical Engineering, 39, 1321-1326(2000).
[45] Asif M, Choi T S. Shape from focus using multilayer feedforward neural networks[J]. IEEE Transactions on Image Processing, 10, 1670-1675(2001).
[46] Ahmad M B, Choi T S. A heuristic approach for finding best focused shape[J]. IEEE Transactions on Circuits and Systems for Video Technology, 15, 566-574(2005).
[47] Ahmad M B, Choi T S. Shape from focus using optimization technique[C](2006).
[48] Shim S O, Choi T S. A novel iterative shape from focus algorithm based on combinatorial optimization[J]. Pattern Recognition, 43, 3338-3347(2010).
[49] Muhammad M S, Choi T S. Sampling for shape from focus in optical microscopy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 564-573(2012).
[50] Jang H S, Muhammad M S, Choi T S. Optimizing image focus for shape from focus through locally weighted non-parametric regression[J]. IEEE Access, 7, 74393-74400(2019).
[51] Mahmood M T, Choi T S. Nonlinear approach for enhancement of image focus volume in shape from focus[J]. IEEE Transactions on Image Processing, 21, 2866-2873(2012).
[52] Wang Z J, Lei M, Yao B L et al. Compact multi-band fluorescent microscope with an electrically tunable lens for autofocusing[J]. Biomedical Optics Express, 6, 4353-4364(2015).
[53] Jabbour J M, Malik B H, Olsovsky C et al. Optical axial scanning in confocal microscopy using an electrically tunable lens[J]. Biomedical Optics Express, 5, 645-652(2014).
[54] Philipp K, Smolarski A, Koukourakis N et al. Volumetric HiLo microscopy employing an electrically tunable lens[J]. Optics Express, 24, 15029-15041(2016).
[55] Zhou H, Zhang X F, Xu Z J et al. Universal membrane-based tunable liquid lens design for dynamically correcting spherical aberration over user-defined focal length range[J]. Optics Express, 27, 37667-37679(2019).
[56] Minhas R, Mohammed A A, Wu Q M J et al. 3D shape from focus and depth map computation using steerable filters[M]. Kamel M, Campilho A. Image analysis and recognition. Lecture notes in computer science, 5627, 573-583(2009).
[57] Gustafsson M G L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy[J]. Journal of Microscopy, 198, 82-87(2000).
[58] Gustafsson M G L. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 102, 13081-13086(2005).
[59] Rego E H, Shao L, Macklin J J et al. Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 109, E135-E143(2012).
[60] Ford T N, Lim D, Mertz J. Fast optically sectioned fluorescence HiLo endomicroscopy[J]. Journal of Biomedical Optics, 17, 021105(2012).
[61] Zhou X, Lei M, Dan D et al. Double-exposure optical sectioning structured illumination microscopy based on Hilbert transform reconstruction[J]. PLoS One, 10, e0120892(2015).
[62] Dan D, Yao B L, Lei M. Structured illumination microscopy for super-resolution and optical sectioning[J]. Chinese Science Bulletin, 59, 1291-1307(2014).
[63] Lim D, Chu K K, Mertz J. Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy[J]. Optics Letters, 33, 1819-1821(2008).
[64] Monneret S, Rauzi M, Lenne P F. Highly flexible whole-field sectioning microscope with liquid-crystal light modulator[J]. Journal of Optics A: Pure and Applied Optics, 8, S461-S466(2006).
[65] Dan D, Lei M, Yao B L et al. DMD-based LED-illumination super-resolution and optical sectioning microscopy[J]. Scientific Reports, 3, 1116(2013).
[66] Qian J, Lei M, Dan D et al. Full-color structured illumination optical sectioning microscopy[J]. Scientific Reports, 5, 14513(2015).
[67] Zhou X, Lei M, Dan D et al. Image recombination transform algorithm for superresolution structured illumination microscopy[J]. Journal of Biomedical Optics, 21, 096009(2016).
[68] Wicker K, Heintzmann R. Single-shot optical sectioning using polarization-coded structured illumination[J]. Journal of Optics, 12, 084010(2010).
[69] Patorski K, Trusiak M, Tkaczyk T. Optically-sectioned two-shot structured illumination microscopy with Hilbert-Huang processing[J]. Optics Express, 22, 9517-9527(2014).
[70] Gong H, Xu D L, Yuan J et al. High-throughput dual-colour precision imaging for brain-wide connectome with cytoarchitectonic landmarks at the cellular level[J]. Nature Communications, 7, 12142(2016).
[71] Qian J, Dang S P, Wang Z J et al. Large-scale 3D imaging of insects with natural color[J]. Optics Express, 27, 4845-4857(2019).
[72] Qian J, Dang S P, Zhou X et al. Fast structured illumination three-dimensional color microscopic imaging method based on Hilbert-transform[J]. Acta Physica Sinica, 69, 128701(2020).
[73] Wang Z J, Feng K, Yang F et al. Breathing colour into fossils: a tomographic system for reconstructing the soft tissue microstructure of amber inclusions[J]. Optics and Lasers in Engineering, 148, 106775(2022).
[74] Zhao T Y, Feng K, Liu P Y et al. Reconstructing the color 3D tomography of lunar samples[J]. Atomic Spectroscopy, 43, 6-12(2022).
[75] Karadaglić D, Wilson T. Image formation in structured illumination wide-field fluorescence microscopy[J]. Micron, 39, 808-818(2008).
[76] Wang M R, Zhao T Y, Wang Z J et al. Three-dimensional natural color imaging based on focus level correlation algorithm using structured illumination microscopy[J]. Frontiers in Physics, 10, 1041577(2022).
[77] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 521, 436-444(2015).
[78] Schmidhuber J. Deep learning in neural networks: an overview[J]. Neural Networks, 61, 85-117(2015).
[79] Wainberg M, Merico D, Delong A et al. Deep learning in biomedicine[J]. Nature Biotechnology, 36, 829-838(2018).
[80] Belthangady C, Royer L A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction[J]. Nature Methods, 16, 1215-1225(2019).
[81] Rivenson Y, Göröcs Z, Günaydin H et al. Deep learning microscopy[J]. Optica, 4, 1437-1443(2017).
[82] Ouyang W, Aristov A, Lelek M et al. Deep learning massively accelerates super-resolution localization microscopy[J]. Nature Biotechnology, 36, 460-468(2018).
[83] Wu Y C, Luo Y L, Chaudhari G et al. Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram[J]. Light: Science & Applications, 8, 25(2019).
[84] de Haan K, Rivenson Y, Wu Y C et al. Deep-learning-based image reconstruction and enhancement in optical microscopy[J]. Proceedings of the IEEE, 108, 30-50(2020).
[85] Weigert M, Schmidt U, Boothe T et al. Content-aware image restoration: pushing the limits of fluorescence microscopy[J]. Nature Methods, 15, 1090-1097(2018).
[86] Barbastathis G, Ozcan A, Situ G H. On the use of deep learning for computational imaging[J]. Optica, 6, 921-943(2019).
[87] Goodfellow I, Pouget-Abadie J, Mirza M et al. 2014. Generative adversarial nets[C], 2672-2680(2014).
[88] Liu Z, Jin L, Chen J et al. A survey on applications of deep learning in microscopy image analysis[J]. Computers in biology and medicine, 134, 104523(2021).
[89] Li B W, Tan S Y, Dong J Y et al. Deep-3D microscope: 3D volumetric microscopy of thick scattering samples using a wide-field microscope and machine learning[J]. Biomedical Optics Express, 13, 284-299(2021).
[90] Ning K F, Zhang X Y, Gao X F et al. Deep-learning-based whole-brain imaging at single-neuron resolution[J]. Biomedical Optics Express, 11, 3567-3584(2020).
[91] Zhuge H M, Summa B, Hamm J et al. Deep learning 2D and 3D optical sectioning microscopy using cross-modality Pix2Pix cGAN image translation[J]. Biomedical Optics Express, 12, 7526-7543(2021).
[92] Bai C, Qian J, Dang S P et al. Full-color optically-sectioned imaging by wide-field microscopy via deep-learning[J]. Biomedical Optics Express, 11, 2619-2632(2020).
[93] Ling C, Zhang C L, Wang M Q et al. Fast structured illumination microscopy via deep learning[J]. Photonics Research, 8, 1350-1359(2020).
[94] Jin L H, Liu B, Zhao F Q et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed[J]. Nature Communications, 11, 1934(2020).
Get Citation
Copy Citation Text
Jingrong Ren, Xiangda Fu, Mengrui Wang, Tianyu Zhao, Zhaojun Wang, Kun Feng, Yansheng Liang, Shaowei Wang, Ming Lei. Advances in Rapid Three-Dimensional Wide Field Microscopy[J]. Chinese Journal of Lasers, 2023, 50(3): 0307104
Category: Biomedical Optical Imaging
Received: Oct. 8, 2022
Accepted: Nov. 2, 2022
Published Online: Feb. 6, 2023
The Author Email: Lei Ming (ming.lei@mail.xjtu.edn.cn)