Laser & Optoelectronics Progress, Volume. 58, Issue 18, 1811006(2021)

Application of Deep Learning in Digital Holographic Microscopy

Zhang Meng1, Hao Ding1, Shouping Nie1, Jun Ma2, and Caojin Yuan1、*
Author Affiliations
  • 1Jiangsu Key Laboratory for Opto-Electronic Technology, School of Physics and Technology, Nanjing Normal University, Nanjing, Jiangsu 210023, China
  • 2School of Electronic Engineering and Optoelectronic Techniques, Nanjing University of Science and Technology, Nanjing, Jiangsu 210094, China
  • show less
    References(101)

    [1] Park Y K, Depeursinge C, Popescu G. Quantitative phase imaging in biomedicine[J]. Nature Photonics, 12, 578-589(2018).

    [2] Gabor D. A new microscopic principle[J]. Nature, 161, 777-778(1948).

    [3] Kim M K. Principles and techniques of digital holographic microscopy[J]. SPIE Reviews, 1, 018005(2010).

    [4] Stepanishen P R, Benjamin K C. Forward and backward projection of acoustic fields using FFT methods[J]. The Journal of the Acoustical Society of America, 71, 803-812(1982).

    [5] Marquet P, Rappaz B, Magistretti P J et al. Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy[J]. Optics Letters, 30, 468-470(2005).

    [6] Kemper B, Carl D D, Schnekenburger J et al. Investigation of living pancreas tumor cells by digital holographic microscopy[J]. Journal of biomedical optics, 11, 034005(2006).

    [7] Stern A, Javidi B. Theoretical analysis of three-dimensional imaging and recognition of micro-organisms with a single-exposure on-line holographic microscope[J]. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 24, 163-168(2007).

    [8] Doblas A I, Sánchez-Ortiga E, Martínez-Corral M et al. Accurate single-shot quantitative phase imaging of biological specimens with telecentric digital holographic microscopy[J]. Journal of Biomedical Optics, 19, 046022(2014).

    [9] Marquet P, Depeursinge C, Magistretti P J. Review of quantitative phase-digital holographic microscopy: promising novel imaging technique to resolve neuronal network activity and identify cellular biomarkers of psychiatric disorders[J]. Neurophotonics, 1, 020901(2014).

    [10] Pourvais Y, Asgari P, Abdollahi P et al. Microstructural surface characterization of stainless and plain carbon steel using digital holographic microscopy[J]. Journal of the Optical Society of America B, 34, B36-B41(2017).

    [11] Dai S Q, Dou J Z, Zhang J W et al. Digital holography based near-field imaging and its application[J]. Acta Optica Sinica, 40, 0111008(2020).

    [12] Cuche E, Marquet P, Depeursinge C. Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms[J]. Applied Optics, 38, 6994-7001(1999).

    [13] Brady D J, Choi K, Marks D L et al. Compressive holography[J]. Optics Express, 17, 13040-13049(2009).

    [14] Liu W B, Wang Z D, Liu X H et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing, 234, 11-26(2017).

    [15] Bottou L. Large-scale machine learning with stochastic gradient descent[M]. //Lechevallier Y, Saporta G. Proceedings of COMPSTAT’2010, 177-186(2010).

    [16] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 521, 436-444(2015).

    [17] Russakovsky O, Deng J, Su H et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 115, 211-252(2015).

    [18] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 25, 1097-1105(2012).

    [19] Zhang J L, Chen Q, Zhang X Y et al. Lens-free on-chip microscopy: theory, advances, and applications[J]. Infrared and Laser Engineering, 48, 0603009(2019).

    [20] Halupka K J, Antony B J, Lee M H et al. Retinal optical coherence tomography image enhancement via deep learning[J]. Biomedical Optics Express, 9, 6205-6221(2018).

    [21] Huang Y Q, Lu Z X, Shao Z M et al. Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network[J]. Optics Express, 27, 12289-12307(2019).

    [22] Rahmani B, Loterie D, Konstantinou G et al. Multimode optical fiber transmission with a deep learning network[J]. Light, Science & Applications, 7, 69(2018).

    [23] He Y C, Wang G, Dong G X et al. Ghost imaging based on deep learning[J]. Scientific Reports, 8, 6469(2018).

    [24] Li Y Z, Xue Y J, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media[J]. Optica, 5, 1181-1190(2018).

    [25] Goy A, Arthur K, Li S et al. Low photon count phase retrieval using deep learning[J]. Physical Review Letters, 121, 243902(2018).

    [26] Zhang G, Guan T, Shen Z Y et al. Fast phase retrieval in off-axis digital holographic microscopy through deep learning[J]. Optics Express, 26, 19388-19405(2018).

    [27] Xiao W, Wang Q X, Pan F et al. Adaptive frequency filtering based on convolutional neural networks in off-axis digital holographic microscopy[J]. Biomedical Optics Express, 10, 1613-1626(2019).

    [28] Rivenson Y, Zhang Y, Günaydın H et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light, Science & Applications, 7, 17141(2018).

    [29] Barbastathis G, Ozcan A, Situ G H. On the use of deep learning for computational imaging[J]. Optica, 6, 921-943(2019).

    [30] Wang F, Wang H, Bian Y M et al. Applications of deep learning in computational imaging[J]. Acta Optica Sinica, 40, 0111002(2020).

    [31] Wang H, Lyu M, Situ G H. eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction[J]. Optics Express, 26, 22603-22614(2018).

    [32] Zuo C, Feng S J, Zhang X Y et al. Deep learning based computational imaging: status, challenges, and future[J]. Acta Optica Sinica, 40, 0111003(2020).

    [33] Albawi S, Mohammed T A, Al-Zawi S. Understanding of a convolutional neural network[C]. //2017 International Conference on Engineering and Technology (ICET), August 21-23, 2017, Antalya, Turkey., 1-6(2017).

    [34] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines[C]. //Proceedings of the 27th International Conference on Machine Learnin (ICML-10), June 21-24, 2010, Haifa, Israel, 807-814(2010).

    [35] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. // Proceedings of the 32nd International Conference on Machine Learning, (ICML) 2015, July 6-11, 2015, Lille, France, 448-456(2015).

    [36] Srivastava N, Hinton G, Krizhevsky A et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 15, 1929-1958(2014).

    [37] He K M, Zhang X Y, Ren S Q et al. Deep residual learning for image recognition[C]. //2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA., 770-778(2016).

    [38] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[M]. //Navab N, Hornegger J, Wells W M, et al. Medical image computing and computer-assisted intervention-MICCAI 2015. Lecture notes in computer science, 9351, 234-241(2015).

    [39] Huber P J. Robust estimation of a location parameter[M]. //Kotz S, Johnson N L. Breakthroughs in statistics. Springer series in statistics, 492-518(1992).

    [40] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution[M]. //Leibe B, Matas J, Sebe N, et al. Computer vision-ECCV 2016. Lecture notes in computer science, 9906, 694-711(2016).

    [41] Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors[J]. Nature, 323, 533-536(1986).

    [42] Qian N. On the momentum term in gradient descent learning algorithms[J]. Neural Networks, 12, 145-151(1999).

    [43] Nesterov Y. A method for solving the convex programming problem with convergence rate O (1/k2)[J]. Dokl. Akad. Nauk SSSR, 269, 543-547(1983).

    [44] Duchi J C, Hazan E, Singer Y. Adaptive subgradient methods for online learning and stochastic optimization[J]. Journal of Machine Learning Research, 12, 257-269(2011).

    [48] McCann M T, Jin K H, Unser M. Convolutional neural networks for inverse problems in imaging: a review[J]. IEEE Signal Processing Magazine, 34, 85-95(2017).

    [49] Leith E N, Upatnieks J. Reconstructed wavefronts and communication theory[J]. Journal of the Optical Society of America, 52, 1123-1130(1962).

    [50] Rivenson Y, Wu Y C, Ozcan A. Deep learning in holography and coherent imaging[J]. Light: Science & Applications, 8, 85(2019).

    [51] Schnars U, Jptner W P O. Digital recording and numerical reconstruction of holograms[J]. Measurement Science and Technology, 13, R85-R101(2002).

    [52] de Nicola S, Finizio A, Pierattini G et al. Angular spectrum method with correction of anamorphism for numerical reconstruction of digital holograms on tilted planes[J]. Optics Express, 13, 9935-9940(2005).

    [53] Sinha A, Lee J, Li S et al. Lensless computational imaging through deep learning[J]. Optica, 4, 1117-1125(2017).

    [54] Wu Y C, Rivenson Y, Zhang Y B et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery[J]. Optica, 5, 704-710(2018).

    [55] Wang K, Dou J, Kemao Q et al. Y-Net: a one-to-two deep learning framework for digital holographic reconstruction[J]. Optics Letters, 44, 4765-4768(2019).

    [56] Wang K, Kemao Q, Di J et al. Y4-Net: a deep learning solution to one-shot dual-wavelength digital holographic reconstruction[J]. Optics Letters, 45, 4220-4223(2020).

    [57] Yin D, Gu Z Z, Zhang Y R et al. Digital holographic reconstruction based on deep learning framework with unpaired data[J]. IEEE Photonics Journal, 12, 1-12(2019).

    [58] Zhu J Y, Park T, Isola P et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. //2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice, Italy., 2242-2251(2017).

    [59] Guo J, Zeng W H, Xie T B. Fast autofocus method based on entropy function[J]. Metrology Technology, 11, 30-32(2003).

    [60] Subbarao M, Tyan J K. Selecting the optimal focus measure for autofocusing and depth-from-focus[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 864-870(1998).

    [61] Memmolo P, Distante C, Paturzo M et al. Automatic focusing in digital holography and its application to stretched holograms[J]. Optics Letters, 36, 1945-1947(2011).

    [62] Lyu M, Yuan C, Li D et al. Fast autofocusing in digital holography using the magnitude differential[J]. Applied Optics, 56, F152-F157(2017).

    [63] Zhang Y, Wang H, Wu Y et al. Edge sparsity criterion for robust holographic autofocusing[J]. Optics Letters, 42, 3824-3827(2017).

    [64] Pitkäaho T, Manninen A, Naughton T J. Focus classification in digital holographic microscopy using deep convolutional neural networks[J]. Proceedings of SPIE, 10414, 104140K(2017).

    [65] Ren Z B, Xu Z M, Lam E Y. Learning-based nonparametric autofocusing for digital holography[J]. Optica, 5, 337-344(2018).

    [66] Ren Z B, Xu Z M, Lam E Y M. End-to-end deep learning framework for digital holographic reconstruction[J]. Advanced Photonics, 1, 016004(2019).

    [67] Pitkäaho T, Manninen A, Naughton T J. Focus prediction in digital holographic microscopy using deep convolutional neural networks[J]. Applied Optics, 58, A202-A208(2019).

    [68] Jaferzadeh K, Hwang S H, Moon I et al. No-search focus prediction at the single cell level in digital holographic imaging with deep convolutional neural network[J]. Biomedical Optics Express, 10, 4276-4289(2019).

    [71] Liu S J, Wang D, Li S J et al. Speckle noise suppression method in holographic display using time multiplexing[J]. Optical Engineering, 56, 063107(2017).

    [73] Bianco V, Paturzo M, Memmolo P et al. Random resampling masks: a non-Bayesian one-shot strategy for noise reduction in digital holography[J]. Optics Letters, 38, 619-621(2013).

    [74] Bianco V, Memmolo P, Paturzo M et al. Quasi noise-free digital holography[J]. Light, Science & Applications, 5, e16142(2016).

    [75] Choi G, Ryu D, Jo Y et al. Cycle-consistent deep learning approach to coherent noise reduction in optical diffraction tomography[J]. Optics Express, 27, 4927-4943(2019).

    [76] Jeon W, Jeong W, Son K et al. Speckle noise reduction for digital holographic images using multi-scale convolutional neural networks[J]. Optics Letters, 43, 4240-4243(2018).

    [77] Yin D, Gu Z Z, Zhang Y R et al. Speckle noise reduction in coherent imaging based on deep learning without clean data[J]. Optics and Lasers in Engineering, 133, 106151(2020).

    [79] Lindley D V. Fiducial distributions and Bayes’ theorem[J]. Journal of the Royal Statistical Society: Series B (Methodological), 20, 102-107(1958).

    [80] Kang I, Zhang F C, Barbastathis G. Phase extraction neural network (PhENN) with coherent modulation imaging (CMI) for phase retrieval at low photon counts[J]. Optics Express, 28, 21578-21600(2020).

    [81] Zhang Z D, Zheng Y J, Xu T N et al. Holo-UNet: hologram-to-hologram neural network restoration for high fidelity low light quantitative phase imaging of live cells[J]. Biomedical Optics Express, 11, 5478-5487(2020).

    [82] Liu T, de Haan K, Rivenson Y et al. Deep learning-based super-resolution in coherent imaging systems[J]. Scientific Reports, 9, 3926(2019).

    [83] Mico V, Zalevsky Z, García-Martínez P et al. Synthetic aperture superresolution with multiple off-axis holograms[J]. Journal of the Optical Society of America A, 23, 3162-3170(2006).

    [84] Bishara W, Su T W, Coskun A F et al. Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution[J]. Optics Express, 18, 11181-11191(2010).

    [85] Luo W, Zhang Y B, Feizi A et al. Pixel super-resolution using wavelength scanning[J]. Light, Science & Applications, 5, e16060(2016).

    [88] Fournier C, Jolivet F, Denis L et al. Pixel super-resolution in digital holography by regularized reconstruction[J]. Applied Optics, 56, 69-77(2016).

    [89] Go T, Lee S, You D et al. Deep learning-based hologram generation using a white light source[J]. Scientific Reports, 10, 8977(2020).

    [90] Rivenson Y, Liu T, Wei Z et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning[J]. Light, Science & Applications, 8, 23(2019).

    [91] Wu Y C, Luo Y L, Chaudhari G et al. Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram[J]. Light, Science & Applications, 8, 25(2019).

    [92] Kamilov U S, Papadopoulos I N, Shoreh M H et al. Learning approach to optical tomography[J]. Optica, 2, 517-522(2015).

    [93] Di J, Han W, Liu S et al. Sparse-view imaging of afiber internal structure in holographic diffraction tomography via a convolutional neural network[J]. Applied Optics, 60, A234-A242(2021).

    [94] Wang H D, Rivenson Y, Jin Y Y et al. Deep-learning enables cross-modality super-resolution in fluorescence microscopy[J]. Nature Methods, 16, 103-110(2019).

    [95] Nguyen T, Bui V, Lam V et al. Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection[J]. Optics Express, 25, 15043-15057(2017).

    [96] Jo Y J, Park S, Jung J H et al. Holographic deep learning for rapid optical screening of anthrax spores[J]. Science Advances, 3, e1700606(2017).

    [97] Kakkava E, Rahmani B, Borhani N et al. Imaging through multimode fibers using deep learning: the effects of intensity versus holographic recording of the speckle pattern[J]. Optical Fiber Technology, 52, 101985(2019).

    [98] Meng Z, Ding L Q, Feng S T et al. Numerical dark-field imaging using deep-learning[J]. Optics Express, 28, 34266-34278(2020).

    [99] Jo Y J, Cho H, Lee S Y et al. Quantitative phase imaging and artificial intelligence: a review[J]. IEEE Journal of Selected Topics in Quantum Electronics, 25, 1-14(2019).

    [100] Pan S J, Yang Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 22, 1345-1359(2010).

    [101] Deng J, Dong W, Socher R et al. ImageNet: a large-scale hierarchical image database[C]. //2009 IEEE Conference on Computer Vision and Pattern Recognition, June 20-25, 2009, Miami, FL, USA, 248-255(2009).

    Tools

    Get Citation

    Copy Citation Text

    Zhang Meng, Hao Ding, Shouping Nie, Jun Ma, Caojin Yuan. Application of Deep Learning in Digital Holographic Microscopy[J]. Laser & Optoelectronics Progress, 2021, 58(18): 1811006

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Imaging Systems

    Received: Jun. 2, 2021

    Accepted: Jul. 20, 2021

    Published Online: Sep. 3, 2021

    The Author Email: Yuan Caojin (yuancj@njnu.edu.cn)

    DOI:10.3788/LOP202158.1811006

    Topics