Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Photonics Research, Volume. 9, Issue 2, B30(2021)

High-fidelity image reconstruction for compressed ultrafast photography via an augmented-Lagrangian and deep-learning hybrid algorithm

Chengshuai Yang1, Yunhua Yao1,6、*, Chengzhi Jin1, Dalong Qi1, Fengyan Cao1, Yilin He1, Jiali Yao1, Pengpeng Ding1, Liang Gao2, Tianqing Jia1, Jinyang Liang3, Zhenrong Sun1, and Shian Zhang1,4,5,7、*
Author Affiliations
  • 1State Key Laboratory of Precision Spectroscopy, School of Physics and Electronic Science, East China Normal University, Shanghai 200062, China
  • 2Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
  • 3Institut National de la Recherche Scientifique, Centre Énergie Matériaux Télécommunications, Laboratory of Applied Computational Imaging, Varennes, Québec J3X1S2, Canada
  • 4Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan 030006, China
  • 5Collaborative Innovation Center of Light Manipulations and Applications, Shandong Normal University, Jinan 250358, China
  • 6e-mail: yhyao@lps.ecnu.edu.cn
  • 7e-mail: sazhang@phy.ecnu.edu.cn
    • show less
    Full TextGet PDF
    Figures&Tables (7)
    Equations (16)
    References (62)
    Cited By (23)
    Tools
    Paper Information
    Topics
    References(62)

    [1] P. R. Poulin, K. A. Nelson. Irreversible organic crystalline chemistry monitored in real time. Science, 313, 1756-1760(2006).

    [2] P. Hockett, C. Z. Bisgaard, O. J. Clarkin, A. Stolow. Time-resolved imaging of purely valence-electron dynamics during a chemical reaction. Nat. Phys., 7, 612-615(2011).

    [3] R. Horstmeyer, H. Ruan, C. Yang. Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue. Nat. Photonics, 9, 563-571(2015).

    [4] J. W. Borst, A. J. Visser. Fluorescence lifetime imaging microscopy in life sciences. Meas. Sci. Technol., 21, 102002(2010).

    [5] H. R. Petty. Spatiotemporal chemical dynamics in living cells: from information trafficking to cell physiology. Biosystems, 83, 217-224(2006).

    [6] T. M. Squires, S. R. Quake. Microfluidics: fluid physics at the nanoliter scale. Rev. Mod. Phys., 77, 977-1026(2005).

    [7] N. Šiaulys, L. Gallais, A. Melninkaitis. Direct holographic imaging of ultrafast laser damage process in thin films. Opt. Lett., 39, 2164-2167(2014).

    [8] R. L. Kodama, P. A. Norreys, K. Mima, A. E. Dangor, R. G. Evans, H. Fujita, Y. Kitagawa, K. Krushelnick, T. Miyakoshi, N. Miyanaga. Fast heating of ultrahigh-density plasma as a step towards laser fusion ignition. Nature, 412, 798-802(2001).

    [9] L. Gao, J. Liang, C. Li, L. V. Wang. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature, 516, 74-77(2014).

    [10] J. Liang, L. Zhu, L. V. Wang. Single-shot real-time femtosecond imaging of temporal focusing. Light Sci. Appl., 7, 42(2018).

    [11] D. Qi, S. Zhang, C. Yang, Y. He, F. Cao, J. Yao, P. Ding, L. Gao, T. Jia, J. Liang. Single-shot compressed ultrafast photography: a review. Adv. Photon., 2, 014003(2020).

    [12] K. Nakagawa, A. Iwasaki, Y. Oishi, R. Horisaki, A. Tsukamoto, A. Nakamura, K. Hirosawa, H. Liao, T. Ushida, K. Goda. Sequentially timed all-optical mapping photography (STAMP). Nat. Photonics, 8, 695-700(2014).

    [13] T. Suzuki, R. Hida, Y. Yamaguchi, K. Nakagawa, T. Saiki, F. Kannari. Single-shot 25-frame burst imaging of ultrafast phase transition of Ge2Sb2Te5 with a sub-picosecond resolution. Appl. Phys. Express, 10, 092502(2017).

    [14] Y. Lu, T. T. Wong, F. Chen, L. Wang. Compressed ultrafast spectral-temporal photography. Phys. Rev. Lett., 122, 193904(2019).

    [15] A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, R. Raskar. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun., 3, 745(2012).

    [16] A. H. Zewail. Four-dimensional electron microscopy. Science, 328, 187-193(2010).

    [17] A. Barty, S. Boutet, M. J. Bogan, S. Hau-Riege, S. Marchesini, K. Sokolowski-Tinten, N. Stojanovic, H. Ehrke, A. Cavalleri, S. Düsterer. Ultrafast single-shot diffraction imaging of nanoscale dynamics. Nat. Photonics, 2, 415-419(2008).

    [18] J. Liang, C. Ma, L. Zhu, Y. Chen, L. Gao, L. V. Wang. Single-shot real-time video recording of a photonic Mach cone induced by a scattered light pulse. Sci. Adv., 3, e1601814(2017).

    [19] J. C. Jing, X. Wei, L. V. Wang. Spatio-temporal-spectral imaging of non-repeatable dissipative soliton dynamics. Nat. Commun., 11, 2059(2020).

    [20] T. Kim, J. Liang, L. Zhu, L. V. Wang. Picosecond-resolution phase-sensitive imaging of transparent objects in a single shot. Sci. Adv., 6, e6200(2020).

    [21] J. Liang, L. Gao, P. Hai, C. Li, L. V. Wang. Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography. Sci. Rep., 5, 15504(2015).

    [22] F. Cao, C. Yang, D. Qi, J. Yao, Y. He, X. Wang, W. Wen, J. Tian, T. Jia, Z. Sun. Single-shot spatiotemporal intensity measurement of picosecond laser pulses with compressed ultrafast photography. Opt. Laser Eng., 116, 89-93(2019).

    [23] L. Zhu, Y. Chen, J. Liang, Q. Xu, L. Gao, C. Ma, L. V. Wang. Space-and intensity-constrained reconstruction for compressed ultrafast photography. Optica, 3, 694-697(2016).

    [24] C. Yang, D. Qi, F. Cao, Y. He, X. Wang, W. Wen, J. Tian, T. Jia, Z. Sun, S. Zhang. Improving the image reconstruction quality of compressed ultrafast photography via an augmented Lagrangian algorithm. J. Opt., 21, 035703(2019).

    [25] Y. Lai, Y. Xue, C. Y. Côté, X. Liu, A. Laramée, N. Jaouen, F. Légaré, L. Tian, J. Liang. Single-shot ultraviolet compressed ultrafast photography. Laser Photon. Rev., 14, 2000122(2020).

    [26] C. Yang, D. Qi, X. Wang, F. Cao, Y. He, W. Wen, T. Jia, J. Tian, Z. Sun, L. Gao. Optimizing codes for compressed ultrafast photography by the genetic algorithm. Optica, 5, 147-151(2018).

    [27] C. Yang, D. Qi, J. Liang, X. Wang, F. Cao, Y. He, X. Ouyang, B. Zhu, W. Wen, T. Jia. Compressed ultrafast photography by multi-encoding imaging. Laser Phys. Lett., 15, 116202(2018).

    [28] C. Li. An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing(2010).

    [29] M. V. Afonso, J. M. Bioucas-Dias, M. A. Figueiredo. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process., 20, 681-695(2010).

    [30] Y. Yang, J. Sun, H. Li, Z. Xu. ADMM-CSNet: a deep learning approach for image compressive sensing. IEEE Trans. Pattern Anal., 42, 521-538(2018).

    [31] J. Zhang, B. Ghanem. ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1828-1837(2018).

    [32] J. Ma, X. Liu, Z. Shou, X. Yuan. Deep tensor ADMM-net for snapshot compressive imaging. Proceedings of the IEEE International Conference on Computer Vision, 10223-10232(2019).

    [33] K. Monakhova, J. Yurtsever, G. Kuo, N. Antipa, K. Yanny, L. Waller. Learned reconstructions for practical mask-based lensless imaging. Opt. Express, 27, 28075-28090(2019).

    [34] Q. Xie, Q. Zhao, D. Meng, Z. Xu. Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery. IEEE Trans. Pattern Anal. Mach. Intell., 40, 1888-1902(2017).

    [35] Y. Wang, J. Peng, Q. Zhao, Y. Leung, X. Zhao, D. Meng. Hyperspectral image restoration via total variation regularized low-rank tensor decomposition. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sensing, 11, 1227-1243(2017).

    [36] L. Wang, C. Sun, Y. Fu, M. H. Kim, H. Huang. Hyperspectral image reconstruction using a deep spatial-spectral prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8032-8041(2019).

    [37] Z. Wu, Y. Sun, A. Matlock, J. Liu, L. Tian, U. S. Kamilov. SIMBA: scalable inversion in optical tomography using deep denoising priors. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sensing, 14, 1163-1175(2020).

    [38] X. Miao, X. Yuan, Y. Pu, V. Athitsos. Lambda-net: reconstruct hyperspectral images from a snapshot measurement. IEEE/CVF International Conference on Computer Vision (ICCV), 4058-4068(2019).

    [39] J. M. Bioucas-Dias, M. A. Figueiredo. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process., 16, 2992-3004(2007).

    [40] E. J. Candes, T. Tao. Near-optimal signal recovery from random projections: universal encoding strategies?. IEEE Trans. Inform. Theory, 52, 5406-5425(2006).

    [41] E. J. Candes, J. K. Romberg, T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math., 59, 1207-1223(2006).

    [42] X. Liu, X. Wang. Fourth-order tensors with multidimensional discrete transforms(2017).

    [43] J. Barzilai, J. M. Borwein. Two-point step size gradient methods. IMA J. Numer. Anal., 8, 141-148(1988).

    [44] M. Raydan. Convergence properties of the Barzilai and Borwein gradient method(1991).

    [45] B. Lim, S. Son, H. Kim, S. Nah, K. Mu Lee. Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 136-144(2017).

    [46] L. Yue, X. Miao, P. Wang, B. Zhang, X. Zhen, X. Cao. Attentional alignment networks. 29th British Machine Vision Conference, 1-14(2018).

    [47] S. Min, X. Chen, Z. Zha, F. Wu, Y. Zhang. A two-stream mutual attention network for semi-supervised biomedical segmentation with noisy labels. Proceedings of the AAAI Conference on Artificial Intelligence, 4578-4585(2019).

    [48] Y. Li, Z. Xiao, X. Zhen, X. Cao. Attentional information fusion networks for cross-scene power line detection. IEEE Geosci. Remote Sens. Lett., 16, 1635-1639(2019).

    [49] Y. Huang, X. Cao, X. Zhen, J. Han. Attentive temporal pyramid network for dynamic scene classification. Proceedings of the AAAI Conference on Artificial Intelligence, 8497-8504(2019).

    [50] D. P. Kingma, J. Ba. Adam: a method for stochastic optimization(2014).

    [51] S. H. Chan, R. Khoshabeh, K. B. Gibson, P. E. Gill, T. Q. Nguyen. An augmented Lagrangian method for video restoration. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 941-944(2011).

    [52] https://drive.google.com/file/d/1PGav_1Yju2lfXlXQsxnvkMPu_j_EWm0B/view. https://drive.google.com/file/d/1PGav_1Yju2lfXlXQsxnvkMPu_j_EWm0B/view

    [53] https://www.pexels.com/video/blue-jellyfish-856882/. https://www.pexels.com/video/blue-jellyfish-856882/

    [54] https://www.pexels.com/video/a-person-using-a-smart-cellular-phone-3143531/. https://www.pexels.com/video/a-person-using-a-smart-cellular-phone-3143531/

    [55] H. Yu, S. Winkler. Image complexity and spatial information. Fifth International Workshop on Quality of Multimedia Experience (QoMEX), 12-17(2013).

    [56] C. Yang, D. Qi, F. Cao, Y. He, J. Yao, P. Ding, X. Ouyang, Y. Yu, T. Jia, S. Xu. Single-shot receive-only ultrafast electro-optical deflection imaging. Phys. Rev. Appl., 13, 024001(2020).

    [57] C. Yang, F. Cao, D. Qi, Y. He, P. Ding, J. Yao, T. Jia, Z. Sun, S. Zhang. Hyperspectrally compressed ultrafast photography. Phys. Rev. Lett., 124, 023902(2020).

    [58] M. Iliadis, L. Spinoulas, A. K. Katsaggelos. Deep fully-connected networks for video compressive sensing. Digit. Signal Process., 72, 9-18(2018).

    [59] K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, A. Ashok. ReconNet: non-iterative reconstruction of images from compressively sensed measurements. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 449-458(2016).

    [60] H. Yao, F. Dai, S. Zhang, Y. Zhang, Q. Tian, C. Xu. DR2-Net: deep residual reconstruction network for image compressive sensing. Neurocomputing, 359, 483-493(2019).

    [61] D. Gedalin, Y. Oiknine, A. Stern. DeepCubeNet: reconstruction of spectrally compressive sensed hyperspectral images with deep neural networks. Opt. Express, 27, 35811-35822(2019).

    [62] J. Nocedal, S. Wright. Numerical Optimization(2006).

    CLP Journals

    [1] Xianglei Liu, João Monteiro, Isabela Albuquerque, Yingming Lai, Cheng Jiang, Shian Zhang, Tiago H. Falk, Jinyang Liang, "Single-shot real-time compressed ultrahigh-speed imaging enabled by a snapshot-to-video autoencoder," Photonics Res. 9, 2464 (2021)

    [2] Li Gao, Yang Chai, Darko Zibar, Zongfu Yu, "Deep learning in photonics: introduction," Photonics Res. 9, DLP1 (2021)

    Tools

    Get Citation

    Copy Citation Text

    Chengshuai Yang, Yunhua Yao, Chengzhi Jin, Dalong Qi, Fengyan Cao, Yilin He, Jiali Yao, Pengpeng Ding, Liang Gao, Tianqing Jia, Jinyang Liang, Zhenrong Sun, Shian Zhang, "High-fidelity image reconstruction for compressed ultrafast photography via an augmented-Lagrangian and deep-learning hybrid algorithm," Photonics Res. 9, B30 (2021)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Special Issue: DEEP LEARNING IN PHOTONICS

    Received: Sep. 15, 2020

    Accepted: Dec. 2, 2020

    Published Online: Jan. 22, 2021

    The Author Email: Yunhua Yao (yhyao@lps.ecnu.edu.cn), Shian Zhang (sazhang@phy.ecnu.edu.cn)

    DOI:10.1364/PRJ.410018

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology