Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Photonics Research, Volume. 9, Issue 2, B18(2021)

Deep plug-and-play priors for spectral snapshot compressive imaging

Siming Zheng1,2、†, Yang Liu3、†, Ziyi Meng4, Mu Qiao5, Zhishen Tong6,7, Xiaoyu Yang1,2, Shensheng Han6,7, and Xin Yuan8、*
Author Affiliations
  • 1Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China
  • 2University of Chinese Academy of Sciences, Beijing 100049, China
  • 3Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
  • 4Beijing University of Posts and Telecommunications, Beijing 100876, China
  • 5New Jersey Institute of Technology, Newark, New Jersey 07102, USA
  • 6Key Laboratory for Quantum Optics and Center for Cold Atom Physics of CAS, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
  • 7Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
  • 8Nokia Bell Labs, Murray Hill, New Jersey 07974, USA
    • show less
    Full TextGet PDF
    Figures&Tables (16)
    Equations (19)
    References (72)
    Cited By (68)
    Tools
    Paper Information
    Topics
    References(72)

    [1] B. E. Bayer. Color imaging array. U.S. patent(1976).

    [2] B. Redding, S. F. Liew, R. Sarma, H. Cao. Compact spectrometer based on a disordered photonic chip. Nat. Photonics, 7, 746-751(2013).

    [3] Z. Wang, Z. Yu. Spectral analysis based on compressive sensing in nanophotonic structures. Opt. Express, 22, 25608-25614(2014).

    [4] J. Bao, M. G. Bawendi. A colloidal quantum dot spectrometer. Nature, 523, 67-70(2015).

    [5] Z. Yang, T. Albrow-Owen, H. Cui, J. Alexander-Webber, F. Gu, X. Wang, T.-C. Wu, M. Zhuge, C. Williams, P. Wang, V. A. Zayats, W. Cai, L. Dai, S. Hofmann, M. Overend, L. Tong, Q. Yang, Z. Sun, T. Hasan. Single-nanowire spectrometers. Science, 365, 1017-1020(2019).

    [6] Z. Wang, S. Yi, A. Chen, M. Zhou, T. S. Luk, A. James, J. Nogan, W. Ross, G. Joe, A. Shahsafi, K. X. Wang, M. A. Kats, Z. Yu. Single-shot on-chip spectral sensors based on photonic crystal slabs. Nat. Commun., 10, 1020(2019).

    [7] A. McClung, S. Samudrala, M. Torfeh, M. Mansouree, A. Arbabi. Snapshot spectral imaging with parallel metasystems. Sci. Adv., 6, eabc7646(2020).

    [8] Y. Kwak, S. M. Park, Z. Ku, A. Urbas, Y. L. Kim. A pearl spectrometer. Nano Lett.(2020).

    [9] D. Donoho. Compressed sensing. IEEE Trans. Inf. Theory, 52, 1289-1306(2006).

    [10] E. J. Candès, J. Romberg, T. Tao. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory, 52, 489-509(2006).

    [11] E. J. Candès, M. B. Wakin. An introduction to compressive sampling. IEEE Signal Process. Mag., 25, 21-30(2008).

    [12] M. E. Gehm, R. John, D. J. Brady, R. M. Willett, T. J. Schulz. Single-shot compressive spectral imaging with a dual-disperser architecture. Opt. Express, 15, 14013-14027(2007).

    [13] G. R. Arce, D. J. Brady, L. Carin, H. Arguello, D. S. Kittle. Compressive coded aperture spectral imaging: an introduction. IEEE Signal Process. Mag., 31, 105-115(2014).

    [14] X. Cao, T. Yue, X. Lin, S. Lin, X. Yuan, Q. Dai, L. Carin, D. J. Brady. Computational snapshot multispectral cameras: toward dynamic capture of the spectral world. IEEE Signal Process. Mag., 33, 95-108(2016).

    [15] J. Bioucas-Dias, M. Figueiredo. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process., 16, 2992-3004(2007).

    [16] Y. Liu, X. Yuan, J. Suo, D. J. Brady, Q. Dai. Rank minimization for snapshot compressive imaging. IEEE Trans. Pattern Anal. Mach. Intell., 41, 2990-3006(2019).

    [17] L. Wang, Z. Xiong, G. Shi, F. Wu, W. Zeng. Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE Trans. Pattern Anal. Mach. Intell., 39, 2104-2111(2017).

    [18] X. Yuan, Y. Liu, J. Suo, Q. Dai. Plug-and-play algorithms for large-scale snapshot compressive imaging. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1447-1457(2020).

    [19] A. Wagadarikar, R. John, R. Willett, D. J. Brady. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt., 47, B44-B51(2008).

    [20] X. Lin, Y. Liu, J. Wu, Q. Dai. Spatial-spectral encoded compressive hyperspectral imaging. ACM Trans. Graph., 33, 233(2014).

    [21] X. Cao, H. Du, X. Tong, Q. Dai, S. Lin. A prism-mask system for multispectral video acquisition. IEEE Trans. Pattern Anal. Mach. Intell., 33, 2423-2435(2011).

    [22] H. Arguello, H. Rueda, Y. Wu, D. W. Prather, G. R. Arce. Higher-order computational model for coded aperture spectral imaging. Appl. Opt., 52, D12-D21(2013).

    [23] L. Wang, Z. Xiong, D. Gao, G. Shi, F. Wu. Dual-camera design for coded aperture snapshot spectral imaging. Appl. Opt., 54, 848-858(2015).

    [24] C. V. Correa, H. Arguello, G. R. Arce. Snapshot colored compressive spectral imager. J. Opt. Soc. Am. A, 32, 1754-1763(2015).

    [25] X. Yuan, T.-H. Tsai, R. Zhu, P. Llull, D. J. Brady, L. Carin. Compressive hyperspectral imaging with side information. IEEE J. Sel. Top. Signal Process., 9, 964-976(2015).

    [26] Z. Liu, S. Tan, J. Wu, E. Li, X. Shen, S. Han. Spectral camera based on ghost imaging via sparsity constraints. Sci. Rep., 6, 25718(2016).

    [27] X. Li, J. A. Greenberg, M. E. Gehm. Single-shot multispectral imaging through a thin scatterer. Optica, 6, 864-871(2019).

    [28] D. S. Jeon, S.-H. Baek, S. Yi, Q. Fu, X. Dun, W. Heidrich, M. H. Kim. Compact snapshot hyperspectral imaging with diffracted rotation. ACM Trans. Graph., 38, 117(2019).

    [29] S.-H. Baek, I. Kim, D. Gutierrez, M. H. Kim. Compact single-shot hyperspectral imaging using a prism. ACM Trans. Graph., 36(2017).

    [30] M. A. T. Figueiredo, R. D. Nowak, S. J. Wright. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process., 1, 586-597(2007).

    [31] M. Aharon, M. Elad, A. Bruckstein. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process., 54, 4311-4322(2006).

    [32] J. Yang, X. Liao, X. Yuan, P. Llull, D. J. Brady, G. Sapiro, L. Carin. Compressive sensing by learning a Gaussian mixture model from measurements. IEEE Trans. Image Process., 24, 106-119(2015).

    [33] G. Barbastathis, A. Ozcan, G. Situ. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [34] X. Yuan, Y. Pu. Parallel lensless compressive imaging via deep convolutional neural networks. Opt. Express, 26, 1962-1977(2018).

    [35] I. Choi, D. S. Jeon, G. Nam, D. Gutierrez, M. H. Kim. High-quality hyperspectral reconstruction using a spectral prior. ACM Trans. Graph., 36, 218(2017).

    [36] Z. Meng, M. Qiao, J. Ma, Z. Yu, K. Xu, X. Yuan. Snapshot multispectral endomicroscopy. Opt. Lett., 45, 3897-3900(2020).

    [37] X. Miao, X. Yuan, Y. Pu, V. Athitsos. λ-net: reconstruct hyperspectral images from a snapshot measurement. IEEE/CVF Conference on Computer Vision (ICCV), 4058-4068(2019).

    [38] L. Wang, C. Sun, Y. Fu, M. H. Kim, H. Huang. Hyperspectral image reconstruction using a deep spatial-spectral prior. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8024-8033(2019).

    [39] Z. Meng, J. Ma, X. Yuan. End-to-end low cost compressive spectral imaging with spatial-spectral self-attention. European Conference on Computer Vision (ECCV), 187-204(2020).

    [40] S. V. Venkatakrishnan, C. A. Bouman, B. Wohlberg. Plug-and-play priors for model based reconstruction. IEEE Global Conference on Signal and Information Processing, 945-948(2013).

    [41] S. Sreehari, S. V. Venkatakrishnan, B. Wohlberg, G. T. Buzzard, L. F. Drummy, J. P. Simmons, C. A. Bouman. Plug-and-play priors for bright field electron tomography and sparse interpolation. IEEE Trans. Comput. Imaging, 2, 408-423(2016).

    [42] S. H. Chan, X. Wang, O. A. Elgendy. Plug-and-play ADMM for image restoration: fixed-point convergence and applications. IEEE Trans. Comput. Imaging, 3, 84-98(2017).

    [43] E. K. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, W. Yin. Plug-and-play methods provably converge with properly trained denoisers(2019).

    [44] L. Zhang, W. Zuo. Image restoration: from sparse and low-rank priors to deep priors. IEEE Signal Process. Mag., 34, 172-179(2017).

    [45] K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process., 26, 3142-3155(2017).

    [46] K. Zhang, W. Zuo, L. Zhang. FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process., 27, 4608-4622(2018).

    [47] A. A. Wagadarikar, N. P. Pitsianis, X. Sun, D. J. Brady. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express, 17, 6368-6388(2009).

    [48] D. Kittle, K. Choi, A. Wagadarikar, D. J. Brady. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt., 49, 6824-6833(2010).

    [49] R. Zhu, T. Tsai, D. J. Brady. Coded aperture snapshot spectral imager based on liquid crystal spatial light modulator. Frontiers in Optics, FW1D-4(2013).

    [50] T.-H. Tsai, X. Yuan, D. J. Brady. Spatial light modulator based color polarization imaging. Opt. Express, 23, 11912-11926(2015).

    [51] X. Yuan. Generalized alternating projection based total variation minimization for compressive sensing. IEEE International Conference on Image Processing (ICIP), 2539-2543(2016).

    [52] S. K. Sahoo, D. Tang, C. Dang. Single-shot multispectral imaging with a monochromatic camera. Optica, 4, 1209-1213(2017).

    [53] K. Monakhova, K. Yanny, N. Aggarwal, L. Waller. Spectral diffusercam: lensless snapshot hyperspectral imaging with a spectral filter array. Optica, 7, 1298-1307(2020).

    [54] J. Wu, E. Li, X. Shen, S. Yao, Z. Tong, C. Hu, Z. Liu, S. Liu, S. Tan, S. Han. Experimental results of the balloon-borne spectral camera based on ghost imaging via sparsity constraints. IEEE Access, 6, 68740-68748(2018).

    [55] E. Candès, J. Romberg, T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math., 59, 1207-1223(2006).

    [56] E. J. Candès, T. Tao. Near-optimal signal recovery from random projections: universal encoding strategies?. IEEE Trans. Inf. Theory, 52, 5406-5425(2006).

    [57] E. J. Candès. The restricted isometry property and its implications for compressed sensing. C.R. Math., 346, 589-592(2008).

    [58] S. Jalali, X. Yuan. Snapshot compressed sensing: performance bounds and algorithms. IEEE Trans. Inf. Theory, 65, 8005-8024(2019).

    [59] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3, 1-122(2011).

    [60] N. Parikh, S. Boyd. Proximal algorithms. Found. Trends Optim., 1, 127-239(2014).

    [61] W. W. Hager. Updating the inverse of a matrix. SIAM Rev., 31, 221-239(1989).

    [62] I. Daubechies, M. Defrise, C. De Mol. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math., 57, 1413-1457(2004).

    [63] M. Gharbi, G. Chaurasia, S. Paris, F. Durand. Deep joint demosaicking and denoising. ACM Trans. Graph., 35(2016).

    [64] M. Tassano, J. Delon, T. Veit. FastDVDnet: towards real-time deep video denoising without flow estimation. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1354-1363(2020).

    [65] A. Maffei, J. M. Haut, M. E. Paoletti, J. Plaza, L. Bruzzone, A. Plaza. A single model CNN for hyperspectral image denoising. IEEE Trans. Geosci. Remote Sens., 58, 2516-2529(2020).

    [66] F. Yasuma, T. Mitsunaga, D. Iso, S. K. Nayar. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Image Process., 19, 2241-2253(2010).

    [67] H. Wallach, A. Paszke, S. Gross, H. Larochelle, F. Massa, A. Beygelzimer, A. Lerer, F. d’ Alché-Buc, J. Bradbury, E. Fox, R. Garnett, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala. PyTorch: an imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32, 8024-8035(2019).

    [68] D. P. Kingma, J. Ba. Adam: a method for stochastic optimization(2014).

    [69] B. Arad, O. Ben-Shahar. Sparse recovery of hyperspectral signal from natural RGB images. European Conference on Computer Vision, 19-34(2016).

    [70] O. Ronneberger, P. Fischer, T. Brox. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 234-241(2015).

    [71] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13, 600-612(2004).

    [72] T. Smith, J. Guild. The C.I.E. colorimetric standards and their use. Trans. Opt. Soc., 33, 73-134(1931).

    CLP Journals

    [1] Lishun Wang, Zongliang Wu, Yong Zhong, Xin Yuan. Snapshot spectral compressive imaging reconstruction using convolution and contextual Transformer[J]. Photonics Research, 2022, 10(8): 1848

    [2] Li Gao, Yang Chai, Darko Zibar, Zongfu Yu. Deep learning in photonics: introduction[J]. Photonics Research, 2021, 9(8): DLP1

    Tools

    Get Citation

    Copy Citation Text

    Siming Zheng, Yang Liu, Ziyi Meng, Mu Qiao, Zhishen Tong, Xiaoyu Yang, Shensheng Han, Xin Yuan. Deep plug-and-play priors for spectral snapshot compressive imaging[J]. Photonics Research, 2021, 9(2): B18

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Special Issue: DEEP LEARNING IN PHOTONICS

    Received: Oct. 5, 2020

    Accepted: Nov. 23, 2020

    Published Online: Jan. 22, 2021

    The Author Email: Xin Yuan (xyuan@bell-labs.com)

    DOI:10.1364/PRJ.411745

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology