Researching | Deep plug-and-play priors for spectral snapshot compressive imaging

Photonics Research, Volume. 9, Issue 2, B18(2021)

Deep plug-and-play priors for spectral snapshot compressive imaging

Siming Zheng^1,2、†, Yang Liu^3、†, Ziyi Meng⁴, Mu Qiao⁵, Zhishen Tong^6,7, Xiaoyu Yang^1,2, Shensheng Han^6,7, and Xin Yuan^8、*

Author Affiliations

¹Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China

²University of Chinese Academy of Sciences, Beijing 100049, China

³Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA

⁴Beijing University of Posts and Telecommunications, Beijing 100876, China

⁵New Jersey Institute of Technology, Newark, New Jersey 07102, USA

⁶Key Laboratory for Quantum Optics and Center for Cold Atom Physics of CAS, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China

⁷Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China

⁸Nokia Bell Labs, Murray Hill, New Jersey 07974, USA

show less

Full Text Get PDF

Figures&Tables (16)

References (72)

Paper Information

Figures & Tables(16)

Fig. 1. Generalized image formation (left) and the discrete matrix-form model (right) of spectral SCI. Here color denotes the corresponding spectral band.

Download full size

View in Article

Fig. 2. Comparison of image-plane coding (upper) and aperture-plane coding (lower) spectral SCI systems in terms of sensing matrix. Here each color block denotes the corresponding transport matrix at that spectral band.

Download full size

View in Article

Fig. 3. Image formation process of a typical spectral SCI system, i.e., SD-CASSI and the reconstruction process using the proposed deep PnP prior algorithm.

Download full size

View in Article

Fig. 4. Network structure of the deep spectral denoising prior.

Download full size

View in Article

Fig. 5. Test spectral data from (a) ICVL [69] and (b) KAIST [35] data sets used in simulation. The reference RGB images with pixel resolution 256×256 are shown here. We crop similar regions of the whole image for spatial sizes of 512×512 and 1024×1024.

Download full size

View in Article

Fig. 6. Simulation results of color-checker with size of 256×256 from KAIST data set compared with the ground truth. PSNR and SSIM results are also shown for each algorithm.

Download full size

View in Article

Fig. 7. Simulation results of exemplar scenes (top, ICVL; bottom, KAIST) with size of 256×256 compared with the ground truth. Spectral curves of selected regions are also plotted to compare with the ground truth.

Download full size

View in Article

Fig. 8. Simulation results of four selected scenes shown in sRGB and spectral curves with spatial size of 512×512 (shown in full size in the far left column). The spectra of the pinned (yellow) region of the close-up are shown on the right.

Download full size

View in Article

Fig. 9. Simulation results of four selected scenes shown in sRGB and spectral curves with spatial size of 1024×1024 (shown in full size in the far left column). The spectra of the pinned (yellow) region of the close-up are shown on the right.

Download full size

View in Article

Fig. 10. Real data, object SD-CASSI data (256×210×33).

Download full size

View in Article

Fig. 11. Real data, bird SD-CASSI data (1021×731×33).

Download full size

View in Article

Fig. 12. Real data, Lego SD-CASSI data (660×550×28).

Download full size

View in Article

Fig. 13. Real data, plant SD-CASSI data (660×550×28).

Download full size

View in Article

Fig. 14. Real data, snapshot multispectral endomicroscopy data (660×660×24).

Download full size

View in Article

Fig. 15. Real data, GISC spectral camera data (330×330×16).

Download full size

View in Article

Table 1. Average PSNR (in dB), SSIM, and Running Time (in Seconds) of 16 Simulation Scenes (8 from ICVL and 8 from KAIST) at Different Spatial Sizes Using Various Algorithms^a

View in Article

Table 1. Average PSNR (in dB), SSIM, and Running Time (in Seconds) of 16 Simulation Scenes (8 from ICVL and 8 from KAIST) at Different Spatial Sizes Using Various Algorithms^a

Spatial Size	Data Set	TwIST			GAP-TV			AE			U-net			PnP
Spatial Size	Data Set	PSNR (dB)	SSIM	Running Time (s)	PSNR (dB)	SSIM	Running Time (s)	PSNR (dB)	SSIM	Running Time (s)	PSNR (dB)	SSIM	Running Time (s)	PSNR (dB)	SSIM	Running Time (s)
$256 \times 256$	ICVL	30.58	0.8731	156.3	32.57	0.8794	130.2	29.41	0.8711	144.2	31.13	0.8897	0.8	35.03	0.9274	132.7
$256 \times 256$	KAIST	27.32	0.8495	156.3	29.66	0.8584	130.2	26.79	0.8498	144.2	29.44	0.8941	0.8	33.21	0.9273	132.7
$512 \times 512$	ICVL	31.82	0.8955	1380.2	33.58	0.8965	399.1	31.22	0.8969	493.6	NA	NA	NA	35.68	0.9319	401.6
$512 \times 512$	KAIST	29.09	0.8944	1380.2	31.38	0.8993	399.1	29.28	0.8974	493.6	NA	NA	NA	34.29	0.9378	401.6
$1024 \times 1024$	ICVL	32.68	0.9159	3657.6	34.22	0.9157	1460.7	32.03	0.9158	2053.5	NA	NA	NA	36.21	0.9434	1453.6
$1024 \times 1024$	KAIST	31.64	0.9099	3657.6	33.66	0.9134	1460.7	31.05	0.9071	2053.5	NA	NA	NA	36.41	0.9433	1453.6

Tools

Get Citation

Copy Citation Text

Siming Zheng, Yang Liu, Ziyi Meng, Mu Qiao, Zhishen Tong, Xiaoyu Yang, Shensheng Han, Xin Yuan, "Deep plug-and-play priors for spectral snapshot compressive imaging," Photonics Res. 9, B18 (2021)

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Special Issue: DEEP LEARNING IN PHOTONICS

Received: Oct. 5, 2020

Accepted: Nov. 23, 2020

Published Online: Jan. 22, 2021

The Author Email: Xin Yuan (xyuan@bell-labs.com)

DOI:10.1364/PRJ.411745

Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology