Researching | Deep plug-and-play self-supervised neural networks for spectral snapshot compressive imaging

Journal of Infrared and Millimeter Waves, Volume. 43, Issue 6, 847(2024)

Deep plug-and-play self-supervised neural networks for spectral snapshot compressive imaging

Xing-Yu ZHANG^1,3, Shou-Zheng ZHU^1,3, Tian-Shu ZHOU^1,3, Hong-Xing QI^1,3, Jian-Yu WANG^1,2,3, Chun-Lai LI^1,2,3、**, and Shi-Jie LIU^1,3、*

Author Affiliations

¹School of Physics and Optoeletronic Engineering，Hangzhou Institute for Advanced Study，University of Chinese Academy of Sciences，Hangzhou 310024，China

²Key Laboratory of Space Active Opto-Electronics Technology，Shanghai Institute of Technical Physics，Chinese Academy of Sciences，Shanghai 200083，China

³University of Chinese Academy of Sciences，Beijing 100049，China

show less

Full Text Get PDF

Figures&Tables (18)

References (31)

Paper Information

Figures & Tables(18)

Fig. 1. CASSI forward model

Download full size

View in Article

Fig. 2. PnP image reconstruction framework

Download full size

View in Article

Fig. 3. Image downsampling method

Download full size

View in Article

Fig. 4. Self-HSIDeCNN network architecture

Download full size

View in Article

Fig. 5. SENet architecture

Download full size

View in Article

Fig. 6. Self-supervised training process

Download full size

View in Article

Fig. 7. RGB image of test scene

Download full size

View in Article

Fig. 8. Comparison of the visual effect of the reconstruction results of different algorithms：（a）RGB image；（b）2D measurements；（c）ground truth；（d）U-net；（e）TSA-net；（f）PnP-HIS；（g）PnP-Self-HSIDeCNN

Download full size

View in Article

Fig. 9. Comparison of visual reconstruction effects in the final band：（a）ground truth；（b）U-net；（c）TSA-net；（d）PnP-HIS；（e）PnP-Self-HSIDeCNN

Download full size

View in Article

Fig. 10. Comparison of spectral curve reconstruction results：（a）RGB image；（b）spectral curve comparison

Download full size

View in Article

Table 1. Comparison of denoising results

View in Article

Table 1. Comparison of denoising results

Noise type	Supervised Model	Self-supervised Model
$σ = 25$	38.12 dB，0.951	39.33 dB，0.978
$σ ϵ [5, 50]$	37.63 dB，0.924	39.16 dB，0.969
$λ = 30$	38.28 dB，0.964	40.25 dB，0.983
$λ ϵ [5, 50]$	38.25 dB，0.966	39.86 dB，0.976

Table 2. The influence of hyperparameter $γ$ on denoising results
View table
View in Article
Table 2. The influence of hyperparameter $γ$ on denoising results
$γ$
PSNR / dB
（ $σ = 25$ ）
SSIM
（ $σ = 25$ ）
PSNR / dB
（ $λ = 30$ ）
SSIM
（ $λ = 30$ ）
0 39.27 0.970 40.26 0.983
1 39.23 0.962 40.23 0.981
2 39.26 0.965 40.34 0.986
5 39.33 0.978 40.25 0.983
20 39.30 0.977 39.89 0.979

Table 3. Comparison of reconstruction results

View in Article

Table 3. Comparison of reconstruction results

	U-net	TSA-net	PnP-HSI	Ours （ $σ = 25$ ）	Ours （ $σ ϵ [5, 50]$ ）	Ours （ $λ = 30$ ）	Ours （ $λ ϵ [5, 50]$ ）
Scene1	26.29dB，0.843	26.47dB，0.855	29.56dB，0.875	31.12dB，0.892	30.44dB，0.843	28.34dB，0.788	28.51dB，0.798
Scene2	37.20dB，0.941	36.98dB，0.935	37.59dB，0.959	39.62dB，0.959	37.01dB，0.921	37.55dB，0.963	38.39dB，0.966
Scene3	33.87dB，0.918	35.07dB，0.917	36.27dB，0.924	35.89dB，0.916	34.31dB，0.888	34.14dB，0.908	34.25dB，0.909
Scene4	32.99dB，0.910	33.16dB，0.929	34.88dB，0.933	35.22dB，0.928	34.13dB，0.870	32.96dB，0.910	33.02dB，0.903
Scene5	20.11dB，0.788	21.14dB，0.794	21.55dB，0.797	23.89dB，0.838	23.94dB，0.808	23.06dB，0.810	22.70dB，0.802
Mean	30.09dB，0.880	30.56dB，0.886	31.97dB，0.898	33.15dB，0.907	31.97dB，0.866	31.21dB，0.876	31.37dB，0.876

Table 4. Comparison of generalisability

View in Article

Table 4. Comparison of generalisability

	U-net	TSA-net	PnP-HSI	Ours （ $σ = 25$ ）
Scene1	25.89dB，0.824	25.88dB，0.838	29.52dB，0.873	31.13dB，0.890
Scene2	36.41dB，0.931	35.49dB，0.927	37.54dB，0.957	39.59dB，0.957
Scene3	31.78dB，0.887	33.80dB，0.904	36.23dB，0.919	35.84dB，0.912
Scene4	31.03dB，0.901	32.03dB，0.917	34.80dB，0.934	35.21dB，0.929
Scene5	20.14dB，0.773	20.50dB，0.787	21.56dB，0.798	23.84dB，0.834
Mean	29.05dB，0.863	29.54dB，0.874	31.93dB，0.899	33.12dB，0.904
Difference	-1.04dB，-0.017	-1.02dB，-0.012	-0.04dB，-0.001	-0.03dB，-0.003
Percentage	-3.46%，-1.93%	-3.34%，-1.35%	-0.13%，-1.11%	-0.09%，-0.33%

Table 5. Influence of hyperparameter $γ$ on reconstruction results

View in Article

Table 5. Influence of hyperparameter $γ$ on reconstruction results

	$γ = 0$		$γ = 1$		$γ = 2$		$γ = 5$		$γ = 20$
	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
Scene1	30.92	0.862	30.42	0.862	30.92	0.878	29.70	0.785	29.89	0.855
Scene2	37.19	0.921	38.54	0.940	40.19	0.951	35.50	0.921	37.67	0.944
Scene3	33.93	0.888	36.28	0.908	36.36	0.913	32.74	0.888	34.28	0.892
Scene4	33.98	0.883	35.07	0.899	35.47	0.894	33.19	0.861	35.38	0.916
Scene5	22.97	0.769	23.86	0.833	23.80	0.843	23.29	0.791	23.18	0.803
Mean	31.79	0.865	32.83	0.888	33.35	0.896	30.88	0.849	32.08	0.882

Table 6. Influence of noise estimation level $σ$ on reconstruction results

View in Article

Table 6. Influence of noise estimation level $σ$ on reconstruction results

	$σ = 100$		$σ = 50$		$σ = 30$		$σ = 5$		$σ ϵ [0, 50]$
	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
Scene1	27.57	0.750	30.21	0.852	31.00	0.888	31.28	0.897	32.01	0.899
Scene2	33.56	0.921	38.16	0.941	39.38	0.955	40.18	0.973	40.32	0.977
Scene3	32.00	0.888	34.69	0.888	35.77	0.911	36.20	0.928	36.83	0.932
Scene4	31.87	0.861	34.59	0.912	35.13	0.927	35.50	0.929	36.14	0.941
Scene5	21.58	0.749	22.97	0.803	23.75	0.832	24.29	0.850	25.88	0.859
Mean	29.32	0.834	32.12	0.879	33.01	0.903	33.49	0.915	34.23	0.921

Table 7. Influence of noise estimation level $σ$ on reconstruction time

View in Article

Table 7. Influence of noise estimation level $σ$ on reconstruction time

	$σ = 100$	$σ = 50$	$σ = 30$	$σ = 5$	$σ ϵ [0, 50]$
Scene1	53.10s	96.57s	99.09s	105.39s	117.74s
Scene2	51.21s	91.74s	95.67s	101.17s	110.92s
Scene3	48.69s	84.51s	89.87s	94.28s	105.03s
Scene4	51.84s	91.35s	96.27s	103.77s	116.19s
Scene5	55.62s	104.61s	112.84s	119.07s	145.56s
Mean	52.09s	93.76s	98.75s	104.74s	119.09s

Table 8. Influence of the number of warm start iterations

View in Article

Table 8. Influence of the number of warm start iterations

Number of warm start iterations	Warm start time / s	Self-supervised reconstruction time / s	Total reconstruction time/ s	PSNR / dB	SSIM
0	0	39.06	39.06	33.79	0.828
10	5.24	40.95	46.19	34.33	0.867
20	10.04	39.69	49.73	34.43	0.867
30	15.76	40.95	56.71	34.44	0.871
40	20.68	40.32	61.00	34.42	0.869
50	26.41	40.32	66.73	34.25	0.869
60	31.23	39.69	70.92	34.21	0.867
70	36.44	39.69	76.13	34.17	0.864
80	41.67	40.32	81.99	34.15	0.869
90	46.88	40.95	87.83	34.12	0.869
100	52.27	39.69	91.96	34.12	0.867

Tools

Get Citation

Copy Citation Text

Xing-Yu ZHANG, Shou-Zheng ZHU, Tian-Shu ZHOU, Hong-Xing QI, Jian-Yu WANG, Chun-Lai LI, Shi-Jie LIU. Deep plug-and-play self-supervised neural networks for spectral snapshot compressive imaging[J]. Journal of Infrared and Millimeter Waves, 2024, 43(6): 847

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Interdisciplinary Research on Infrared Science

Received: Feb. 29, 2024

Accepted: --

Published Online: Dec. 13, 2024

The Author Email: LI Chun-Lai (lichunlai@mail.sitp.ac.cn), LIU Shi-Jie (liushijie@ucas.ac.cn)

DOI:10.11972/j.issn.1001-9014.2024.06.016

Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology