Chinese Optics, Volume. 17, Issue 6, 1316(2024)
Image reconstruction of snapshot multispectral camera based on an attention residual network
Figures & Tables(14)
Fig. 1. Original image acquisition and reconstruction process for a single-sensor multispectral camera equipped with MSFA
Fig. 5. Visual comparison of the de-mosaicing effect of test images under the D65 light source in sRGB color space
Fig. 6. Visual comparison of de-mosaicing error maps of test images in different scenarios at 679 nm
Fig. 7. Visual comparison of de-mosaicing error maps in different bands of test image CD
Table 1. PSNR, SSIM, and SAM values of the test image under the A light source
View tableView in ArticleTable 1. PSNR, SSIM, and SAM values of the test image under the A light source
PSNR ↑ SSIM ↑ SAM ↓ WB PPID MGCC Ours WB PPID MGCC Ours WB PPID MGCC Ours balloons 39.99 43.23 45.52 45.93 0.9977 0.9988 0.9992 0.9993 4.892 3.693 3.409 3.329 beads 28.35 31.03 32.75 33.57 0.9610 0.9768 0.9776 0.9858 10.087 8.002 7.571 6.182 Egyptian 35.22 40.77 40.96 43.65 0.9908 0.9947 0.9953 0.9972 11.358 10.648 8.653 7.570 feathers 32.38 36.32 37.03 38.48 0.9908 0.9962 0.9964 0.9976 8.351 6.782 6.036 5.960 paints 32.39 36.29 35.79 38.63 0.9939 0.9973 0.9973 0.9984 6.557 5.178 4.891 4.500 pompoms 36.37 38.28 40.54 41.02 0.9953 0.9968 0.9978 0.9987 4.395 3.524 3.003 2.916 CD 36.54 38.54 39.01 39.75 0.9939 0.9953 0.9956 0.9969 4.211 4.043 3.956 4.145 Character 29.27 34.91 38.91 39.51 0.9942 0.9981 0.9989 0.9992 6.103 3.811 3.119 3.049 ChartRes 30.24 31.26 31.45 32.63 0.9931 0.9976 0.9984 0.9996 3.962 2.761 2.529 1.300 Average 33.41 36.73 37.99 39.24 0.9900 0.9946 0.9951 0.9969 6.657 5.383 4.796 4.327
Table 2. PSNR, SSIM, and SAM values of the test image under the D65 light source
View tableView in ArticleTable 2. PSNR, SSIM, and SAM values of the test image under the D65 light source
PSNR ↑ SSIM ↑ SAM ↓ WB PPID MGCC Ours WB PPID MGCC Ours WB PPID MGCC Ours balloons 40.73 44.16 46.94 48.36 0.9981 0.9991 0.9995 0.9997 3.629 3.114 2.700 2.539 beads 25.75 31.28 33.21 34.06 0.9662 0.9826 0.9808 0.9898 10.316 8.324 6.957 6.243 Egyptian 39.29 42.50 42.58 44.86 0.9929 0.9875 0.9965 0.9979 10.642 8.015 7.038 6.919 feathers 32.67 36.72 37.26 39.12 0.9916 0.9931 0.9966 0.9980 7.377 5.891 5.412 4.843 paints 31.18 36.07 36.41 38.90 0.9922 0.9974 0.9887 0.9986 6.408 4.788 3.974 3.905 pompoms 36.88 38.90 41.19 42.65 0.9962 0.9978 0.9985 0.9989 4.032 3.232 2.693 2.753 CD 34.54 38.21 40.61 41.53 0.9949 0.9952 0.9961 0.9967 3.209 2.942 2.772 2.726 Character 28.88 34.90 39.63 39.87 0.9936 0.9984 0.9994 0.9996 6.867 4.100 3.108 2.907 ChartRes 29.34 29.65 32.54 33.21 0.9951 0.9958 0.9969 0.9971 2.840 2.290 1.561 1.324 Average 33.25 36.93 38.93 40.28 0.9912 0.9941 0.9947 0.9974 6.147 4.744 4.023 3.795
Table 3. PSNR, SSIM, and SAM values of the test image under the F12 light source
View tableView in ArticleTable 3. PSNR, SSIM, and SAM values of the test image under the F12 light source
PSNR ↑ SSIM ↑ SAM ↓ WB PPID MGCC Ours WB PPID MGCC Ours WB PPID MGCC Ours balloons 36.68 37.83 38.60 38.91 0.9937 0.9952 0.9961 0.9989 5.540 5.640 5.537 5.955 beads 25.84 27.65 27.67 29.92 0.9390 0.9566 0.9673 0.9720 14.958 13.669 13.532 11.424 Egyptian 36.23 37.46 38.23 39.47 0.9825 0.9857 0.9862 0.9913 6.150 5.740 4.763 3.795 feathers 30.61 32.76 33.51 34.70 0.9829 0.9887 0.9889 0.9928 11.526 11.194 11.251 10.897 paints 28.02 30.89 31.98 33.93 0.9761 0.9871 0.9912 0.9939 9.801 9.440 9.415 9.113 pompoms 32.17 32.81 33.92 33.97 0.9874 0.9889 0.9907 0.9909 6.776 6.355 6.150 5.936 CD 35.58 35.99 36.26 37.06 0.9869 0.9872 0.9898 0.9988 7.934 6.691 5.957 6.622 Character 26.11 28.86 32.33 32.64 0.9824 0.9908 0.9954 0.9962 7.794 6.881 6.835 6.767 ChartRes 25.81 26.32 27.61 29.33 0.9768 0.9814 0.9826 0.9916 3.845 3.157 2.837 1.859 Average 30.78 32.28 33.34 34.44 0.9786 0.9846 0.9875 0.9918 8.258 7.640 7.364 6.929
Table 4. Quantitative comparison of PSNR, SSIM, and SAM of test images processed by different de-mosaicing methods under three light sources
View tableView in ArticleTable 4. Quantitative comparison of PSNR, SSIM, and SAM of test images processed by different de-mosaicing methods under three light sources
WB PPID MGCC Ours PSNR 29.37 34.56 35.69 37.81 SSIM 0.8926 0.9705 0.9902 0.9936 SAM 8.541 6.473 5.647 5.312
Table 5. Comparison of running times of different de-mosaicing methods（Unit：ms）
View tableView in ArticleTable 5. Comparison of running times of different de-mosaicing methods（Unit：ms）
WB PPID MGCC Ours CPU 254.35 2134.5 - - GPU - - 2.65 2.12
Table 6. The ablation of different network architectures
View tableView in ArticleTable 6. The ablation of different network architectures
多分支架构 残差密集块 PSNR SSIM SAM × × 36.94 0.9869 7.263 × √ 37.21 0.9907 6.726 √ √ 38.59 0.9938 6.218
Table 7. The ablation of different attention mechanism schemes
View tableView in ArticleTable 7. The ablation of different attention mechanism schemes
注意力机制 PSNR SSIM SAM RN 36.24 0.9804 8.352 CA 36.79 0.9873 6.316 MAM 37.28 0.9927 6.047 SCAM 38.46 0.9951 5.247
Tools
Get Citation
Copy Citation Text
Gang-qi YAN, Zong-lin LIANG, Yan-song SONG, Ke-yan DONG, Bo ZHANG, Tian-ci LIU, Lei ZHANG, Yan-bo WANG. Image reconstruction of snapshot multispectral camera based on an attention residual network[J]. Chinese Optics, 2024, 17(6): 1316
Paper Information
Category:
Received: Oct. 31, 2023
Accepted: --
Published Online: Jan. 14, 2025
The Author Email:
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20