Three dimensional-CNN Classification Method of Mural Multispectral Image Pigments Based on Multiscale Feature Fusion

Yunle Ding; Huiqin Wang; Ke Wang; Zhan Wang; Gang Zhen

doi:10.3788/LOP202259.2230001

Laser & Optoelectronics Progress, Volume. 59, Issue 22, 2230001(2022)

Three dimensional-CNN Classification Method of Mural Multispectral Image Pigments Based on Multiscale Feature Fusion

Yunle Ding¹, Huiqin Wang^1、*, Ke Wang¹, Zhan Wang², and Gang Zhen²

¹School of Information and Control Engineering, Xi'an University of Architecture and Technology, Xi'an 710055, Shaanxi, China

²Shaanxi Provincial Institute of Cultural Relics Protection, Xi'an 710075, Shaanxi, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(15)

Fig. 1. Schematic diagrams of 2D-CNN and 3D-CNN. (a) 2D-CNN; (b) 3D-CNN

Download full size

Fig. 2. Residual learning

Download full size

Fig. 3. Schematic diagrams of atrous convolution kernel. (a) r=1; (b) r=2; (c) r=3

Download full size

Fig. 4. Sserial hole convolution module

Download full size

Fig. 5. Multiscale feature fusion module

Download full size

Fig. 6. Schematic of MFAC-Res3D-CNN

Download full size

Fig. 7. Multispectral image acquisition system of murals

Download full size

Fig. 8. Multispectral images of murals in each band

Download full size

Fig. 9. Simulated mural. (a) Simulated mural image; (b) pseudo color image; (c) truth image

Download full size

Fig. 10. Comparison of different network classification results. (a) Truth image; (b) SVM; (c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN

Download full size

Fig. 11. Comparison of different network classification details. (a) Truth image; (b) SVM;(c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN

Download full size

Table 1. Sample of multispectral image dataset of simulated murals
View table
Table 1. Sample of multispectral image dataset of simulated murals
Number Category Color Number of samples
0 Background 871622
1 Mercuric sulfide 57778
2 Mineral green 49641
3 Chrome yellow 294816
4 Graphite 58765
5 Lazurite 77061
6 Minium 1417

Table 2. Classification confusion matrix of article model

View table

Table 2. Classification confusion matrix of article model

Category	Background	Mercuric sulfide	Mineral green	Chrome yellow	Graphite	Lazurite	Minium
Background	865044	1036	1677	2548	1082	215	20
Mercuric sulfide	573	57166	22	17	0	0	0
Mineral green	1677	100	47859	5	0	0	0
Chrome yellow	2692	91	1	292031	0	1	0
Graphite	2049	0	7	136	56573	0	0
Lazurite	1452	0	22	52	18	75517	0
Minium	135	0	0	1	0	0	1281

Table 3. Comparison of dataset classification accuracy results

View table

Table 3. Comparison of dataset classification accuracy results

Category	SVM	2D-CNN	Res-3D-CNN	MFAC-Res3D-CNN
Background	84.36	97.49	97.34	99.24
Mercuric sulfide	92.36	91.05	95.43	98.94
Mineral green	78.53	92.34	94.25	96.41
Chrome yellow	88.53	99.32	97.50	99.05
Graphite	80.31	99.88	94.64	96.26
Lazurite	86.72	84.90	97.01	97.99
Minium	63.78	82.63	89.87	90.40
OA	84.72	91.45	97.57	98.87
AA	82.08	92.51	95.14	96.89
Kappa	78.60	89.98	95.41	98.04

Table 4. Comparison of training and testing time of different algorithms
View table
Table 4. Comparison of training and testing time of different algorithms
Time SVM 2D-CNN Res-3D-CNN MFAC-Res3D-CNN
Train 678.5 1037.8 1265.6 1301.5
Test 7.98 10.75 15.95 17.05

Tools

Get Citation

Copy Citation Text

Yunle Ding, Huiqin Wang, Ke Wang, Zhan Wang, Gang Zhen. Three dimensional-CNN Classification Method of Mural Multispectral Image Pigments Based on Multiscale Feature Fusion[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2230001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites