Laser & Optoelectronics Progress, Volume. 61, Issue 18, 1812004(2024)

Anti-Disturbance Cross-Scene Multispectral Imaging Pigment Classification Method for Painted Cultural Relics

Ruanzhao Guo1, Ke Wang1, Huiqin Wang1、*, Zhan Wang2, Gang Zhen2, Yuan Li3, and Jiachen Li1
Author Affiliations
  • 1School of Information and Control Engineering, Xi'an University of Architecture and Technology, Xi'an 710055, Shaanxi, China
  • 2Shaanxi Provincial Institute of Cultural Relics Protection, Xi'an 710075, Shaanxi, China
  • 3Xi'an Museum, Xi'an 710074, Shaanxi, China
  • show less
    Figures & Tables(32)
    Schematic diagram of the process for anti-disturbance cross-scene multispectral imaging for classification of pigments in painted cultural relics
    Spectral image of the mural in the 660 nm channel
    Flowchart of anti-spectral disturbance
    Schematic diagram of the overlapping region of two shots. (a) Sketch of the mural sub-lens shooting; (b) multispectral images of overlapping region of sub-lens; (c) spectral image and grayscale histogram of 660 nm overlapping region
    Basic image and histogram specification images. (a) Basic image Lij; (b) histogram specification image Nij
    Anti-spectral perturbation results. (a) Basic imag Sij; (b) anti-spectral perturbation image Fij; (c) Sij histogram;(d) Fij histogram;(e) two-shot multi-spectral image stitching after anti-spectral perturbation
    Painted cultural relic pigment classification network structure with cross-scene domain shift attributes
    Diagram of encoder
    Structure diagram of feature extractor
    Flowchart of classification and identification of pigments in painted cultural relics
    Schematic diagram of multispectral imaging system
    Experimental data of anti-spectral perturbation by sub-shot. (a) Schematic diagram of shot shooting of simulated murals; (b) shot 1 captures multispectral images at 620 nm; (c) shot 2 captures multispectral images at 620 nm; (d) multispectral images before resisting spectral perturbation at 620 nm; (e) multispectral images after resisting spectral perturbation at 620 nm
    Ground truth and marking diagram of simulated mural. (a) Ground truth of simulated mural; (b) marking diagram of simulated mural
    OA and Kappa coefficients of each algorithm before and after anti-spectral disturbance
    Images of simulated pigment plate and simulated mural at 620 nm. (a) Multispectral image of pigment plate at 620 nm; (b) interest marker image of pigment board points; (c) simulation mural of single exposure shot at 620 nm
    Generalization ability experiment. (a) Generated data distribution; (b) data distribution after processing by the discriminator
    Results of cross-scene classification between paint panels and simulated murals. (a) Ground truth; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) proposed method
    Schematic diagram of the scene shooting of the Mingwang statue in Dule Temple
    Colored and labeled images of mural. (a) Mural color image; (b) shot 1 training samples
    Classification effect of different methods on murals. (a) Proposed method; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) SVM
    Schematic diagram of XRF test points and multispectral images of the Thirteenth Venerable
    XRF analysis results of the Thirteenth Venerable. (a) XRF analysis result of point 1; (b) XRF analysis result of point 2; (c) XRF analysis result of point 3; (d) XRF analysis result of point 4
    Pigment multispectral image training set
    Diagram of the pigment distribution of the Thirteenth Holiness
    • Table 1. Simulated mural multi-spectral image sample

      View table

      Table 1. Simulated mural multi-spectral image sample

      NumberCategoryNumber of samples
      0Background898151
      1Vermilion56752
      2Lazurite80086
      3Minium1167
      4Mineral green44533
      5Chrome yellow301516
      6Graphite49720
    • Table 2. Classification results of different methods before and after anti-spectral perturbation

      View table

      Table 2. Classification results of different methods before and after anti-spectral perturbation

      AlgorithmHPDM-SPRN27BASSNet28DFFN29Proposed
      Before resisting spectral perturbation
      After resisting spectralperturbation
    • Table 3. Classification accuracy of different methods before and after anti-spectral disturbance

      View table

      Table 3. Classification accuracy of different methods before and after anti-spectral disturbance

      ClassHPDM-SPRN27BASSNet28DFFN29Proposed
      Before calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibration
      Vermilion67.1298.0896.3094.1770.7597.9197.8899.8
      Lazurite97.9197.9299.0299.2998.3398.7596.7394.5
      Minium95.2097.5191.9594.1795.4697.3473.69100
      Mineral green94.3275.0068.7682.9783.5275.2957.9494.9
      Chrome yellow82.0491.2591.9095.6894.6993.9197.0099.3
      Graphite85.1388.0982.7781.5689.5883.7697.3890.9
      OA /%84.1791.3490.6593.6891.2892.5793.7897.49
      Kappa×10076.8086.7285.5890.1386.5288.4695.8398.30
    • Table 4. Quality objective evaluation results of image

      View table

      Table 4. Quality objective evaluation results of image

      AlgorithmRMSEPSNRSSIM
      Before calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibration
      DHCNet231.841.5722.4225.020.960.96
      FDSSC242.101.5621.8625.110.960.97
      SSRN251.851.9920.8522.950.960.96
      SSKR262.041.6520.4623.300.950.96
      HPDM-SPRN271.981.6820.7024.350.940.96
      BASSNet281.761.4524.6325.540.960.97
      DFFN291.491.5525.2324.650.970.97
      Proposed1.530.7027.5128.540.990.99
    • Table 5. Samples of simulated mural multispectral images

      View table

      Table 5. Samples of simulated mural multispectral images

      NumberCategoryNumber of samples(source domain)Number of samples(target domain)
      0Background396043939600
      1Vermilion2876356775
      2Lazurite2971579070
      3Minium127851270
      4Mineral green4681449590
      5Chrome yellow19805284795
    • Table 6. Classification accuracy of different methods on pigment cross-scene

      View table

      Table 6. Classification accuracy of different methods on pigment cross-scene

      ClassDHCNet23FDSSC24SSRN25SSKR26HPDM-SPRN27BASSNet28DFFN29Proposed
      Vermilion61.4299.9098.94010099.93089.66
      Lazurite10099.9898.630099.9610095.29
      Minium94.4110096.851.811.2657.3263.6298.11
      Mineral green73.5399.8199.8910016.0426.1446.1199.27
      Chrome yellow63.5470.5734.4942.730.2488.8582.7685.91
      OA /%70.5382.1860.0536.3313.8785.3671.7889.37
      Kappa×10056.2573.4148.6816.89-5.6575.8654.2979.02
    • Table 7. Results of objective evaluation of image quality cross-scene

      View table

      Table 7. Results of objective evaluation of image quality cross-scene

      AlgorithmRMSEPSNRSSIM
      DHCNet232.8821.960.95
      FDSSC242.0319.690.95
      SSRN253.0716.350.92
      SSKR265.0520.350.95
      HPDM-SPRN276.3319.270.95
      BASSNet282.4526.490.97
      DFFN293.2624.010.96
      Proposed1.41825.930.98
    • Table 8. Ablation experiments of different modules

      View table

      Table 8. Ablation experiments of different modules

      ModelOA /%Kappa×100
      Remove encoder82.6273.54
      Remove contrastive learning84.4074.82
      Remove adversarial learning86.3476.58
      Proposed89.3779.02
    Tools

    Get Citation

    Copy Citation Text

    Ruanzhao Guo, Ke Wang, Huiqin Wang, Zhan Wang, Gang Zhen, Yuan Li, Jiachen Li. Anti-Disturbance Cross-Scene Multispectral Imaging Pigment Classification Method for Painted Cultural Relics[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1812004

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Instrumentation, Measurement and Metrology

    Received: Jan. 3, 2024

    Accepted: Feb. 5, 2024

    Published Online: Sep. 14, 2024

    The Author Email: Huiqin Wang (hqwang@xauat.edu.cn)

    DOI:10.3788/LOP240448

    CSTR:32186.14.LOP240448

    Topics