Optics and Precision Engineering, Volume. 32, Issue 7, 1087(2024)

Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer

Haibin WU1... Shiyu DAI1, Aili WANG1,*, Iwahori YUJI2 and Xiaoyu YU3 |Show fewer author(s)
Author Affiliations
  • 1Heilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, College of Measurement and Control Technology and Communication Engineering, Harbin University of Science and Technology, Harbin50080, China
  • 2Department of Computer Science, Chubu University, Aichi487-8501, Japan
  • 3College of Electron and Information, University of Electronic Science and Technology of China,Zhongshan Institute, Zhongshan528400, China
  • show less
    Figures & Tables(15)
    Model architecture of CLCT-Net
    Schematic diagram of shared feature extraction network
    Schematic diagram of HSI encoder
    Schematic diagram of LiDAR encoder
    Pseudo color map and ground-truth map of Houston2013 dataset
    Pseudo color map and ground-truth map of Trento dataset
    Feature visualizations of Houston2013 dataset
    Feature visualizations of Trento dataset
    Classification results of different methods on Houston2013 dataset
    Classification results of different methods on Trento dataset
    • Table 1. Land class details in Houston2013 dataset

      View table
      View in Article

      Table 1. Land class details in Houston2013 dataset

      Class nameTrain numTest numColor
      Healthy grass1981 053
      Stressed grass1901 064
      Synthetic grass192505
      Trees1881 056
      Soil1861 056
      Water182143
      Residential1961 072
      Commercial1911 053
      Road1931 059
      Highway1911 036
      Railway1811 054
      Parking Lot11921 041
      Parking Lot2184285
      Tennis court181247
      Running track187473
      Total2 83212 197
    • Table 2. Land class details in Trento dataset

      View table
      View in Article

      Table 2. Land class details in Trento dataset

      Class nameTrain numTest numColor
      Apples1293 905
      Buildings152 778
      Ground105374
      Woods1548 969
      Wineyard18410 317
      Roads1223 052
      Total81929 395
    • Table 3. Comparison of classification accuracy of different methods on Houston2013 dataset

      View table
      View in Article

      Table 3. Comparison of classification accuracy of different methods on Houston2013 dataset

      ClassTwo-BranchEndNetMDL-MiddleMAHiDFNetSpectrum-LiDARTB-HSICLCT-Net

      C1

      C2

      C3

      C4

      C5

      C6

      C7

      C8

      C9

      C10

      C11

      C12

      C13

      C14

      C15

      83.10

      84.10

      100.00

      93.09

      100.00

      99.30

      92.82

      82.34

      84.70

      65.44

      88.24

      89.53

      92.28

      96.76

      99.79

      81.58

      83.65

      100.00

      93.09

      99.91

      95.10

      82.65

      81.29

      88.29

      89.00

      83.78

      90.39

      82.46

      100.00

      98.10

      83.10

      85.06

      99.60

      91.57

      98.86

      100.00

      97.64

      88.13

      85.93

      74.42

      84.54

      95.39

      87.37

      95.14

      100.00

      98.53

      92.87

      91.11

      98.10

      98.38

      98.58

      99.15

      80.94

      98.04

      72.81

      72.71

      76.80

      95.80

      99.53

      100.53

      49.17

      35.93

      72.12

      65.08

      59.63

      24.54

      75.61

      75.23

      74.74

      62.37

      85.37

      50.13

      41.26

      28.46

      70.24

      100.0

      98.04

      30.64

      99.28

      99.62

      93.38

      85.66

      92.76

      94.46

      88.41

      96.16

      84.15

      96.30

      100.00

      90.27

      87.07

      98.05

      96.67

      94.78

      99.34

      77.14

      89.50

      83.31

      94.33

      92.84

      95.79

      86.26

      87.37

      100.00

      94.22

      OA87.9888.5289.5589.5860.8185.2292.01
      AA90.1189.9591.0591.3657.9989.9691.78
      K×10086.9887.5987.5988.7457.6784.0291.33
    • Table 4. Comparison of classification accuracy of different methods on Trento dataset

      View table
      View in Article

      Table 4. Comparison of classification accuracy of different methods on Trento dataset

      ClassTwo-BranchEndNetMDL-MiddleMAHiDFNetSpectrum-LiDARTB-HSICLCT-Net

      C1

      C2

      C3

      C4

      C5

      C6

      99.78

      97.93

      99.93

      99.46

      98.96

      91.68

      88.19

      98.49

      95.19

      99.30

      91.96

      90.14

      99.50

      97.55

      99.10

      99.90

      99.71

      92.25

      99.91

      88.92

      97.53

      99.98

      99.90

      99.78

      74.00

      62.45

      26.00

      99.54

      98.45

      88.94

      99.19

      81.24

      63.46

      99.93

      97.35

      94.52

      99.30

      97.49

      96.45

      99.29

      99.70

      96.28

      OA98.3694.1798.7398.5984.9495.4298.90
      AA97.9693.8898.0097.5574.9089.2898.10
      K×10097.8392.2298.3298.1280.5693.8998.54
    • Table 5. FLOPs and parameters of different classification models

      View table
      View in Article

      Table 5. FLOPs and parameters of different classification models

      Method#param./MFLOPs/M
      Two-Branch1225
      EndNet0.070.49

      MDL-Middle

      MAHiDFNet

      0.25

      77

      4.7

      155

      CLCT-Net5.1384
    Tools

    Get Citation

    Copy Citation Text

    Haibin WU, Shiyu DAI, Aili WANG, Iwahori YUJI, Xiaoyu YU. Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer[J]. Optics and Precision Engineering, 2024, 32(7): 1087

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Oct. 23, 2023

    Accepted: --

    Published Online: May. 28, 2024

    The Author Email:

    DOI:10.37188/OPE.20243207.1087

    Topics