Classification Based on Hyperspectral Image and LiDAR Data with Contrastive Learning

Shihan Li; Haiyang Hua; Hao Zhang

doi:10.3788/LOP230540

Laser & Optoelectronics Progress, Volume. 60, Issue 22, 2228006(2023)

Classification Based on Hyperspectral Image and LiDAR Data with Contrastive Learning

Shihan Li^1,2,3,4, Haiyang Hua^1,2、*, and Hao Zhang^1,2,3,4

Author Affiliations

¹Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, Liaoning , China

²Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences, Shenyang 110016, Liaoning , China

³Institute for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, Liaoning , China

⁴University of Chinese Academy of Sciences, Beijing 100049, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(24)

Fig. 1. Framework of the proposed method. (a) Contrastive learning; (b) fine tune

Download full size

Fig. 2. Convolution network module

Download full size

Fig. 3. Densenet block

Download full size

Fig. 4. Transformer Encoder block

Download full size

Fig. 5. Multi-head attention mechanism

Download full size

Fig. 6. Houston 2013 dataset. (a) HSI data; (b) LiDAR data; (c) ground-truth label

Download full size

Fig. 7. Trento dataset. (a) HSI data; (b) LiDAR data; (c) ground-truth label

Download full size

Fig. 8. Classification results on the Houston 2013 dataset

Download full size

Fig. 9. Classification results on the Trento dataset

Download full size

Fig. 10. Classification results on the Houston 2013 dataset when $u = 5$ . (a) EndNet; (b) CCRNet; (c) two brance CNN;(d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Fig. 11. Classification results on the Houston 2013 dataset when $u = 10$ . (a) EndNet; (b) CCRNet; (c) two brance CNN; (d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Fig. 12. Classification results on the Houston 2013 dataset when $u = 15$ . (a) EndNet; (b) CCRNet; (c) two brance CNN;(d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Fig. 13. Classification results on the Trento dataset when $u = 2$ . (a) EndNet; (b) CCRNet; (c) two brance CNN; (d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Fig. 14. Classification results on the Trento dataset when $u = 3$ . (a) EndNet; (b) CCRNet; (c) two brance CNN; (d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Fig. 15. Classification results on the Trento dataset when $u = 4$ . (a) EndNet; (b) CCRNet; (c) two brance CNN; (d) MAHiDFNet; (e) ours; (f) ground-truth label

Download full size

Table 1. Overall classification accuracy on the Houston 2013 dataset

View table

Table 1. Overall classification accuracy on the Houston 2013 dataset

Algorithm	$u = 5$	$u = 10$	$u = 15$	$u = 20$	$u = 25$	$u = 50$	$u = 100$
EndNet	54.38	68.94	74.12	77.33	80.66	85.54	91.36
CCRNet	47.97	65.87	77.95	82.28	84.38	93.24	94.25
two brance CNN	57.74	75.25	77.60	81.60	84.58	89.89	93.80
MAHiDFNet	55.12	75.00	82.93	86.08	85.67	89.95	91.21
ours	78.47	88.87	90.60	91.36	93.75	95.26	96.77

Table 2. Overall classification accuracy on the Trento dataset

View table

Table 2. Overall classification accuracy on the Trento dataset

Algorithm	$u = 2$	$u = 3$	$u = 4$	$u = 5$	$u = 6$	$u = 9$	$u = 12$
EndNet	58.70	73.02	81.11	76.63	79.68	85.29	88.20
CCRNet	67.80	72.00	70.50	78.90	83.64	86.24	93.94
two brance CNN	78.06	89.21	90.73	94.63	96.34	96.72	95.48
MAHiDFNet	79.77	84.96	92.80	95.87	98.13	99.09	98.98
ours	88.12	97.1	97.19	97.54	98.30	99.09	99.23

Table 3. Average time of each epoch of training
View table
Table 3. Average time of each epoch of training
Dataset EndNet CCRNet two brance CNN MAHiDFNet contrast learning fine tune ours
Houston 2013 0.13 0.40 36.45 6.80 3.82 0.44 3.37
Trento 0.20 0.20 26.28 2.06 0.94 0.39 0.72

Table 4. Overall classification accuracy of contrastive learning ablation experiments on the Houston 2013 dataset
View table
Table 4. Overall classification accuracy of contrastive learning ablation experiments on the Houston 2013 dataset
Condition $u = 5$ $u = 10$ $u = 15$ $u = 20$ $u = 25$ $u = 50$ $u = 100$
without contrast learning 63.62 72.69 81.58 85.14 85.71 93.13 93.41
with contrast learning 78.47 88.87 90.60 91.36 93.75 95.26 96.77

Table 5. Overall classification accuracy of contrastive learning ablation experiments on the Trento dataset
View table
Table 5. Overall classification accuracy of contrastive learning ablation experiments on the Trento dataset
Condition $u = 2$ $u = 3$ $u = 4$ $u = 5$ $u = 6$ $u = 9$ $u = 12$
without contrast learning 83.53 93.36 92.81 95.54 95.38 98.22 98.45
with contrast learning 88.12 97.12 97.19 97.54 98.30 99.09 99.23

Table 6. Overall classification accuracy on the Houston 2013 dataset with different data modality
View table
Table 6. Overall classification accuracy on the Houston 2013 dataset with different data modality
Data modality $u = 5$ $u = 10$ $u = 15$ $u = 20$ $u = 25$ $u = 50$ $u = 100$
HSI 59.79 72.77 78.63 82.81 84.10 90.63 93.95
LiDAR 45.42 58.07 62.95 67.38 70.32 78.52 85.22
HSI+ LiDAR 78.47 88.87 90.60 91.36 93.75 95.26 96.77

Table 7. Overall classification accuracy on the Trento dataset with different data modality
View table
Table 7. Overall classification accuracy on the Trento dataset with different data modality
Data modality $u = 2$ $u = 3$ $u = 4$ $u = 5$ $u = 6$ $u = 9$ $u = 12$
HSI 91.39 85.97 87.62 92.29 91.86 94.44 95.42
LiDAR 78.69 81.63 85.35 85.83 83.65 87.17 87.21
HSI+ LiDAR 88.12 97.12 97.19 97.54 98.30 99.09 99.23

Table 8. Overall classification accuracy of different sample sizes
View table
Table 8. Overall classification accuracy of different sample sizes
Dataset $s = 8$ $s = 9$ $s = 10$ $s = 11$ $s = 12$ $s = 13$ $s = 14$ $s = 15$
Houston 2013 87.97 89.89 88.34 90.60 90.72 93.13 91.14 94.10
Trento 97.74 97.28 98.06 97.54 97.49 97.47 97.73 97.00

Table 9. Overall classification accuracy of different reduction dimensions
View table
Table 9. Overall classification accuracy of different reduction dimensions
Dataset $k = 1$ $k = 3$ $k = 5$ $k = 7$ $k = 9$ $k = 11$
Houston 2013 74.94 87.44 91.05 90.60 92.22 93.03
Trento 96.25 96.45 97.59 97.54 96.93 97.39

Tools

Get Citation

Copy Citation Text

Shihan Li, Haiyang Hua, Hao Zhang. Classification Based on Hyperspectral Image and LiDAR Data with Contrastive Learning[J]. Laser & Optoelectronics Progress, 2023, 60(22): 2228006

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Remote Sensing and Sensors

Received: Jan. 30, 2023

Accepted: Mar. 13, 2023

Published Online: Nov. 6, 2023

The Author Email: Haiyang Hua (c3i11@sia.cn)

DOI:10.3788/LOP230540

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Overall classification accuracy on the Houston 2013 dataset

Table 1. Overall classification accuracy on the Houston 2013 dataset

Table 2. Overall classification accuracy on the Trento dataset

Table 2. Overall classification accuracy on the Trento dataset

Table 3. Average time of each epoch of training

Table 3. Average time of each epoch of training

Table 4. Overall classification accuracy of contrastive learning ablation experiments on the Houston 2013 dataset

Table 4. Overall classification accuracy of contrastive learning ablation experiments on the Houston 2013 dataset

Table 5. Overall classification accuracy of contrastive learning ablation experiments on the Trento dataset

Table 5. Overall classification accuracy of contrastive learning ablation experiments on the Trento dataset

Table 6. Overall classification accuracy on the Houston 2013 dataset with different data modality

Table 6. Overall classification accuracy on the Houston 2013 dataset with different data modality

Table 7. Overall classification accuracy on the Trento dataset with different data modality

Table 7. Overall classification accuracy on the Trento dataset with different data modality

Table 8. Overall classification accuracy of different sample sizes

Table 8. Overall classification accuracy of different sample sizes

Table 9. Overall classification accuracy of different reduction dimensions

Table 9. Overall classification accuracy of different reduction dimensions