Self-Supervised Feature Learning Method for Hyperspectral Images Based on Mixed Convolutional Networks

Data information	IP	HU	LK	HC
Spectral range /μm	0.4-2.5	0.38-1.05	0.4-1.0	0.4-1.0
Number of bands	200	144	270	274
Spatial resolution /m	20	2.5	0.463	0.109
Image size	145×145	349×1905	550×400	1217×303
Number of labeled data	10249	15029	204542	257530
Number of classes	16	15	9	16
Number of samples used in self-supervised learning	10249	15029	50000	50000
Number of samples used in fine-tuning	80	75	45	80

Table 2. Classification results of different models in IP dataset

View table

Table 2. Classification results of different models in IP dataset

No.	AD-HybridSN	A2S2K-ResNet	SSFTT	SPRLT	DMVL	SS-MFN
1	98.05	94.39	99.76	98.05	51.64	98.05
2	45.49	38.40	52.14	58.49	66.83	68.21
3	50.56	42.87	52.98	47.77	73.43	63.84
4	91.12	84.22	90.30	89.83	61.84	91.38
5	79.23	71.80	80.73	83.28	82.30	81.63
6	91.26	80.62	94.59	94.08	93.75	94.69
7	100.00	100.00	100.00	100.00	26.31	100.00
8	97.06	84.69	92.92	97.10	91.98	98.60
9	100.00	98.67	100.00	100.00	22.68	100.00
10	64.38	52.00	71.88	77.67	71.80	71.96
11	64.70	56.20	58.45	58.76	90.81	72.17
12	63.40	46.80	58.08	67.28	72.68	73.47
13	97.75	98.70	98.10	99.60	86.79	99.20
14	88.20	86.38	88.91	90.08	96.01	99.02
15	80.50	82.23	77.90	86.38	87.81	84.44
16	98.64	99.09	99.20	99.32	90.96	97.39
Kappa	66.69±4.68	57.67±7.93	66.74±3.80	69.50±3.35	76.04±3.16	76.87±2.25
OA	70.16±4.42	62.36±7.13	70.29±3.51	72.67±3.06	78.14±2.87	79.46±2.03
AA	81.90±2.23	76.07±5.74	82.25±1.94	84.23±1.59	72.98±3.95	87.13±1.12

Table 3. Classification results of different models in HU dataset

View table

Table 3. Classification results of different models in HU dataset

No.	AD-HybridSN	A2S2K-ResNet	SSFTT	SPRLT	DMVL	SS-MFN
1	77.89	85.35	79.23	80.26	78.64	84.36
2	82.37	83.46	85.52	82.34	72.62	89.23
3	95.72	98.70	96.23	99.51	98.67	97.57
4	82.53	93.27	83.51	88.88	77.80	91.69
5	97.97	80.93	98.47	99.53	83.99	99.98
6	81.81	79.41	83.19	90.28	99.40	81.31
7	62.73	68.62	60.82	52.88	78.49	82.14
8	49.69	39.55	43.47	53.04	55.41	61.10
9	59.89	62.95	54.88	36.35	76.00	71.86
10	65.36	41.12	71.12	53.04	85.66	75.54
11	72.52	57.08	81.08	58.63	87.98	87.67
12	65.20	45.70	62.61	58.76	60.56	80.28
13	91.42	82.41	86.06	77.37	81.46	95.99
14	99.98	95.44	99.79	100.00	100.00	100.00
15	98.61	98.67	95.91	99.04	99.98	90.06
Kappa	73.63±3.77	68.15±3.66	73.72±3.84	68.85±2.17	74.03±4.08	83.06±1.79
OA	75.54±3.51	70.48±3.40	75.67±3.56	71.14±2.02	75.59±3.76	84.32±1.65
AA	78.91±2.68	74.18±3.06	78.79±3.21	75.33±1.52	82.44±2.88	85.92±1.09

Table 4. Classification results of different models in LK dataset

View table

Table 4. Classification results of different models in LK dataset

No.	AD-HybridSN	A2S2K-ResNet	SSFTT	SPRLT	DMVL	SS-MFN
1	93.02	97.43	95.11	93.27	92.29	98.95
2	95.35	93.39	92.57	52.92	82.42	95.23
3	98.70	93.55	98.84	79.86	38.66	98.73
4	84.56	83.06	72.88	66.46	95.59	88.36
5	87.52	86.75	92.78	63.32	35.91	99.53
6	91.45	96.99	96.03	95.72	90.60	96.69
7	93.10	99.32	95.86	97.79	99.81	94.85
8	77.20	80.21	73.64	71.74	87.32	75.76
9	83.06	81.74	85.44	77.42	52.66	88.11
Kappa	86.69±3.74	89.90±3.72	84.04±4.30	78.51±2.53	84.42±2.85	90.96±3.10
OA	89.60±3.03	92.14±2.97	87.45±3.51	82.99±2.14	87.83±2.34	92.97±2.48
AA	89.33±1.95	90.27±2.22	89.24±1.72	77.61±1.34	75.03±2.05	92.91±1.81

Table 5. Classification results of different models in HC dataset

View table

Table 5. Classification results of different models in HC dataset

No.	AD-HybridSN	A2S2K-ResNet	SSFTT	SPRLT	DMVL	SS-MFN
1	64.30	29.08	53.38	56.68	76.47	76.54
2	56.30	37.49	66.89	34.67	77.98	75.03
3	83.37	53.05	67.99	39.48	38.98	89.08
4	96.41	67.14	87.32	70.53	37.40	98.35
5	97.26	71.84	75.46	83.62	25.39	99.01
6	54.47	23.50	44.32	23.05	13.16	71.97
7	73.41	81.30	84.39	78.68	45.32	94.88
8	55.38	36.47	48.86	38.15	51.03	61.32
9	58.55	26.67	53.33	34.26	43.87	71.70
10	78.44	45.62	77.26	57.48	34.19	91.05
11	70.85	55.07	76.29	61.46	52.78	77.56
12	75.48	36.83	64.78	48.17	30.50	85.79
13	48.07	26.11	45.50	35.37	35.74	56.30
14	55.91	49.98	51.83	48.73	78.67	73.61
15	82.55	69.72	76.53	79.83	29.78	90.40
16	86.27	75.36	88.94	89.43	90.01	96.43
Kappa	66.79±5.75	44.96±9.42	64.51±6.51	55.17±4.15	55.23±3.26	79.50±2.75
OA	70.88±5.36	50.92±9.09	68.97±6.04	60.85±4.07	61.12±3.19	82.31±2.40
AA	71.06±2.47	49.08±6.80	66.44±6.16	54.97±2.20	47.58±2.70	81.81±1.92

Table 6. Experimental results of model ablation

View table

Table 6. Experimental results of model ablation

Model	Description	IP	HU	LK	HC
SS-MFN	Proposed method	79.46	84.32	92.97	82.31
Model-1	Using PCA instead of FA	76.51	82.37	92.59	76.40
Model-2	Without contrastive learning	78.18	77.94	90.61	81.79
Model-3	Without SOP	77.60	80.17	91.12	81.70

Table 7. Experimental results of contrastive learning-related settings

View table

Table 7. Experimental results of contrastive learning-related settings

Experimental target	Experimental setting	IP	HU	LK	HC
SS-MFN	—	79.46	84.32	92.97	82.31
Data augmentation	Using Crop only	80.84	78.73	87.51	82.14
Data augmentation	Using RCA only	77.89	81.05	92.74	80.01
Projection head	Without BN	78.57	82.70	92.99	81.89
Encoder	Linear probe	72.25	79.26	86.87	70.77

Table 8. Running time of each method
View table
Table 8. Running time of each method
Stage AD-HybridSN A2S2K-ResNet SSFTT SPRLT DMVL SS-MFN
Self-supervised learning — — — — 3386.69 1538.39
Fine-tuning 9.97 217.03 6.76 72.23 0.49 15.06
Testing 3.23 3.20 0.92 6.94 2.80 3.24

Tools

Get Citation

Copy Citation Text

Fan Feng, Yongsheng Zhang, Jin Zhang, Bing Liu, Ying Yu. Self-Supervised Feature Learning Method for Hyperspectral Images Based on Mixed Convolutional Networks[J]. Acta Optica Sinica, 2024, 44(18): 1828007

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites