Spatial-spectral Transformer for classification of medical hyperspectral images

Layer name	Parameter
Linear_1	Linear， 81×81
Spectral-wise Downsampling	Conv2D， 3×3， 80， stride 1， padding 1
Linear_2	Linear， 81×384
Conv2D （Spatial Attention）	Conv2D， kernel_size×kernel_size， 1， stride 1， padding 1
MLP （Spectral Attention）	Conv2D， 3×3， 160， stride 1， padding 1
MLP （Spectral Attention）	Conv2D， 3×3， 80， stride 1， padding 1
Conv3D_1	Conv3D， 3×3×3， 2， stride 1， padding 1
Conv3D_2	Conv3D， 3×3×3， 2， stride 1， padding 1
Conv2D_1	Conv2D， 3×3， 80， stride 1， padding 1
Conv2D_2	Conv2D， 3×3， 80， stride 1， padding 1
Linear_3	Linear， 128×144
Linear_4	Linear， 80×4

Table 2. Experimental setup on Brain and BloodCell HSI datasets

View table

View in Article

Table 2. Experimental setup on Brain and BloodCell HSI datasets

Setting Item	Dataset
Setting Item	Brain HSI	BloodCell HSI
Training Set	50% of patients	BloodCell1-3
Valid Set	50% of patients	BloodCell1-3
Test Set	50% of patients	BloodCell2-2
Training Samples	30 samples/class
Valid Samples	10 samples/class
Test Samples	All samples
Loss Function	Cross Entropy Loss
Epoch	300
Batchsize	128
Optimizer	Adamw
Python Version	3.7
Deep Learning Framework	Paddle
GPU	Tesla V100 （32 G）
Computing Platform	Baidu AI Studio

Table 3. Ablation analysis of the proposed S³AT with a combination of different components
View table
View in Article
Table 3. Ablation analysis of the proposed S³AT with a combination of different components
NV S³A Metrics
OA/% AA/% KC/%
1 × 72.83 72.49 62.88
1 √ 76.22 76.93 73.23
2 √ 79.09 77.64 69.92
3 √ 80.62 78.37 74.96
4 √ 79.33 78.00 71.61
5 √ 76.43 76.42 67.80

Table 4. Classification results of different algorithms on Brain HSI Dataset

View table

View in Article

Table 4. Classification results of different algorithms on Brain HSI Dataset

Class	CNN-based Methods			Transformer-based Methods
Class	HybridSN	SSRN	DBDA	ViT	CTMixer	SSFTT	S³AT
Background Normal Tumor	61.64±5.26	72.76±2.01	67.29±6.76	62.63±9.95	71.42±3.85	60.53±4.50	77.27±9.97
	31.73±9.75	37.32±9.61	35.28±9.04	15.90±4.59	35.25±9.32	23.08±5.26	73.00±9.99
	84.02±4.70	84.25±3.59	82.51±1.46	67.45±12.37	83.09±1.83	83.84±0.87	83.48±2.15
Blood	98.95±2.12	100.00±0.00	95.12±10.22	69.22±16.45	99.99±0.03	99.55±0.93	95.35±7.88
OA	68.32±3.75	72.25±2.67	68.99±3.98	51.75±5.93	71.07±2.77	65.66±1.35	82.25±2.88
AA	69.08±3.29	73.58±2.15	70.05±3.79	53.80±5.07	72.44±2.28	66.75±1.20	82.27±3.03
KC	58.15±4.79	63.13±3.47	58.87±5.30	36.72±7.42	61.76±3.52	54.65±1.74	76.17±3.86

Table 5. Classification results of different algorithms on BloodCell HSI Dataset

View table

View in Article

Table 5. Classification results of different algorithms on BloodCell HSI Dataset

Class	CNN-based Methods			Transformer-based Methods
Class	HybridSN	SSRN	DBDA	ViT	CTMixer	SSFTT	S³AT
RedBloodCell Background WhiteBloodCell	87.16±1.21	85.35±3.07	85.60±1.21	70.79±3.08	88.52±1.48	88.39±3.59	89.22±2.69
	96.71±1.46	96.82±3.79	95.54±1.61	93.77±4.19	96.59±3.93	95.67±1.29	98.64±3.79
	61.67±3.68	84.60±2.60	83.44±1.83	76.50±3.39	55.35±3.26	47.32±2.55	80.90±2.93
OA	89.21±3.88	88.71±2.40	88.25±3.45	77.13±3.19	90.45±1.61	89.50±1.39	91.74±2.79
AA	81.85±1.32	89.26±1.75	88.19±0.85	80.35±2.01	80.37±1.64	77.13±1.48	88.97±1.75
KC	77.06±2.89	76.66±3.02	75.53±1.44	57.78±3.36	78.34±0.77	77.06±2.32	81.86±2.19

Table 6. Parameters， FLOPs and inference time comparison of different algorithms
View table
View in Article
Table 6. Parameters， FLOPs and inference time comparison of different algorithms
Parameters/（×10⁶）
FlOPs/
（×10⁹）
Inference Time/s
HybridSN 14.33 8.15 0.055 6
SSRN 1.26 20.78 0.096 4
DBDA 2.31 32.20 0.169 1
ViT 18.82 54.12 0.179 3
CTMixer 0.61 1.63 0.018 8
SSFTT 3.63 3.32 0.027 6
S³AT 2.00 7.00 0.045 6

Tools

Get Citation

Copy Citation Text

Yuan LI, Xu SHI, Zhengchun YANG, Qijuan TAN, Hong HUANG. Spatial-spectral Transformer for classification of medical hyperspectral images[J]. Optics and Precision Engineering, 2023, 31(18): 2752

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Information Sciences

Received: Jan. 23, 2023

Accepted: --

Published Online: Oct. 12, 2023

The Author Email: TAN Qijuan (hhuang@cqu.edu.cn), HUANG Hong (jiangliao2000@163.com)

DOI:10.37188/OPE.20233118.2752

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Model parameter of S3AT

Table 1. Model parameter of S3AT

Table 2. Experimental setup on Brain and BloodCell HSI datasets

Table 2. Experimental setup on Brain and BloodCell HSI datasets

Table 3. Ablation analysis of the proposed S3AT with a combination of different components

Table 3. Ablation analysis of the proposed S3AT with a combination of different components

Table 4. Classification results of different algorithms on Brain HSI Dataset

Table 4. Classification results of different algorithms on Brain HSI Dataset

Table 5. Classification results of different algorithms on BloodCell HSI Dataset

Table 5. Classification results of different algorithms on BloodCell HSI Dataset

Table 6. Parameters， FLOPs and inference time comparison of different algorithms

Table 6. Parameters， FLOPs and inference time comparison of different algorithms

微信扫一扫：分享

Table 1. Model parameter of S³AT

Table 1. Model parameter of S³AT

Table 3. Ablation analysis of the proposed S³AT with a combination of different components

Table 3. Ablation analysis of the proposed S³AT with a combination of different components