Optics and Precision Engineering, Volume. 31, Issue 18, 2752(2023)

Spatial-spectral Transformer for classification of medical hyperspectral images

Yuan LI1, Xu SHI1, Zhengchun YANG2, Qijuan TAN3、*, and Hong HUANG1、*
Author Affiliations
  • 1Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
  • 2Women and Children’s Hospital of Chongqing Medical University, Chongqing 401147, China
  • 3Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 40000, China
  • show less
    References(29)

    [1] [1] 李伟, 吕蒙, 陈天虹, 等. 高光谱图像在生物医学中的应用[J]. 中国图象图形学报, 2021, 26(8): 1764-1785.LIW, LYU M, CHENT H, et al. Application of a hyperspectral image in medical field: a review[J]. Journal of Image and Graphics, 2021, 26(8): 1764-1785.(in Chinese)

    [2] [2] CHENY R, WANGY N, ZHANGH, et al. 医学高光谱显微成像与智能分析关键技术研究及应用[J]. AI-View, 2022(3): 22-32.(in Chinese)陈煜嵘, 王耀南, 张辉, 等. 医学高光谱显微成像与智能分析关键技术研究及应用[J]. 人工智能, 2022(3): 22-32.

    [3] UL A, REHMAN, UL A, REHMAN. A review of the medical hyperspectral imaging systems and unmixing algorithms’ in biological tissues[J]. Photodiagnosis and Photodynamic Therapy, 33, 102165(2021).

    [4] [4] 闫敬文, 陈宏达, 刘蕾. 高光谱图像分类的研究进展[J]. 光学 精密工程, 2019, 27(3): 680-693. doi: 10.3788/ope.20192703.0680YANJ W, CHENH, LIUL. Overview of hyperspectral image classification [J]. Opt. Precision Eng., 2019, 27(3): 680-693. (in Chinese). doi: 10.3788/ope.20192703.0680

    [5] [5] 宋蓓蓓, 马穗娜, 何帆, 等. Res2-Unet深度学习网络的RGB-高光谱图像重建[J]. 光学 精密工程, 2022, 30(13): 1606-1619. doi: 10.37188/OPE.2021.0433SONGB B, MAS N, HEF, et al. Hyperspectral reconstruction from RGB images based on Res2-Unet deep learning network[J]. Opt. Precision Eng., 2022, 30(13): 1606-1619.(in Chinese). doi: 10.37188/OPE.2021.0433

    [6] [6] 时旭, 李远, 黄鸿. 面向高光谱显微图像血细胞分类的空-谱可分离卷积神经网络[J]. 光学 精密工程, 2022, 30(8):960-969. doi: 10.37188/ope.20223008.0960SHIX, LIY, HUANGH. Spatial-spectral separable convolutional neural network for cell classification of hyperspectral microscopic images[J]. Opt. Precision Eng., 2022, 30(8):960-969.(in Chinese). doi: 10.37188/ope.20223008.0960

    [7] [7] 郑少佳, 邱崧, 李庆利, 等. 傅里叶变换通道注意力网络的胆管癌高光谱图像分割[J]. 中国图象图形学报, 2021, 26(8):1836-1846. doi: 10.11834/jig.210207ZHENGS J, QIUS, LIQ L, et al. Fourier transform channel attention network for cholangiocarcinoma hyperspectral image segmentation[J]. Journal of Image and Graphics, 2021, 26(8):1836-1846.(in Chinese). doi: 10.11834/jig.210207

    [8] FABELO H, ORTEGA S, SZOLNA A et al. In-vivo hyperspectral human brain image database for brain cancer detection[J]. IEEE Access, 7, 39098-39116(2019).

    [9] BØTKER J, WU J X, RANTANEN J[M]. Data Handling in Science and Technology(2020).

    [10] YIFAN, DUAN, YIFAN, DUAN. Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images[J]. Optics & Laser Technology, 112, 530-538(2019).

    [11] RUIZ L, MARTÍN A, URBANOS G et al. Multiclass brain tumor classification using hyperspectral imaging and supervised machine learning[C], 1-6(2020).

    [12] BALTUSSEN E J M, KOK E N D, DE KONING S G B et al. Hyperspectral imaging for tissue classification, a way toward smart laparoscopic colorectal surgery[J]. Journal of Biomedical Optics, 24(2019).

    [13] HUANG Q, LI W, ZHANG B C et al. Blood cell classification based on hyperspectral imaging with modulated Gabor and CNN[J]. IEEE Journal of Biomedical and Health Informatics, 24, 160-170(2020).

    [14] WEI X L, LI W, ZHANG M M et al. Medical hyperspectral image classification based on end-to-end fusion deep neural network[J]. IEEE Transactions on Instrumentation and Measurement, 68, 4481-4492(2019).

    [15] ZHANG Q, WANG Y, QIU S et al. 3D-PulCNN: pulmonary cancer classification from hyperspectral images using convolution combination unit based CNN[J]. Journal of Biophotonics, 14(2021).

    [16] HU B L, DU J, ZHANG Z F et al. Tumor tissue classification based on micro-hyperspectral technology and deep learning[J]. Biomedical Optics Express, 10, 6370(2019).

    [18] ZHOU Z H, QIU S, WANG Y et al. Swin-spectral transformer for cholangiocarcinoma hyperspectral image segmentation[C], 1-6(23).

    [19] LI N Y, XUE J Q, JIA S. Spectral context-aware transformer for cholangiocarcinoma hyperspectral image segmentation[C], 209-213(9).

    [20] LI Y, SHI X, YANG L P et al. MC-GAT: multi-layer collaborative generative adversarial transformer for cholangiocarcinoma classification from hyperspectral pathological images[J]. Biomedical Optics Express, 13, 5794-5812(2022).

    [21] PENG Y S, ZHANG Y W, TU B et al. Spatial-spectral transformer with cross-attention for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-15(2022).

    [22] OUYANG E, LI B, HU W J et al. When multigranularity meets spatial–spectral attention: a hybrid transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 61, 1-18(2023).

    [23] LIU B, YU A Z, GAO K L et al. DSS-TRM: deep spatial-spectral transformer for hyperspectral image classification[J]. European Journal of Remote Sensing, 55, 103-114(2022).

    [24] ROY S K, KRISHNA G, DUBEY S R et al. HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 17, 277-281(2020).

    [25] ZHONG Z L, LI J, LUO Z M et al. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 56, 847-858(2018).

    [26] LI R, ZHENG S Y, DUAN C X et al. Classification of hyperspectral image based on double-branch dual-attention mechanism network[J]. Remote Sensing, 12, 582(2020).

    [27] HONG D F, HAN Z, YAO J et al. SpectralFormer: rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-15(2022).

    [28] SUN L, ZHAO G R, ZHENG Y H et al. Spectral–spatial feature tokenization transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-14(2022).

    [29] ZHANG J J, MENG Z, ZHAO F et al. Convolution transformer mixer for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 19, 1-5(2022).

    Tools

    Get Citation

    Copy Citation Text

    Yuan LI, Xu SHI, Zhengchun YANG, Qijuan TAN, Hong HUANG. Spatial-spectral Transformer for classification of medical hyperspectral images[J]. Optics and Precision Engineering, 2023, 31(18): 2752

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Information Sciences

    Received: Jan. 23, 2023

    Accepted: --

    Published Online: Oct. 12, 2023

    The Author Email: Qijuan TAN (hhuang@cqu.edu.cn), Hong HUANG (jiangliao2000@163.com)

    DOI:10.37188/OPE.20233118.2752

    Topics