Optics and Precision Engineering, Volume. 31, Issue 18, 2752(2023)

Spatial-spectral Transformer for classification of medical hyperspectral images

Yuan LI1... Xu SHI1, Zhengchun YANG2, Qijuan TAN3,* and Hong HUANG1,* |Show fewer author(s)
Author Affiliations
  • 1Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
  • 2Women and Children’s Hospital of Chongqing Medical University, Chongqing 401147, China
  • 3Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 40000, China
  • show less
    References(29)

    [1] [1] 李伟, 吕蒙, 陈天虹, 等. 高光谱图像在生物医学中的应用[J]. 中国图象图形学报, 2021, 26(8): 1764-1785.LIW, LYU M, CHENT H, et al. Application of a hyperspectral image in medical field: a review[J]. Journal of Image and Graphics, 2021, 26(8): 1764-1785.(in Chinese)

    [2] [2] CHENY R, WANGY N, ZHANGH, et al. 医学高光谱显微成像与智能分析关键技术研究及应用[J]. AI-View, 2022(3): 22-32.(in Chinese)陈煜嵘, 王耀南, 张辉, 等. 医学高光谱显微成像与智能分析关键技术研究及应用[J]. 人工智能, 2022(3): 22-32.

    [3] UL A, REHMAN, UL A, REHMAN. A review of the medical hyperspectral imaging systems and unmixing algorithms’ in biological tissues. Photodiagnosis and Photodynamic Therapy, 33, 102165(2021).

    [4] [4] 闫敬文, 陈宏达, 刘蕾. 高光谱图像分类的研究进展[J]. 光学 精密工程, 2019, 27(3): 680-693. doi: 10.3788/ope.20192703.0680YANJ W, CHENH, LIUL. Overview of hyperspectral image classification [J]. Opt. Precision Eng., 2019, 27(3): 680-693. (in Chinese). doi: 10.3788/ope.20192703.0680

    [5] [5] 宋蓓蓓, 马穗娜, 何帆, 等. Res2-Unet深度学习网络的RGB-高光谱图像重建[J]. 光学 精密工程, 2022, 30(13): 1606-1619. doi: 10.37188/OPE.2021.0433SONGB B, MAS N, HEF, et al. Hyperspectral reconstruction from RGB images based on Res2-Unet deep learning network[J]. Opt. Precision Eng., 2022, 30(13): 1606-1619.(in Chinese). doi: 10.37188/OPE.2021.0433

    [6] [6] 时旭, 李远, 黄鸿. 面向高光谱显微图像血细胞分类的空-谱可分离卷积神经网络[J]. 光学 精密工程, 2022, 30(8):960-969. doi: 10.37188/ope.20223008.0960SHIX, LIY, HUANGH. Spatial-spectral separable convolutional neural network for cell classification of hyperspectral microscopic images[J]. Opt. Precision Eng., 2022, 30(8):960-969.(in Chinese). doi: 10.37188/ope.20223008.0960

    [7] [7] 郑少佳, 邱崧, 李庆利, 等. 傅里叶变换通道注意力网络的胆管癌高光谱图像分割[J]. 中国图象图形学报, 2021, 26(8):1836-1846. doi: 10.11834/jig.210207ZHENGS J, QIUS, LIQ L, et al. Fourier transform channel attention network for cholangiocarcinoma hyperspectral image segmentation[J]. Journal of Image and Graphics, 2021, 26(8):1836-1846.(in Chinese). doi: 10.11834/jig.210207

    [8] H FABELO, S ORTEGA, A SZOLNA et al. In-vivo hyperspectral human brain image database for brain cancer detection. IEEE Access, 7, 39098-39116(2019).

    [9] J BØTKER, J X WU, J RANTANEN. Data Handling in Science and Technology(2020).

    [10] YIFAN, DUAN, YIFAN, DUAN. Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images. Optics & Laser Technology, 112, 530-538(2019).

    [11] L RUIZ, A MARTÍN, G URBANOS et al. Multiclass brain tumor classification using hyperspectral imaging and supervised machine learning, 1-6(2020).

    [12] E J M BALTUSSEN, E N D KOK, S G B DE KONING et al. Hyperspectral imaging for tissue classification, a way toward smart laparoscopic colorectal surgery. Journal of Biomedical Optics, 24(2019).

    [13] Q HUANG, W LI, B C ZHANG et al. Blood cell classification based on hyperspectral imaging with modulated Gabor and CNN. IEEE Journal of Biomedical and Health Informatics, 24, 160-170(2020).

    [14] X L WEI, W LI, M M ZHANG et al. Medical hyperspectral image classification based on end-to-end fusion deep neural network. IEEE Transactions on Instrumentation and Measurement, 68, 4481-4492(2019).

    [15] Q ZHANG, Y WANG, S QIU et al. 3D-PulCNN: pulmonary cancer classification from hyperspectral images using convolution combination unit based CNN. Journal of Biophotonics, 14(2021).

    [16] B L HU, J DU, Z F ZHANG et al. Tumor tissue classification based on micro-hyperspectral technology and deep learning. Biomedical Optics Express, 10, 6370(2019).

    [18] Z H ZHOU, S QIU, Y WANG et al. Swin-spectral transformer for cholangiocarcinoma hyperspectral image segmentation, 1-6(23).

    [19] N Y LI, J Q XUE, S JIA. Spectral context-aware transformer for cholangiocarcinoma hyperspectral image segmentation, 209-213(9).

    [20] Y LI, X SHI, L P YANG et al. MC-GAT: multi-layer collaborative generative adversarial transformer for cholangiocarcinoma classification from hyperspectral pathological images. Biomedical Optics Express, 13, 5794-5812(2022).

    [21] Y S PENG, Y W ZHANG, B TU et al. Spatial-spectral transformer with cross-attention for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-15(2022).

    [22] E OUYANG, B LI, W J HU et al. When multigranularity meets spatial–spectral attention: a hybrid transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 61, 1-18(2023).

    [23] B LIU, A Z YU, K L GAO et al. DSS-TRM: deep spatial-spectral transformer for hyperspectral image classification. European Journal of Remote Sensing, 55, 103-114(2022).

    [24] S K ROY, G KRISHNA, S R DUBEY et al. HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17, 277-281(2020).

    [25] Z L ZHONG, J LI, Z M LUO et al. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56, 847-858(2018).

    [26] R LI, S Y ZHENG, C X DUAN et al. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12, 582(2020).

    [27] D F HONG, Z HAN, J YAO et al. SpectralFormer: rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-15(2022).

    [28] L SUN, G R ZHAO, Y H ZHENG et al. Spectral–spatial feature tokenization transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1-14(2022).

    [29] J J ZHANG, Z MENG, F ZHAO et al. Convolution transformer mixer for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 19, 1-5(2022).

    Tools

    Get Citation

    Copy Citation Text

    Yuan LI, Xu SHI, Zhengchun YANG, Qijuan TAN, Hong HUANG. Spatial-spectral Transformer for classification of medical hyperspectral images[J]. Optics and Precision Engineering, 2023, 31(18): 2752

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Information Sciences

    Received: Jan. 23, 2023

    Accepted: --

    Published Online: Oct. 12, 2023

    The Author Email: TAN Qijuan (hhuang@cqu.edu.cn), HUANG Hong (jiangliao2000@163.com)

    DOI:10.37188/OPE.20233118.2752

    Topics