Optics and Precision Engineering, Volume. 32, Issue 21, 3244(2024)

3DRes-ViT knee osteoarthritis classification model based on multimodal fusion

Yu SONG1, Rui XU1, Xiaodong CAI2, and Xin WANG1、*
Author Affiliations
  • 1College of Computer Science and Engineering, Changchun University of Technology, Changchun34000, China
  • 2Information Department, Jilin Qianwei Hospital, Changchun13001, China
  • show less
    References(29)

    [1] LESPASIO M J, PIUZZI N S, HUSNI M E et al. Knee Osteoarthritis: A Primer[J]. Perm J, 21, 16-183(2017).

    [2] OEI E H G, RUNHAAR J. Imaging of early-stage osteoarthritis: the needs and challenges for diagnosis and classification[J]. Skeletal Radiology, 52, 2031-2036(2023).

    [3] KELLGREN J H, LAWRENCE J S. Radiological assessment of osteo-arthrosis[J]. Annals of the Rheumatic Diseases, 16, 494-502(1957).

    [4] ROEMER F W, KWOH C K, HAYASHI D et al. The role of radiography and MRI for eligibility assessment in DMOAD trials of knee OA[J]. Nature Reviews Rheumatology, 14, 372-380(2018).

    [5] JURAS V, CHANG G, REGATTE R R. Current status of functional MRI of osteoarthritis for diagnosis and prognosis[J]. Current Opinion in Rheumatology, 32, 102-109(2020).

    [6] HUNTER D J, GUERMAZI A, LO G H et al. Evolution of semi-quantitative whole joint assessment of knee OA: MOAKS (MRI osteoarthritis knee score)[J]. Osteoarthritis and Cartilage, 19, 990-1002(2011).

    [7] NORMAN B, PEDOIA V, NOWOROLSKI A et al. Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs[J]. Journal of Digital Imaging, 32, 471-477(2019).

    [8] THOMAS K A, KIDZIŃSKI Ł, HALILAJ E et al. Automated classification of radiographic knee osteoarthritis severity using deep neural networks[J]. Radiology Artificial Intelligence, 2(2020).

    [9] SWIECICKI A, LI N Y, O’DONNELL J et al. Deep learning-based algorithm for assessment of knee osteoarthritis severity in radiographs matches performance of radiologists[J]. Computers in Biology and Medicine, 133, 104334(2021).

    [10] TIULPIN A, SAARAKKALA S. Automatic grading of individual knee osteoarthritis features in plain radiographs using deep convolutional neural networks[J]. Diagnostics, 10, 932(2020).

    [11] ALSHAREEF E A, EBRAHIM F, LAMAMI Y et al. Knee osteoarthritis severity grading using vision transformer[J]. Journal of Intelligent & Fuzzy Systems, 43, 8303-8313(2022).

    [12] GUIDA C, ZHANG M, SHAN J. Knee osteoarthritis classification using 3D CNN and MRI[J]. Applied Sciences, 11, 5196(2021).

    [13] YEOH P S Q, LAI K W, GOH S L et al. Transfer learning-assisted 3D deep learning models for knee osteoarthritis detection: data from the osteoarthritis initiative[J]. Frontiers in Bioengineering and Biotechnology, 11, 1164655(2023).

    [14] GUIDA C, ZHANG M, SHAN J. Improving knee osteoarthritis classification using multimodal intermediate fusion of X-ray, MRI, and clinical information[J]. Neural Computing and Applications, 35, 9763-9772(2023).

    [15] KARIM M R, JIAO J, DOHMEN T et al. DeepKneeExplainer: explainable knee osteoarthritis diagnosis from radiographs and magnetic resonance imaging[J]. IEEE Access, 9, 39757-39780(2021).

    [16] 吴穗岚, 陈乐, 曾涛. 基于卷积神经网络的膝关节炎患者的膝盖红外图像分类[J]. 中国计量大学学报, 30, 185-190(2019).

         WU S L, CHEN L, ZENG T. Knee infrared image classification of knee arthritis patients based on convolutional neural networks[J]. Journal of China University of Metrology, 30, 185-190(2019).

    [17] HE F ZH, NIU K, TANG SH et al. Study on machine learning model of primary bone tumor around knee joint assisted diagnosis based on X-ray images[J]. Progress in Modern Biomedicine, 21, 2842-2847(2021).

         何方舟, 牛凯, 唐顺. 基于X线图像的膝关节周围原发性骨肿瘤辅助诊断的机器学习模型研究[J]. 现代生物医学进展, 21, 2842-2847(2021).

    [18] 曹传贵, 林强, 满正行. 基于VGG的SPECT骨扫描图像关节炎分类[J]. 西北民族大学学报(自然科学版), 42, 36-45(2021).

         CAO CH G, LIN Q, MAN ZH X et al. VGG-based classification of arthritis in bone SPECT images[J]. Journal of Northwest Minzu University (Natural Science), 42, 36-45(2021).

    [19] 任会峰, 伏建雄, 鄢锋. 基于旋转不变非线性局部模糊编码的膝骨关节炎辅助诊断[J]. 软件导刊, 21, 25-31(2022).

         REN H F, FU J X, YAN F et al. Knee osteoarthritis aided diagnoses based on rotation invariant nonlinear local fuzzy coding[J]. Software Guide, 21, 25-31(2022).

    [20] DOSOVITSKIY A, BEYER L, KOLESNIKOV A et al. An image is worth 16x16 words: transformers for image recognition at scale[J]. ArXiv e-Prints(2020).

    [21] CHEN P J, GAO L L, SHI X S et al. Fully automatic knee osteoarthritis severity grading using deep neural networks with a novel ordinal loss[J]. Computerized Medical Imaging and Graphics, 75, 84-92(2019).

    [22] WANG Q L, WU B G, ZHU P F et al. ECA-net: efficient channel attention for deep convolutional neural networks[C], 13, 2020(2020).

    [23] ZHANG B F, TAN J M, CHO K et al. Attention-based CNN for KL grade classification: data from the osteoarthritis initiative[C], 3, 2020(2020).

    [24] LIN T Y, GOYAL P, GIRSHICK R et al. Focal loss for dense object detection[C](29).

    [25] BHAGWAN S, SONY A. Generative adversarial networks for synthesizing high-quality medical images[C], 29, 2024(2024).

    Tools

    Get Citation

    Copy Citation Text

    Yu SONG, Rui XU, Xiaodong CAI, Xin WANG. 3DRes-ViT knee osteoarthritis classification model based on multimodal fusion[J]. Optics and Precision Engineering, 2024, 32(21): 3244

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 12, 2024

    Accepted: --

    Published Online: Jan. 24, 2025

    The Author Email: Xin WANG (wangxin315@ccut.edu.cn)

    DOI:10.37188/OPE.20243221.3244

    Topics