Research Progress in Deep Learning for Magnetic Resonance Diagnosis of Knee Osteoarthritis

Shuchen Lin; Dejian Wei; Shuai Zhang; Hui Cao; Yuzheng Du

doi:10.3788/LOP232102

Deep learning model

Characteristic

Superiority

Limitation

DenseNet^［9］

A convolutional neural network structure with dense connections that can share parameters and avoid feature redundancy

It can effectively extract features from knee joint disease images， helping doctors accurately determine the lesion area

The network structure is complex，memory consumption is high， and more computing resources are required

ResNet^［10］

Introducing a convolutional neural network structure with residual connections，which directly transmits input information to subsequent layers through residual connections， solving the problems of gradient vanishing and gradient explosion

To assist doctors in analyzing and predicting patients’ disease progression，and provide personalized treatment plans

The model has a large depth and may encounter gradient vanishing， and exploding problems during training

Mask R-CNN^［11］

An image segmentation model based on regional convolutional neural networks that can achieve pixel level segmentation while detecting objects

It can help doctors quickly and accurately locate and segment knee joint lesion areas，suitable for targets with different scales，shapes， and categories

Requires more computing resources and requires high computing power

U-Net^［12］

A simple image segmentation model consisting of an encoder and decoder， and using skip connections to achieve information transmission of feature maps

It can help doctors segment image data of knee joint diseases in patients，provide more detailed lesion information， and assist doctors in formulating treatment plans

Relying on a large amount of annotated data，the segmentation effect of the overall structure is poor

Imaging type

Imaging principle

Superiority

Limitation

Radiography （X-ray）

By utilizing the principle of X-rays penetrating objects and being received by detectors， image reconstruction is performed on the differences in the absorption of rays， achieving imaging of the internal structure of the human body^［18］

1．Evaluate bone structures， such as osteophytes， subchondral cysts， and cysts^［19］

2．Detecting joint gap width and indirectly evaluating joint cartilage ^［20-21］

1.Unable to depict soft tissue

2.Unable to visualize cartilage directly

3.Inadequate monitoring of disease progression ^［22］

Computer tomography

By utilizing rotational X-ray imaging technology， a large amount of X-ray data is obtained， and through computer reconstruction algorithms， high-resolution images of the human cross-section are generated ^［23］

Used for bone structure examination， providing a good 3D visualization imaging ^{［17，24］}

1.Expose patients to radiation ^［25］

2.Invasive ^［26］

3.Can only recognize KOA late detection ^［27］

Optical coherence tomography

By utilizing the interference of light and the measurement of optical path difference， the intensity and phase information of the interference signal are recorded to achieve high-resolution cross-sectional image reconstruction of tissues ^［28-29］

Assessing the disease status of KOA patients and monitoring changes in articular cartilage thickness ^［30］

1.OCT imaging is an invasive process

2.Dependent on operator’s technology

Ultrasound imaging technology

Utilize the propagation and reflection characteristics of ultrasound in human tissues to receive and process reflected ultrasound signals， and generate images of human tissues^［31-32］

Used to detect synovitis and synovial inflammation in patients with knee osteoarthritis， without the need for contrast agents or exposure to radiation ^［24-25］

1.The obtained images and their interpretation rely too heavily on the skills and experience of technical personnel ^［27］

2.No significant role in early detection of KOA ^{［27，33］}

Magnetic resonance imaging

Utilizing the characteristics of nuclear spin under strong magnetic field and radio frequency excitation， detecting the energy released by spin recovery， and reconstructing high-resolution images of human tissue ^［34］

1.Visualize the entire joint structure， including direct three-dimensional visualization of bones and cartilage

2.Differentiate various tissue types with high anatomical resolution

3.Highly sensitive to structural changes in OA patients without exposure to radiation

4.Non-invasive^［35］

1.The cost of inspection is relatively high

2.MR images can be affected by artifacts， which can affect diagnosis ^［27］

Multiple sequence type of magnetic resonance imaging

Method

Medical effect

Characteristic

MERGE imaging^［38］

High anatomical visibility of bone and joint anatomy and joint spaces

Able to display defects on joint surfaces

Complementary to conventional sequences may detect lesions with smaller structures

3D SPGR imaging^［28］

Provide high-resolution 3D image data

Accurate quantitative evaluation of cartilage thickness and volume has been achieved， increasing the signal strength of cartilage relative to adjacent tissues and joint contents （such as synovial fluid）

Can avoid partial volume artifacts

3D DESS imaging^［39］

Collect two or more gradient echoes， each pair separated by refocusing pulses， and combine the data of the two in image reconstruction

Used to detect cartilage lesions in the knee joint； Can quantitatively evaluate cartilage thickness and volume

Shorter collection time， higher signal-to-noise ratio， and higher cartilage fluid contrast； Not easily affected by artifacts

SPACE imaging^［40］

Collect images at isotropic spatial resolution， allowing for multi plane reformatting

Used for the diagnosis of cartilage lesions. Cartilage fluid CNR and its ability to distinguish between cartilage and surrounding tissues are weak

It has the best signal-to-noise ratio and signal-to-noise ratio efficiency， low sensitivity to artifacts， but longer acquisition time

3D FSE imaging^［41］

Provide images with high contrast resolution and isotropic spatial resolution

3D evaluation for dissecting the structure of the knee joint can effectively evaluate the abnormalities of subchondral bone marrow

Isotropic voxels allow for multi plane reformatting of image data， reducing some volume artifacts. Relying on flipping angle modulation to reduce blur， parallel imaging to reduce acquisition time

Deep learning model

Magnetic resonance sequence

Magnetic field strength /T

Detection range

result

Detection

Characteristic

Mask R-CNN^［57］

FS-T2-weighted

3.0

Detect and identify normal and torn meniscus， and determine the direction of tear

AUC is 0.906

A mask region based convolutional neural network （R-CNN） is trained to make it more robust through set aggregation， and cascaded into a shallow convolutional network to classify the direction of tearing

DarkNet-53^［58］

T1-weighted （PD） FSE with fat saturation， T2-weighted FSE with fat saturation sequences

3.0

Automatic detection of meniscus tear on the sagittal plane

Accuracy of the internal dataset is 0.958

Using a 24-layer CNN and a 2-layer fully connected DarkNet-53 network as the backbone network， feature compression is performed on the annotated training images of the meniscus region. Using gradient weighted class activation Maps （Grad CAMs） to detect meniscal tears

ResNet^［59］

DESS， IW TSE

3.0

Detection of meniscus tears in three anatomical subregions (anterior horn, corpus, posterior horn) of the medial meniscus (MM) and lateral meniscus (LM)

AUC values of MM are 0.84， 0.88， and 0.86. AUC values of LM are 0.95， 0.91， and 0.90

A multi task deep learning framework combining boundary box regression for meniscus tear detection is proposed， allowing CNN to implicitly consider the corresponding region of interest （ROI） of the meniscus

Mask R-CNN^［60］

SAG FS PDW， SAG FS T2-weighted

1.5/3.0

Diagnosis of healthy meniscus， torn meniscus， and degenerative meniscus

Diagnostic accuracy is 87.50%，86.96%, and 84.78%, respectively

This paper adopts the ResNet50 architecture as the backbone network， combined with a feature pyramid network and feature mapping from bottom to top. Simultaneously， a region proposal network （RPN） is used to determine the ROI in the network

U-Net^［61］

SAG 3D PDW COR， SAG FS sensitive， MRI

3.0

Diagnose the severity of knee joint lesions

Sensitivity is 89.81%， specificity is 81.98%

This paper proposes the concept of a fully automated deep learning pipeline for identifying OA degenerative morphological features in meniscus and PFJ cartilage. At the same time， a weak supervision method is also applied to annotate the presence or absence of lesions.

Deep learning model

Magnetic resonance sequence

Magnetic field strength /T

Detection range

Detection result

Characteristic

CenterNet^［64］

DESS

3.0

Accurately detect KOA and determine its severity

The testing accuracy is 99.14%， and the cross validation accuracy is 98.97%

Using the modified CenterNet to obtain input from the predicted bounding box， a more accurate bounding box can be obtained based on the voting score of each pixel in the previous box. In addition， distillation knowledge was also used

Improved U-Net^［65］

DESS

3.0

Segmentation of femur and tibia

Dice coefficient are 0.985 and 0.984，respectively

Ronneberger et al.^［12］ improved the U-Net model and modified its parameters

Model in Ref.［66］

DESS

3.0

Segmentation of femur and tibia

Femur DSC=0.85，

Tibia DSC=0.84

This paper develops a new MRI algorithm for quantitative automatic segmentation of human knee cartilage volume. This method utilizes filtered and optimized texture analysis techniques and Bayesian decision criteria-based techniques to achieve automatic separation of cartilage and synovial fluid

ResNet， DenseNet， and convolutional variational autoencoder （CVAE） ^［67］

intermediate-weighted turbo spin-echo （IW-TSE）

‒

Segmentation of femur and tibia

Dice coefficient are 0.985 and 0.987， respectively

Ability to detect the presence of OA by comparing three deep learning models

Model in Ref.［68］

proton density weighted（PDW）

3.0

Segmentation of femoral， tibial trabeculae， and cortical bone

Dice similarity coefficient （DSC） of the femur and tibia is 0.9611 ± 0.0052 and 0.9591 ± 0.0173， respectively

This paper proposes a method based on proton density weighted comparison. This method uses the level set method of local energy for 3D rough segmentation and corrects the non-uniformity problem of image data. By generating strength lines layer by layer and optimizing trabecular boundaries， the detection of cortical boundaries is ultimately achieved

DRD U-Net^［69］

DESS

‒

Segmentation of femur and tibia

Dice coefficient is 0.931

This model takes the residual module as the basic module， utilizes parallel extended convolution modules， and designs a deep supervised module for multi output fusion， directly utilizing features from different levels to achieve information complementarity

Deep learning model

Magnetic resonance sequence

Magnetic field strength /T

Detection range

Detection result

Characteristic

MRNet^［75］

coronal T1 weighted， coronal T2 with fat saturation， sagittal proton density （PD） weighted， sagittal T2 with fat saturation， and axial PD weighted with fat saturation

1.5， 3.0

Detection of ACL tear in the knee joint

AUC is 0.965 on the internal validation set

The MRNet model improves the accuracy and generalization ability by using data preprocessing and model fine-tuning. Evaluate the performance of the model through cross validation and evaluation indicators， and conduct explanatory analysis of the model using class activation mapping （CAM）

DenseNet^［76］

Sagittal PDW and fat suppression T2 weighted fast spin echo images

3.0

Feasibility of detecting ACL tear in the knee joint

The diagnostic performance of this model is not significantly different from that of clinical radiologists

By studying DenseNet， VGG16， and AlexNet models， it was found that DenseNet provides the best diagnostic performance in detecting ACL tears

ResNet^［77］

PDW

1.5

Detection of ACL injury

The AUC for healthy tearing， partial tearing， and complete tearing are 0.980， 0.970， and 0.99， respectively

A custom 14 layer ResNet-14 architecture convolutional neural network （CNN） was constructed using class balancing and data augmentation techniques， which includes six convolutional operations in different directions

A classification convolutional neural network was constructed based on the structure of 3D DenseNet^［78］

2D PDW-SPAIR sequence

1.5， 3.0

Evaluating the feasibility of anterior cruciate ligament injury

The accuracy is 0.957， and the sensitivity is 0.976

AUC is 0.960

This paper constructs a classification convolutional neural network based on the 3D DenseNet architecture by using different inputs and two other algorithms （VGG16 and ResNet）

Deep learning model construction based on multimodal feature fusion^［79］

T2 WI-SPAIR sagittal and T2 weighted cross-sectional images

‒

Diagnose ACL damage

Accuracy up to 96.28%

This paper proposes a deep learning based magnetic resonance imaging feature method， which analyzes and extracts image features related to anterior cruciate ligament injury， thereby achieving automatic learning and accurate recognition of anterior cruciate ligament injury

Composed of two CNNs^［80］

T2 weighted sequence based on coronal and sagittal fat suppression proton density

1.0， 1.5， 3.0

Predicting ACL tear

AUC is 0.939

Construct a deep learning based ACL tear detector using knee joint MRI databases from different manufacturers and magnetic fields. Automatically label reports related to MRI examination using natural language processing algorithms

Shuchen Lin, Dejian Wei, Shuai Zhang, Hui Cao, Yuzheng Du. Research Progress in Deep Learning for Magnetic Resonance Diagnosis of Knee Osteoarthritis[J]. Laser & Optoelectronics Progress, 2024, 61(14): 1400005

Category: Reviews

Received: Sep. 12, 2023

Accepted: Nov. 8, 2023

Published Online: Jul. 17, 2024

The Author Email: Hui Cao (caohui63@163.com)

DOI:10.3788/LOP232102

CSTR:32186.14.LOP232102

Table 1. Summary of commonly used deep learning models

Table 1. Summary of commonly used deep learning models

Table 2. Comparison of different medical imaging methods

Table 2. Comparison of different medical imaging methods

Table 3. Summary of magnetic resonance multi-sequence imaging technology

Table 3. Summary of magnetic resonance multi-sequence imaging technology

Table 5. Comparison of detection algorithms and magnetic resonance imaging for meniscus

Table 5. Comparison of detection algorithms and magnetic resonance imaging for meniscus

Table 6. Comparison of detection algorithms and magnetic resonance imaging for femur and tibia

Table 6. Comparison of detection algorithms and magnetic resonance imaging for femur and tibia

Table 7. Comparison of detection algorithms and magnetic resonance imaging for anterior cruciate ligament

Table 7. Comparison of detection algorithms and magnetic resonance imaging for anterior cruciate ligament