Application of Segment Anything Model in Medical Image Segmentation

Tong Wu; Haoji Hu; Yang Feng; Qiong Luo; Dong Xu; Weizeng Zheng; Neng Jin; Chen Yang; Jincao Yao

doi:10.3788/CJL240614

Chinese Journal of Lasers, Volume. 51, Issue 21, 2107102(2024)

Application of Segment Anything Model in Medical Image Segmentation

Tong Wu^1,2, Haoji Hu^1,2、*, Yang Feng³, Qiong Luo⁴, Dong Xu^5,6, Weizeng Zheng⁷, Neng Jin⁴, Chen Yang^5,6, and Jincao Yao^5,6

Author Affiliations

¹University of Illinois Urbana-Champaign Institute, Zhejiang University, Hangzhou 314400, Zhejiang , China

²College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, Zhejiang , China

³Angelalign Research Institute, Anglealign Technology Inc., 200433, Shanghai , China

⁴Department of Obstetrics, Women's Hospital School of Medicine Zhejiang University, Hangzhou 310006, Zhejiang , China

⁵Zhejiang Cancer Hospital, Hangzhou 331022, Zhejiang , China

⁶Hangzhou Institute of Medicine (HIM), Chinese Academy of Sciences, Hangzhou 310000, Zhejiang , China

⁷Department of Radiology, Women's Hospital School of Medicine Zhejiang University, Hangzhou 310006, Zhejiang , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(8)

Fig. 1. Overview of the architecture of SAM

Download full size

Fig. 2. Architecture of medical SAM adapter (MSA)^[72]

Download full size

Fig. 3. Architecture of SemiSAM^[122]

Download full size

Fig. 4. Architecture of ASLseg^[123]

Download full size

Fig. 5. Segmentation results of SAM, MedSAM, and SAM-Med2D networks on the pregnant pelvis MRI dataset (top row) and the ABUS tumor dataset (bottom row) under the condition of box and point prompts

Download full size

Table 1. Experimental analysis related to the direct application of SAM on medical datasets (In the performance column, ">" means better than, "<" means worse than)

View table

Table 1. Experimental analysis related to the direct application of SAM on medical datasets (In the performance column, ">" means better than, "<" means worse than)

Ref.	Modality	Segmentation task	Dataset	Prompt	Performance
［27］	WSI	Skin cancer tumour	TCGA^［28］	20 points	>SOTA
	WSI	Glomerular unit （CAP）， glomerular tuft （TUFT）， distal tubular （DT）， proximal tubular （PT）， arteries （VES）， and peritubular capillaries （PTC）	Private datasets （Using digital renal biopsies from the NEPTUNE study^［29］）	Points	<SOTA
	WSI	Cell nuclei	MoNuSeg^［30］	Points	<SOTA
［31］	CT	Liver tumour	Private datasets	Points	<SOTA
［32］	CT	Tumour	Private datasets	Points	<SOTA
［33］	CT	Abdominal organs	AMOS^［34］	Points， boxs	<SOTA
［35］	MRI	Brain tumour	BraTS ^［36］		Verified that SAM can achieve high segmentation accuracy in expert-interactive workflows
［37］	MRI	Brain tumour	BraTS ^［36］	Points， boxs	<SOTA
［38］	MRI	Brain	ATLAS^［39］， WMH^［40］， and private datasets	Boxs	>SOTA
［41］	Ultrasound	Breast tumour	BUSI^［42］		Verified that the ViT-L version of SAM has the best segmentation performance compared to other versions
［43］	Endoscopy	Polyp	Kvasir-SEG^［44］， CVC-ClinicDB^［45］，CVC-ColonDB^［46］，ETIS^［47］， and CVC300^［48］	Points	<SOTA
［49］	Surgical images	Surgical instrument	EndoVis^［50-51］	Points	<SOTA
［49］	Surgical images	Surgical instrument	EndoVis^［50-51］	Boxs	>SOTA
［4］	CT	Lung nodules	CIR^［52］	Points， boxs	<SOTA
		Pancreatic parenchyma and mass	Pancreas^［53-54］
		Liver tumor	LiTS^［55］
	MRI	Ventricular endocardium on cardiac MRI	ACDC^［56］
		Prostate	Prostate^［57］
		Left atrium	LA^［58］
		Hippocampus	Hippo ^［54］
	MRI， flair	Brain tumor	BraTS^{［36，59-60］}
	Ultrasound	Breast tumor	BUID^［42］
	Colonoscopy	Polyps	Kvasir^［44］
	Dermoscopy	Skin lesion	ISIC ^［61-62］
	X-ray	Lung	Chest X-ray ^［63］
［64］	CT	Liver and liver tumor	LiTs^［55］	Points， boxs	<SOTA Verified that the box prompts-assisted results are best in most cases
	MRI	Hippocampus	ADNI^［65］
	Ultrasound	Breast tumour	BUSI^［42］
	Ultrasound	Thyroid nodule and thyroid gland	TN3K^［66］
	Endoscopy	Polyp	Kvasir^［44］，CVC-ClinicDB ^［45］，CVC-ColonDB ^［46］，CVC-300^［48］， and ETIS^［47］
	Foot ulcer	Foot ulcer images	FUSC^［67］
	OCT	Macular edema	OCT^［68］
	X-ray	Lung	COVID-Lung^［69］

Table 2. Segmentation capacity of SAM networks fine-tuned on medical datasets with different data volumes for two modal images

View table

Table 2. Segmentation capacity of SAM networks fine-tuned on medical datasets with different data volumes for two modal images

Network	MRI			Ultrasound
Network	Images in training	DSC /%	mIoU /%	Images in training	DSC /%	mIoU /%
SAM	0	62.44±23.77	61.13±23.47	0	52.01±28.03	40.01±25.93
MedSAM	360000	71.63±8.87	69.81±11.35	30000	56.65±22.80	42.85±21.19
SAM-Med2D	700000	76.23±7.60	71.08±9.24	380000	75.11±17.79	62.78±18.83

Table 3. Comparison of segmentation effects on ABUS tumor dataset before and after combining MT and SAM

View table

Table 3. Comparison of segmentation effects on ABUS tumor dataset before and after combining MT and SAM

Method	Prompt	DSC /%	mIoU /%	Accuracy /%	HD95 /mm
MT	None	39.88±39.82	33.52±34.86	55.70±49.53	6.37±25.32
MT+SAM	Point	51.23±36.21	42.54±31.25	64.13±42.98	4.75±22.56
MT+SAM	Point and box	54.46±37.44	45.08±29.39	66.75±40.58	4.26±18.83

Tools

Get Citation

Copy Citation Text

Tong Wu, Haoji Hu, Yang Feng, Qiong Luo, Dong Xu, Weizeng Zheng, Neng Jin, Chen Yang, Jincao Yao. Application of Segment Anything Model in Medical Image Segmentation[J]. Chinese Journal of Lasers, 2024, 51(21): 2107102

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites