Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network

Fig. 5. Fused images of unsupervised learning and semisupervised learning. (a) MRI-T1 image; (b) MRI-T2 image; (c) (d) CT images; (e) (f) fused images of unsupervised training; (g) (h) fused images of semisupervised training

Download full size

Fig. 6. Fused results of MRI-T1 and MRI-T2 images. (a) MRI-T1; (b) MRI-T2; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 7. Fused results of MRI-T1 and CT images. (a) MRI-T1; (b) CT; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 8. Fused results of MRI-T2 and CT images. (a) MRI-T1; (b) CT; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 9. Implementation of SSL-FWGAN for fusing MRI and PET images

Download full size

Fig. 10. Fused results of MRI-T1 and PET images. (a) MRI-T1; (b) PET; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 11. Fused results of MRI-T2 and PET images. (a) MRI-T1; (b) PET; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Table 1. Index results of unsupervised training and semisupervised training

View table

Table 1. Index results of unsupervised training and semisupervised training

Training strategy	Fused result of MRI-T1 and CT images				Fused result of MRI-T2 and CT images
Training strategy	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Unsupervised training	0.65	30.92	0.39	8.41	0.61	28.81	0.33	8.31
Semisupervised training	0.72	31.48	0.80	8.94	0.78	30.61	0.74	8.61

Table 2. Index results of different methods for fusing gray images

View table

Table 2. Index results of different methods for fusing gray images

Method	MRI-T1 and MRI-T2 images				MRI-T1 and CT images				MRI-T2 and CT images
Method	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Deepfuse	0.88	20.58	0.38	5.28	0.93	19.29	0.34	2.84	0.97	16.94	0.35	4.06
FusionGAN	0.60	14.62	0.29	4.44	0.61	11.96	0.23	4.46	0.67	10.27	0.21	2.54
PF-GAN	0.68	33.58	0.35	7.76	0.67	26.53	0.37	6.06	0.69	29.19	0.32	5.75
DDcGAN	0.61	33.51	0.35	7.92	0.60	30.94	0.37	3.65	0.54	30.84	0.26	8.74
DIDFuse	0.75	28.14	0.45	6.71	0.74	25.65	0.36	4.34	0.71	24.72	0.36	6.35
U2Fusion	0.71	24.83	0.34	6.75	0.68	22.52	0.37	6.04	0.69	20.73	0.37	5.79
SSL-FWGAN	0.76	33.74	0.76	9.32	0.74	32.41	0.79	8.69	0.74	32.34	0.75	9.48

Table 3. Index results of different methods for fusing color images

View table

Table 3. Index results of different methods for fusing color images

Method	MRI-T1 and PET images				MRI-T2 and PET images
Method	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Deepfuse	0.91	23.71	0.49	7.81	0.91	13.09	0.47	4.37
FusionGAN	0.43	14.09	0.29	4.94	0.49	10.68	0.29	3.40
PF-GAN	0.54	34.26	0.51	10.66	0.60	18.59	0.42	6.28
DDcGAN	0.47	43.59	0.54	13.50	0.52	26.49	0.34	8.33
DIDFuse	0.47	23.44	0.26	7.52	0.67	16.00	0.40	5.05
U2Fusion	0.55	26.85	0.34	9.13	0.64	15.24	0.27	4.40
SSL-FWGAN	0.56	47.28	0.57	15.56	0.49	27.09	0.44	9.21

Tools

Get Citation

Copy Citation Text

Haitao Yin, Yongying Yue. Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2215005

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Machine Vision

Received: Aug. 12, 2021

Accepted: Oct. 13, 2021

Published Online: Sep. 23, 2022

The Author Email: Haitao Yin (haitaoyin@njupt.edu.cn)

DOI:10.3788/LOP202259.2215005

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology