Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement

Wanxin Xiao; Huafeng Li; Yafei Zhang; Minghong Xie; Fan Li

doi:10.3788/LOP202259.0617029

Laser & Optoelectronics Progress, Volume. 59, Issue 6, 0617029(2022)

Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement

Wanxin Xiao^1,2, Huafeng Li^1,2, Yafei Zhang^1,2、*, Minghong Xie¹, and Fan Li^1,2

¹Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming , Yunnan 650500, China

²Yunnan Key Laboratory of Artificial Intelligence, Kunming , Yunnan 650500, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(10)

Fig. 1. Basic framework of multi-scale feature learning and edge enhancement fusion network

Download full size

Fig. 2. Fusion results obtained by the CNN and NSCT-PAPCNN methods. (a) Source CT image; (b) source MR-T2 image; (c) CNN; (d) NSCT-PRPCNN

Download full size

Fig. 3. Source image samples in testing dataset

Download full size

Fig. 4. Visual comparison of the different methods. (a) Source CT image; (b) source MR-T2 image; (c) FW-Net; (d) GF; (e) CNN; (f) NSCT; (g) NSCT-PRPCNN; (h) proposed method

Download full size

Fig. 5. Fusion results without and with edge reinforcement branches. (a) Without edge reinforcement branch; (b) with edge reinforcement branch

Download full size

Fig. 6. Fusion results obtained by the proposed method in MR-T1 and MR-T2 image fusion task. (a) Source MR-T1 image; (b) source MR-T2 image; (c) proposed method

Download full size

Table 1. Structure of MFEnet

View table

Table 1. Structure of MFEnet

Module

Convolutional layer

Size

Number of input channels

Number of output channels

Activation layer

Feature extraction module

192

ReLU

Reconstruction module

C10

ReLU

Table 2. Algorithms of MFEnet training and testing

View table

Table 2. Algorithms of MFEnet training and testing

Algorithm MFEnet training and testing algorithms

Training

Input：Training set source image $I_{i}$

Output：Reconstructed image $I_{o}$

1）Randomly select m source images from the training set $\{I^{(1)}, \dots, I^{(m)}\}$

2）Input m source images into the feature extraction module to generate source image features $F$

3）Input $F$ into the reconstruction module to generate a reconstructed image $I_{o}$

4）Use Adam optimizer to update the parameters of the feature extraction module and reconstruction module：

$\nabla_{θ} \frac{1}{W H} ({‖I_{o} - I_{i}‖}_{F}^{2} + α {‖\nabla I_{o} - \nabla I_{i}‖}_{F}^{2})$

5）If the number of iterations is equal to epoch，the training ends，otherwise repeat steps 1）-4）

Testing

Input：Testing set source images $I_{1}$ and $I_{2}$

Output：Fused image $I_{f}$

1）Input $I_{1}$ and $I_{2}$ into the feature extraction module to get the source image features $F_{1}$ and $F_{2}$

2）Input $I_{1}$ and $I_{2}$ into the edge enhancement module to get the source image edge maps $E_{1}$ and $E_{2}$

3）Input $F_{1}$ and $F_{2}$ into the fusion module and the reconstruction module to obtain the intermediate fusion image $I_{f}^{m}$

4）Combine $I_{f}^{m}$ ， $E_{1}$ ，and $E_{2}$ to get the final fusion image $I_{f}$

Table 3. Average value of quality evaluation index of 20 fused images

View table

Table 3. Average value of quality evaluation index of 20 fused images

Method

FMI_pixel

SCD

SSIM

Q_Y

Time /s

FW-Net

CNN

NSCT

NSCT-PRPCNN

Proposed method

0.8182

0.8595

0.8637

0.873

0.8448

0.8741

0.7776

0.8910

1.1123

0.969

1.2690

1.2879

0.5108

0.6612

0.5717

0.6160

0.6350

0.7412

0.0072

0.0167

0.0134

0.0155

0.0122

0.0069

0.6600

0.8407

0.7165

0.7311

0.7575

0.8422

0.0199

0.0912

9.601

3.43

9.601

0.0189

Table 4. Average value of the quality evaluation index of fusion results with four different fusion rules

View table

Table 4. Average value of the quality evaluation index of fusion results with four different fusion rules

Fusion rule

FMI_pixel

SCD

SSIM

Q_Y

Addition^［35］

Average^［36］

Weighted average^［37］

Max value^［34］

0.8728

0.8689

0.8708

0.8741

1.1951

1.1703

1.2456

1.2879

0.7422

0.7270

0.7337

0.7412

0.0071

0.0068

0.0073

0.0069

0.8116

0.8095

0.8119

0.8422

Tools

Get Citation

Copy Citation Text

Wanxin Xiao, Huafeng Li, Yafei Zhang, Minghong Xie, Fan Li. Medical Image Fusion Based on Multi-Scale Feature Learning and Edge Enhancement[J]. Laser & Optoelectronics Progress, 2022, 59(6): 0617029

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Medical Optics and Biotechnology

Received: Oct. 25, 2021

Accepted: Nov. 29, 2021

Published Online: Mar. 8, 2022

The Author Email: Yafei Zhang (zyfeimail@163.com)

DOI:10.3788/LOP202259.0617029

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology