Optics and Precision Engineering, Volume. 32, Issue 2, 252(2024)
Multimodal medical image fusion method based on structural functional cross neural network
Figures & Tables(20)
Fig. 7. Comparison of MRI-PET image fusion in "mild Alzheimer's disease".
Fig. 8. Comparison of MRI-SPECT image fusion of "metastatic bronchogenic carcinoma".
Fig. 11. Comparison of infrared and visible image fusion in three different scenes
Fig. 12. Four different network structures for ablation experiments
Fig. 13. Histogram of the mean values of 20 images of four different network structures for the ablation experiment
Table 1. Specific parameters of network model
View tableView in ArticleTable 1. Specific parameters of network model
Module Layer k S P I O Input Conv 5×5 1 1 1 32 Conv1 3×3 1 1 32 32 Conv2 3×3 1 1 32 32 G-R Block Conv3 3×3 1 1 32 32 Conv4 3×3 1 1 32 32 Conv5 3×3 1 1 64 32 Conv1 3×3 1 1 32 32 Conv2 3×3 1 1 32 32 R-G Block Conv3 3×3 1 1 32 32 Conv4 3×3 1 1 32 32 Conv5 3×3 1 1 64 32 Output1 Conv 1×1 1 1 64 1 Conv1 1×1 1 1 33 64 Conv2 3×3 1 1 64 64 Conv3 1×1 1 1 64 16 M-R Blocks Conv4 3×3 1 1 16 16 Conv5 1×1 1 1 16 4 Conv6 3×3 1 1 4 4 Conv7 1×1 1 1 68 4 Conv8 3×3 1 1 4 4 Output2 Conv 3×3 1 1 20 1
Table 2. Objective evaluation indexes of MRI-PET image fusion in "mild Alzheimer's disease"
View tableView in ArticleTable 2. Objective evaluation indexes of MRI-PET image fusion in "mild Alzheimer's disease"
AG EN SF MI QAB/F CC Time MLEPF 8.221 8 4.311 0 40.430 8 2.653 4 0.353 6 0.585 7 7.751 4 NSST 7.962 6 4.205 1 39.369 8 2.514 4 0.367 5 0.276 5 10.595 1 LEGFF 10.986 9 4.793 1 49.882 2 2.303 3 0.573 0 0.284 0 0.572 6 MATR 9.118 2 3.434 9 45.902 4 1.841 9 0.577 8 0.287 0 9.758 7 CNP 10.947 6 3.711 8 51.913 4 2.126 8 0.546 4 0.775 9 6.529 8 SDNet 11.150 9 5.189 2 49.373 4 2.552 6 0.519 7 0.610 4 0.047 0 CFL 11.023 0 3.991 7 52.121 8 2.444 4 0.574 2 0.586 3 34.727 1 Ours 12.763 0 5.590 2 58.207 9 2.683 6 0.580 5 0.783 7 0.116 6
Table 3. Objective evaluation indexes of MRI-SPECT image fusion in "metastatic bronchogenic carcinoma"
View tableView in ArticleTable 3. Objective evaluation indexes of MRI-SPECT image fusion in "metastatic bronchogenic carcinoma"
AG EN SF MI QAB/F CC Time MLEPF 5.692 3 5.058 0 26.656 3 3.457 8 0.506 7 0.860 8 7.553 1 NSST 6.160 9 5.057 0 29.595 9 3.495 7 0.561 6 0.864 8 49.169 4 LEGFF 6.498 0 5.142 9 31.044 2 2.948 8 0.620 4 0.871 9 0.654 8 MATR 5.963 4 4.146 0 30.029 0 2.610 8 0.652 9 0.533 9 0.343 9 CNP 6.616 7 3.975 2 34.040 3 2.599 5 0.604 8 0.635 7 8.965 4 SDNet 7.868 7 5.337 5 36.571 3 2.708 8 0.531 2 0.787 5 0.046 5 CFL 6.492 4 4.442 0 32.997 0 2.948 1 0.621 5 0.840 8 28.659 7 Ours 8.476 5 5.748 2 42.331 2 3.188 5 0.603 5 0.877 5 0.236 5
Table 4. Objective evaluation indexes of "meningioma" MRI-CT image fusion
View tableView in ArticleTable 4. Objective evaluation indexes of "meningioma" MRI-CT image fusion
AG EN SF MI QAB/F CC Time MLEPF 4.009 2 5.124 5 17.820 1 1.531 4 0.265 7 0.659 4 9.632 5 NSST 8.486 1 5.232 2 41.925 7 2.819 9 0.644 0 0.891 0 8.734 9 LEGFF 8.885 1 5.223 5 42.716 3 2.949 1 0.617 1 0.849 0 0.438 3 MATR 8.296 7 4.896 8 40.518 5 3.314 2 0.607 2 0.647 3 0.288 3 CNP 8.950 9 5.019 5 41.972 8 3.329 5 0.625 2 0.707 7 3.587 8 SDNet 8.201 1 4.772 6 38.926 9 3.113 4 0.549 8 0.899 3 0.040 0 CFL 8.320 0 4.545 0 40.693 3 3.038 7 0.580 2 0.799 0 14.614 2 Ours 8.578 4 5.248 0 41.987 2 3.339 9 0.564 5 0.905 2 0.224 0
Table 5. Objective evaluation index of multi-focus image fusion of "people"
View tableView in ArticleTable 5. Objective evaluation index of multi-focus image fusion of "people"
AG EN SF MI QAB/F CC Time LEGFF 5.275 5 7.183 6 17.516 4 6.694 3 0.708 0 0.985 4 0.438 3 CFL 5.064 3 7.216 9 17.219 7 6.093 7 0.697 6 0.987 2 10.555 4 LRD 3.980 7 7.165 4 15.009 1 5.645 2 0.561 2 0.958 0 193.902 3 SDNet 4.763 3 7.445 8 17.838 0 6.822 6 0.633 4 0.985 9 0.040 0 DCPCNN 3.939 3 7.154 7 16.524 9 7.057 9 0.684 4 0.976 4 11.689 1 Ours 5.931 6 7.496 6 17.694 0 7.485 8 0.719 1 0.987 9 0.612 8
Table 6. Objective evaluation index of infrared and visible image fusion in three different scenes
View tableView in ArticleTable 6. Objective evaluation index of infrared and visible image fusion in three different scenes
Image Method AG EN SF MI QAB/F CC Time LEGFF 5.019 1 5.743 1 12.305 5 1.788 4 0.464 5 0.782 5 0.704 7 CFL 1.916 9 5.604 3 5.842 8 2.490 5 0.417 4 0.429 0 15.712 8 helicopter CCF 3.469 7 5.625 7 8.412 1 2.678 2 0.460 5 0.452 7 2.752 6 SMVIF 3.205 5 5.024 2 7.829 2 1.796 4 0.510 1 0.753 5 6.923 1 ResNet 1.792 5 4.859 4 4.502 1 2.114 7 0.397 7 0.783 3 11.689 1 Ours 5.634 8 6.359 3 15.662 6 1.870 9 0.299 6 0.914 4 0.195 6 LEGFF 6.899 5 6.372 1 18.459 5 1.811 3 0.441 0 0.487 9 0.446 8 CFL 6.812 7 4.860 5 15.291 8 2.143 5 0.414 4 0.668 3 17.998 4 Nato camp CCF 5.470 9 6.793 4 16.148 1 2.161 9 0.423 3 0.651 5 31.268 2 SMVIF 6.330 8 5.278 2 15.644 2 1.523 5 0.425 2 0.516 6 6.521 4 ResNet 6.250 5 3.286 8 9.928 9 1.622 6 0.345 1 0.517 4 1.423 6 Ours 6.920 9 6.386 0 18.601 5 2.189 7 0.364 2 0.706 3 0.356 8 LEGFF 7.176 5 6.457 2 17.418 5 1.404 0 0.431 5 0.720 2 0.446 8 CFL 2.946 4 6.720 0 7.824 8 3.324 1 0.389 6 0.759 7 17.998 4 Movie_18 CCF 5.063 2 6.771 4 12.318 6 3.087 2 0.453 4 0.762 4 31.268 2 SMVIF 4.576 1 5.654 1 10.878 4 1.281 4 0.448 8 0.691 8 6.521 4 ResNet 2.585 9 5.524 9 6.315 5 1.546 5 0.340 1 0.727 5 1.423 6 Ours 8.632 5 6.998 1 21.409 5 1.998 2 0.313 3 0.766 4 0.356 8
Tools
Get Citation
Copy Citation Text
Jing DI, Wenqing GUO, Li REN, Yan YANG, Jing LIAN. Multimodal medical image fusion method based on structural functional cross neural network[J]. Optics and Precision Engineering, 2024, 32(2): 252
Paper Information
Category:
Received: May. 5, 2023
Accepted: --
Published Online: Apr. 2, 2024
The Author Email: GUO Wenqing (344385945@qq.com)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20