Optics and Precision Engineering, Volume. 32, Issue 2, 252(2024)

Multimodal medical image fusion method based on structural functional cross neural network

Jing DI, Wenqing GUO*, Li REN, Yan YANG, and Jing LIAN
Author Affiliations
  • School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730070, China
  • show less
    References(34)

    [1] LI W S, ZHANG Y, WANG G F et al. DFENet: a dual-branch feature enhanced network integrating transformers and convolutional feature learning for multimodal medical image fusion[J]. Biomedical Signal Processing and Control, 80, 104402(2023).

    [2] LIU R S, LIU J Y, JIANG Z Y et al. A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 30, 1261-1274(2021).

    [3] CHAO Z, DUAN X G, JIA S F et al. Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network[J]. Applied Soft Computing, 118, 108542(2022).

    [4] YU N N, LI J J, HUA Z. Decolorization algorithm based on contrast pyramid transform fusion[J]. Multimedia Tools and Applications, 81, 15017-15039(2022).

    [5] [5] 林剑萍, 廖一鹏. 结合分数阶显著性检测及量子烟花算法的NSST域图像融合[J]. 光学 精密工程, 2021, 29(6): 1406-1419. doi: 10.37188/OPE.20212906.1406LINJ P, LIAOY P. A novel image fusion method with fractional saliency detection and QFWA in NSST[J]. Opt. Precision Eng., 2021, 29(6): 1406-1419.(in Chinese). doi: 10.37188/OPE.20212906.1406

    [6] TAWFIK N, ELNEMR H A, FAKHR M et al. Multimodal medical image fusion using stacked auto-encoder in NSCT domain[J]. Journal of Digital Imaging, 35, 1308-1325(2022).

    [7] [7] 杨艳春, 裴佩佩, 党建武, 等. 基于交替梯度滤波器和改进PCNN的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(9): 1123-1138. doi: 10.37188/OPE.20223009.1123YANGY C, PEIP P, DANGJ W, et al. Infrared and visible image fusion based on alternating gradient filter and improved PCNN[J]. Opt. Precision Eng., 2022, 30(9): 1123-1138.(in Chinese). doi: 10.37188/OPE.20223009.1123

    [8] ECKHORN R, REITBOCK H J, ARNDT M et al[M]. A neural network for feature linking via synchronous activity: results from cat visual cortex and from simulations(1989).

    [9] LARA-HERNANDEZ A, RIENMULLER T, JUAREZ I et al. Deep learning-based image registration in dynamic myocardial perfusion CT imaging[J]. IEEE Transactions on Medical Imaging, 42, 684-696(2023).

    [10] REHMAN M, OBAYYA M et al. Machine learning based skin lesion segmentation method with novel borders and hair removal techniques[J]. PLoS One, 17(2022).

    [11] ZHAO C, WANG T F, LEI B Y. Medical image fusion method based on dense block and deep convolutional generative adversarial network[J]. Neural Computing and Applications, 33, 6595-6610(2021).

    [12] [12] 杨艳春, 高晓宇, 党建武, 等. 基于WEMD和生成对抗网络重建的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(3): 320-330. doi: 10.37188/OPE.20223003.0320YANGY C, GAOX Y, DANGJ W, et al. Infrared and visible image fusion based on WEMD and generative adversarial network reconstruction[J]. Opt. Precision Eng., 2022, 30(3): 320-330.(in Chinese). doi: 10.37188/OPE.20223003.0320

    [13] WANG L F, LIU Y, MI J et al. MSE-Fusion: weakly supervised medical image fusion with modal synthesis and enhancement[J]. Engineering Applications of Artificial Intelligence, 119, 105744(2023).

    [14] [14] 陈永, 张娇娇, 王镇. 多尺度密集连接注意力的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(18): 2253-2266. doi: 10.37188/ope.20223018.2253CHENY, ZHANGJ J, WANGZ. Infrared and visible image fusion based on multi-scale dense attention connection network[J]. Opt. Precision Eng., 2022, 30(18): 2253-2266.(in Chinese). doi: 10.37188/ope.20223018.2253

    [15] FANG M, PENG S Y, LIANG Y J et al. A multimodal fusion model with multi-level attention mechanism for depression detection[J]. Biomedical Signal Processing and Control, 82, 104561(2023).

    [16] TANG W, HE F Z, LIU Y et al. MATR: multimodal medical image fusion via multiscale adaptive transformer[J]. IEEE Transactions on Image Processing, 31, 5134-5149(1003).

    [17] ZHANG H, MA J Y. SDNet: a versatile squeeze-and-decomposition network for real-time image fusion[J]. International Journal of Computer Vision, 129, 2761-2785(2021).

    [18] WANG Q L, WU B G, ZHU P F et al. ECA-net: efficient channel attention for deep convolutional neural networks[C], 11531-11539(2020).

    [19] ZHU X Z, CHENG D Z, ZHANG Z et al. An empirical study of spatial attention mechanisms in deep networks[C], 6687-6696(2019).

    [20] LI S T, YANG B. Multifocus image fusion using region segmentation and spatial frequency[J]. Image and Vision Computing, 26, 971-979(2008).

    [21] ZHAO W D, WANG D, LU H C. Multi-focus image fusion with a natural enhancement via a joint multi-level deeply supervised convolutional neural network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 29, 1102-1115(2019).

    [22] PIELLA G, HEIJMANS H. A new quality metric for image fusion[C](2003).

    [23] DESHMUKH M, BHOSALE U. Image fusion and image quality assessment of fused images[J]. International Journal of Image Processing (IJIP), 4, 484(2010).

    [24] VAN AARDT J. Assessment of image fusion procedures using entropy, image quality, and multispectral classification[J]. Journal of Applied Remote Sensing, 2(2008).

    [25] TAN W, THITØN W, XIANG P et al. Multi-modal brain image fusion based on multi-level edge-preserving filtering[J]. Biomedical Signal Processing and Control, 64, 102280(2021).

    [26] TAN W, TIWARI P, PANDEY H M et al. Multimodal medical image fusion algorithm in the era of big data[J]. Neural Computing and Applications, 1-21(2020).

    [27] ZHANG Y, XIANG W H, ZHANG S L et al. Local extreme map guided multi-modal brain image fusion[J]. Frontiers in Neuroscience, 16, 1055451(2022).

    [28] LI B, PENG H, LUO X H et al. Medical image fusion method based on coupled neural P systems in nonsubsampled shearlet transform domain[J]. International Journal of Neural Systems, 31, 2050050(2021).

    [29] VESHKI F G, VOROBYOV S A. Coupled feature learning via structured convolutional sparse coding for multimodal image fusion[C], 2500-2504(2022).

    [30] LI X X, GUO X P, HAN P F et al. Laplacian redecomposition for multimodal medical image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 69, 6880-6890(2020).

    [31] AGRAWAL C, YADAV S K, SINGH S P et al. A simplified parameter adaptive DCPCNN based medical image fusion[C], 489-501(2022).

    [32] CHEN J, WU K L, CHENG Z et al. A saliency-based multiscale approach for infrared and visible image fusion[J]. Signal Processing, 182, 107936(2021).

    [33] LI H, WU X J, DURRANI T S. Infrared and visible image fusion with ResNet and zero-phase component analysis[J]. Infrared Physics and Technology, 102, 103039(2019).

    [34] VESHKI F G, OUZIR N, VOROBYOV S A et al. Multimodal image fusion via coupled feature learning[J]. Signal Processing, 200, 108637(2022).

    CLP Journals

    [1] Meng WU, Qianwen ZHANG, Zengguo SUN, Jiankai XIANG, Ge GUO. X-ray image fusion for ancient bronze mirror based on intuitionistic fuzzy set entropy measure and salient feature detection[J]. Optics and Precision Engineering, 2025, 33(2): 262

    [2] Meng WU, Qianwen ZHANG, Zengguo SUN, Jiankai XIANG, Ge GUO. X-ray image fusion for ancient bronze mirror based on intuitionistic fuzzy set entropy measure and salient feature detection[J]. Optics and Precision Engineering, 2025, 33(2): 262

    Tools

    Get Citation

    Copy Citation Text

    Jing DI, Wenqing GUO, Li REN, Yan YANG, Jing LIAN. Multimodal medical image fusion method based on structural functional cross neural network[J]. Optics and Precision Engineering, 2024, 32(2): 252

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: May. 5, 2023

    Accepted: --

    Published Online: Apr. 2, 2024

    The Author Email: Wenqing GUO (344385945@qq.com)

    DOI:10.37188/OPE.20243202.0252

    Topics