Optics and Precision Engineering, Volume. 32, Issue 2, 252(2024)

Multimodal medical image fusion method based on structural functional cross neural network

Jing DI, Wenqing GUO*, Li REN, Yan YANG, and Jing LIAN
Author Affiliations
  • School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730070, China
  • show less
    References(34)

    [1] W S LI, Y ZHANG, G F WANG et al. DFENet: a dual-branch feature enhanced network integrating transformers and convolutional feature learning for multimodal medical image fusion. Biomedical Signal Processing and Control, 80, 104402(2023).

    [2] R S LIU, J Y LIU, Z Y JIANG et al. A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 30, 1261-1274(2021).

    [3] Z CHAO, X G DUAN, S F JIA et al. Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network. Applied Soft Computing, 118, 108542(2022).

    [4] N N YU, J J LI, Z HUA. Decolorization algorithm based on contrast pyramid transform fusion. Multimedia Tools and Applications, 81, 15017-15039(2022).

    [5] [5] 林剑萍, 廖一鹏. 结合分数阶显著性检测及量子烟花算法的NSST域图像融合[J]. 光学 精密工程, 2021, 29(6): 1406-1419. doi: 10.37188/OPE.20212906.1406LINJ P, LIAOY P. A novel image fusion method with fractional saliency detection and QFWA in NSST[J]. Opt. Precision Eng., 2021, 29(6): 1406-1419.(in Chinese). doi: 10.37188/OPE.20212906.1406

    [6] N TAWFIK, H A ELNEMR, M FAKHR et al. Multimodal medical image fusion using stacked auto-encoder in NSCT domain. Journal of Digital Imaging, 35, 1308-1325(2022).

    [7] [7] 杨艳春, 裴佩佩, 党建武, 等. 基于交替梯度滤波器和改进PCNN的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(9): 1123-1138. doi: 10.37188/OPE.20223009.1123YANGY C, PEIP P, DANGJ W, et al. Infrared and visible image fusion based on alternating gradient filter and improved PCNN[J]. Opt. Precision Eng., 2022, 30(9): 1123-1138.(in Chinese). doi: 10.37188/OPE.20223009.1123

    [8] R ECKHORN, H J REITBOCK, M ARNDT et al. A neural network for feature linking via synchronous activity: results from cat visual cortex and from simulations(1989).

    [9] A LARA-HERNANDEZ, T RIENMULLER, I JUAREZ et al. Deep learning-based image registration in dynamic myocardial perfusion CT imaging. IEEE Transactions on Medical Imaging, 42, 684-696(2023).

    [10] M REHMAN, M OBAYYA et al. Machine learning based skin lesion segmentation method with novel borders and hair removal techniques. PLoS One, 17(2022).

    [11] C ZHAO, T F WANG, B Y LEI. Medical image fusion method based on dense block and deep convolutional generative adversarial network. Neural Computing and Applications, 33, 6595-6610(2021).

    [12] [12] 杨艳春, 高晓宇, 党建武, 等. 基于WEMD和生成对抗网络重建的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(3): 320-330. doi: 10.37188/OPE.20223003.0320YANGY C, GAOX Y, DANGJ W, et al. Infrared and visible image fusion based on WEMD and generative adversarial network reconstruction[J]. Opt. Precision Eng., 2022, 30(3): 320-330.(in Chinese). doi: 10.37188/OPE.20223003.0320

    [13] L F WANG, Y LIU, J MI et al. MSE-Fusion: weakly supervised medical image fusion with modal synthesis and enhancement. Engineering Applications of Artificial Intelligence, 119, 105744(2023).

    [14] [14] 陈永, 张娇娇, 王镇. 多尺度密集连接注意力的红外与可见光图像融合[J]. 光学 精密工程, 2022, 30(18): 2253-2266. doi: 10.37188/ope.20223018.2253CHENY, ZHANGJ J, WANGZ. Infrared and visible image fusion based on multi-scale dense attention connection network[J]. Opt. Precision Eng., 2022, 30(18): 2253-2266.(in Chinese). doi: 10.37188/ope.20223018.2253

    [15] M FANG, S Y PENG, Y J LIANG et al. A multimodal fusion model with multi-level attention mechanism for depression detection. Biomedical Signal Processing and Control, 82, 104561(2023).

    [16] W TANG, F Z HE, Y LIU et al. MATR: multimodal medical image fusion via multiscale adaptive transformer. IEEE Transactions on Image Processing, 31, 5134-5149(1003).

    [17] H ZHANG, J Y MA. SDNet: a versatile squeeze-and-decomposition network for real-time image fusion. International Journal of Computer Vision, 129, 2761-2785(2021).

    [18] Q L WANG, B G WU, P F ZHU et al. ECA-net: efficient channel attention for deep convolutional neural networks, 11531-11539(2020).

    [19] X Z ZHU, D Z CHENG, Z ZHANG et al. An empirical study of spatial attention mechanisms in deep networks, 6687-6696(2019).

    [20] S T LI, B YANG. Multifocus image fusion using region segmentation and spatial frequency. Image and Vision Computing, 26, 971-979(2008).

    [21] W D ZHAO, D WANG, H C LU. Multi-focus image fusion with a natural enhancement via a joint multi-level deeply supervised convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 29, 1102-1115(2019).

    [22] G PIELLA, H HEIJMANS. A new quality metric for image fusion(2003).

    [23] M DESHMUKH, U BHOSALE. Image fusion and image quality assessment of fused images. International Journal of Image Processing (IJIP), 4, 484(2010).

    [24] J VAN AARDT. Assessment of image fusion procedures using entropy, image quality, and multispectral classification. Journal of Applied Remote Sensing, 2(2008).

    [25] W TAN, W THITØN, P XIANG et al. Multi-modal brain image fusion based on multi-level edge-preserving filtering. Biomedical Signal Processing and Control, 64, 102280(2021).

    [26] W TAN, P TIWARI, H M PANDEY et al. Multimodal medical image fusion algorithm in the era of big data. Neural Computing and Applications, 1-21(2020).

    [27] Y ZHANG, W H XIANG, S L ZHANG et al. Local extreme map guided multi-modal brain image fusion. Frontiers in Neuroscience, 16, 1055451(2022).

    [28] B LI, H PENG, X H LUO et al. Medical image fusion method based on coupled neural P systems in nonsubsampled shearlet transform domain. International Journal of Neural Systems, 31, 2050050(2021).

    [29] F G VESHKI, S A VOROBYOV. Coupled feature learning via structured convolutional sparse coding for multimodal image fusion, 2500-2504(2022).

    [30] X X LI, X P GUO, P F HAN et al. Laplacian redecomposition for multimodal medical image fusion. IEEE Transactions on Instrumentation and Measurement, 69, 6880-6890(2020).

    [31] C AGRAWAL, S K YADAV, S P SINGH et al. A simplified parameter adaptive DCPCNN based medical image fusion, 489-501(2022).

    [32] J CHEN, K L WU, Z CHENG et al. A saliency-based multiscale approach for infrared and visible image fusion. Signal Processing, 182, 107936(2021).

    [33] H LI, X J WU, T S DURRANI. Infrared and visible image fusion with ResNet and zero-phase component analysis. Infrared Physics and Technology, 102, 103039(2019).

    [34] F G VESHKI, N OUZIR, S A VOROBYOV et al. Multimodal image fusion via coupled feature learning. Signal Processing, 200, 108637(2022).

    Tools

    Get Citation

    Copy Citation Text

    Jing DI, Wenqing GUO, Li REN, Yan YANG, Jing LIAN. Multimodal medical image fusion method based on structural functional cross neural network[J]. Optics and Precision Engineering, 2024, 32(2): 252

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: May. 5, 2023

    Accepted: --

    Published Online: Apr. 2, 2024

    The Author Email: GUO Wenqing (344385945@qq.com)

    DOI:10.37188/OPE.20243202.0252

    Topics