Optics and Precision Engineering, Volume. 32, Issue 2, 252(2024)

Multimodal medical image fusion method based on structural functional cross neural network

Jing DI, Wenqing GUO*, Li REN, Yan YANG, and Jing LIAN
Author Affiliations
  • School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730070, China
  • show less

    To solve the problems of texture detail blurring and low contrast in multimodal medical image fusion, a multimodal medical image fusion method with structural-functional crossed neural networks was proposed. Firstly, this method designed a structural and functional cross neural network model based on the structural and functional information of medical images. Within each structural-functional cross module, a residual network model was also incorporated. This approach not only effectively extracted the structural and functional information from anatomical and physiological medical images but also facilitated interaction between structural and functional information. As a result, it effectively captured texture details from multi-source medical images, creating fused images that closely align with human visual characteristics. Secondly, a new attention mechanism module was constructed by utilizing the effective channel attention mechanism and spatial attention mechanism model (ECA-S), which continuously adjusted the weights of structural and functional information to fuse images, thereby improving the contrast and contour information of the fused image, and to make the fused image color more natural and realistic. Finally, a decomposition process from the fused image to the source image was designed, and since the quality of the decomposed image depends directly on the fusion result, the decomposition process could make the fused image contain more texture detail information and contour information of the source image. By comparing with seven high-level methods for medical image fusion proposed in recent years, the objective evaluation indexes of AG, EN, SF, MI, QAB/F and CC of this paper's method are improved by 22.87%, 19.64%, 23.02%, 12.70%, 6.79% and 30.35% on average, respectively, indicating that this paper's method can obtain fusion results with clearer texture details, higher contrast and better contours in subjective visual and objective indexes are better than other seven high-level contrast methods.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Jing DI, Wenqing GUO, Li REN, Yan YANG, Jing LIAN. Multimodal medical image fusion method based on structural functional cross neural network[J]. Optics and Precision Engineering, 2024, 32(2): 252

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: May. 5, 2023

    Accepted: --

    Published Online: Apr. 2, 2024

    The Author Email: GUO Wenqing (344385945@qq.com)

    DOI:10.37188/OPE.20243202.0252

    Topics