Optics and Precision Engineering, Volume. 31, Issue 20, 3050(2023)

CT and PET medical image fusion based on LL-GG-LG Net

Tao ZHOU1,2, Xiangxiang ZHANG1,2、*, Huiling LU3, Qi LI1,2, and Qianru CHENG1,2
Author Affiliations
  • 1School of computer science and engineering, North Minzu University, Yinchuan75002, China
  • 2Key Laboratory of Image and Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan75001, China
  • 3School of Medical information engineering, Ningxia Medical University, Yinchuan750004, China
  • show less

    Multimodal medical image fusion plays a crucial role in clinical medical applications. Most of the existing methods have focused on local feature extraction, whereas global dependencies have been insufficiently explored; furthermore, interactions between global and local information have not been considered. This has led to difficulties in effectively addressing the complexity of patterns and the similarity between the surrounding tissue (background) and the lesion area (foreground) in terms of intensity. To address such issues, this paper proposes an LL-GG-LG Net model for PET and CT medical image fusion. Firstly, a Local-Local fusion (LL) module is proposed, which uses a two-level attention mechanism to better focus on local detailed information features. Next, a Global-Global fusion (GG) module is designed, which introduces local information into the global information by adding a residual connection mechanism to the Swin Transformer, thereby improving the Transformer's attention to local information. Subsequently, a Local-Global fusion (LG) module is proposed based on a differentiable architecture search adaptive dense fusion network, which fully captures global relationships and retains local cues, thereby effectively solving the problem of high similarity between background and focus areas. The model's effectiveness is validated using a clinical multimodal lung medical image dataset. The experimental results show that, compared to seven other methods, the proposed method objectively improves the perceptual image fusion quality evaluation indexes such as the average gradient (AG), edge intensity (EI), QAB/F, spatial frequency (SF), standard deviation (SD) and information entropy (IE) edge retention by 21.5%, 11%, 4%, 13%, 9%, and 3%, respectively, on average. The model can highlight the information of the lesion areas. Moreover, the fused image structure is clear, and detailed texture information can be obtained.

    Tools

    Get Citation

    Copy Citation Text

    Tao ZHOU, Xiangxiang ZHANG, Huiling LU, Qi LI, Qianru CHENG. CT and PET medical image fusion based on LL-GG-LG Net[J]. Optics and Precision Engineering, 2023, 31(20): 3050

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Information Sciences

    Received: Mar. 31, 2023

    Accepted: --

    Published Online: Nov. 28, 2023

    The Author Email: Xiangxiang ZHANG (zxx19990503@163.com)

    DOI:10.37188/OPE.20233120.3050

    Topics