Optics and Precision Engineering, Volume. 31, Issue 20, 3050(2023)
CT and PET medical image fusion based on LL-GG-LG Net
Multimodal medical image fusion plays a crucial role in clinical medical applications. Most of the existing methods have focused on local feature extraction, whereas global dependencies have been insufficiently explored; furthermore, interactions between global and local information have not been considered. This has led to difficulties in effectively addressing the complexity of patterns and the similarity between the surrounding tissue (background) and the lesion area (foreground) in terms of intensity. To address such issues, this paper proposes an LL-GG-LG Net model for PET and CT medical image fusion. Firstly, a Local-Local fusion (LL) module is proposed, which uses a two-level attention mechanism to better focus on local detailed information features. Next, a Global-Global fusion (GG) module is designed, which introduces local information into the global information by adding a residual connection mechanism to the Swin Transformer, thereby improving the Transformer's attention to local information. Subsequently, a Local-Global fusion (LG) module is proposed based on a differentiable architecture search adaptive dense fusion network, which fully captures global relationships and retains local cues, thereby effectively solving the problem of high similarity between background and focus areas. The model's effectiveness is validated using a clinical multimodal lung medical image dataset. The experimental results show that, compared to seven other methods, the proposed method objectively improves the perceptual image fusion quality evaluation indexes such as the average gradient (AG), edge intensity (EI), QAB/F, spatial frequency (SF), standard deviation (SD) and information entropy (IE) edge retention by 21.5%, 11%, 4%, 13%, 9%, and 3%, respectively, on average. The model can highlight the information of the lesion areas. Moreover, the fused image structure is clear, and detailed texture information can be obtained.
Get Citation
Copy Citation Text
Tao ZHOU, Xiangxiang ZHANG, Huiling LU, Qi LI, Qianru CHENG. CT and PET medical image fusion based on LL-GG-LG Net[J]. Optics and Precision Engineering, 2023, 31(20): 3050
Category: Information Sciences
Received: Mar. 31, 2023
Accepted: --
Published Online: Nov. 28, 2023
The Author Email: Xiangxiang ZHANG (zxx19990503@163.com)