Optics and Precision Engineering, Volume. 32, Issue 4, 549(2024)

Using image smoothing structure information to guide image inpainting

Jiajun ZHANG1... Jing LIAN1,2,*, Jizhao LIU2, Zilong DONG1 and Huaikun ZHANG2 |Show fewer author(s)
Author Affiliations
  • 1School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730000, China
  • 2School of Information Science and Engineering, Lanzhou University, Lanzhou730000, China
  • show less

    Using image structure features for image inpainting is a new method that has emerged in recent years with the widespread application of deep learning techniques. This method can generate plausible content within missing areas, but the restoration results heavily rely on the extracted content of image structures. In practical training, errors can propagate and accumulate, directly impacting the quality of the generated image when there is noise or distortion in the image structure. This method is still in the exploratory phase and faces challenges such as difficulty in network training, poor robustness, and inconsistent semantic context in generated images.To address these issues, this paper proposed a parallel network structure for image inpainting guided by smooth image structures. The generated content of the smooth image structure was not directly used as input for the next-level network but served as guidance information for the decoding layer of network. Additionally, to better match and balance the feature relationship between the structure and the image, this paper combined transformer and introduces a multi-scale feature guidence module. This module utilized the powerful modeling capability of transformers to establish connections between global features, matching and balancing features between structure and image textures.Experimental results demonstrate that the proposed method effectively restores missing content in images on three commonly used datasets and can be used as an image editing tool for object removal.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Jiajun ZHANG, Jing LIAN, Jizhao LIU, Zilong DONG, Huaikun ZHANG. Using image smoothing structure information to guide image inpainting[J]. Optics and Precision Engineering, 2024, 32(4): 549

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 19, 2023

    Accepted: --

    Published Online: Apr. 2, 2024

    The Author Email: LIAN Jing (lian322scc@163.com)

    DOI:10.37188/OPE.20243204.0549

    Topics