Chinese Journal of Liquid Crystals and Displays, Volume. 37, Issue 1, 86(2022)

Video inpainting based on residual convolution attention network

LI De-cai1、*, YAN Qun1,2, YAO Jian-min1,2, LIN Zhi-xian1, and DONG Ze-yu1
Author Affiliations
  • 1[in Chinese]
  • 2[in Chinese]
  • show less

    Video inpainting, which aims at filling in missing regions of a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In order to solve the problems of discontinuous semantic information, video blurriness and temporal artifact, and more and more complex network design, the overall speed of the network becoming slow, this paper proposes a residual convolution attention network (RCAN) for video inpainting. By introducing the self-attention mechanism and the global attention mechanism into the residual network, the ability of the network to learn the spatio-temporal features of all input frames is enhanced. This method proposes a spatial-temporal adversarial loss function to optimize RCAN, which improves the quality of video inpainting. At the same time, the network can define the number of layers and parameters with a high degree of freedom to improve the practical application ability of the network. Experimental results show that the network can achieve an average inpainting result in that the PSNR is 3068 dB, the SSIM is 0.961, and the FID is 0.113 on DAVIS and YouTube-VOS data sets. This method meets the inpainting quality requirements of the actual scene on the model and provides a new idea for video inpainting.

    Tools

    Get Citation

    Copy Citation Text

    LI De-cai, YAN Qun, YAO Jian-min, LIN Zhi-xian, DONG Ze-yu. Video inpainting based on residual convolution attention network[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(1): 86

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 24, 2021

    Accepted: --

    Published Online: Mar. 1, 2022

    The Author Email: LI De-cai (n191127093@fzu.edu.cn)

    DOI:10.37188/cjlcd.2021-0196

    Topics