Optics and Precision Engineering, Volume. 31, Issue 16, 2430(2023)

Multi-stage frame alignment video super- resolution network

Sen WANG... Yang ZHU, Yinhui ZHANG*, Qingjian WANG and Zifen HE |Show fewer author(s)
Author Affiliations
  • Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China
  • show less

    Video-Super Resolution (VSR) aims to reconstruct low-resolution video frame sequences into high-resolution video frame sequences. Compared with single image super-resolution, VSR usually relies on the height-dependent information of neighboring frames to reconstruct the current frame because of the added information of temporal dimension. How to align adjacent frames and obtain highly correlated information between frames is the key issue of VSR task. In this paper, the VSR task is divided into three stages: deblurring, alignment, and reconstruction. In the deblurring stage, the current frame is pre-aligned with adjacent frames to obtain feature information highly related to the current frame, and the details of the current frame are enhanced to achieve more feature information extraction in the initial stage. In the alignment stage, the highly correlated information in adjacent frames is used to further strengthen the feature information in the current frame by performing a secondary alignment operation on the input features. In the reconstruction stage, raw low-resolution frames are aggregated to provide more feature information at the end of the network. In this paper, we use Multi-Layer Perceptron (MLP) instead of the traditional convolution operation to construct a feature extraction module, and also perform a secondary alignment of the generated feature information to refine the image features to obtain better video frame reconstruction results. The experimental results show that the proposed algorithm achieves a higher accuracy of video frame sequence reconstruction on a variety of publicly available datasets while achieving a lower number of network parameters and a more coherent video sequence reconstruction performance.

    Tools

    Get Citation

    Copy Citation Text

    Sen WANG, Yang ZHU, Yinhui ZHANG, Qingjian WANG, Zifen HE. Multi-stage frame alignment video super- resolution network[J]. Optics and Precision Engineering, 2023, 31(16): 2430

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Information Sciences

    Received: Dec. 14, 2022

    Accepted: --

    Published Online: Sep. 5, 2023

    The Author Email: ZHANG Yinhui (zhangyinhui@kust.edu.cn)

    DOI:10.37188/OPE.20233116.2430

    Topics