Optics and Precision Engineering, Volume. 31, Issue 16, 2430(2023)
Multi-stage frame alignment video super- resolution network
Video-Super Resolution (VSR) aims to reconstruct low-resolution video frame sequences into high-resolution video frame sequences. Compared with single image super-resolution, VSR usually relies on the height-dependent information of neighboring frames to reconstruct the current frame because of the added information of temporal dimension. How to align adjacent frames and obtain highly correlated information between frames is the key issue of VSR task. In this paper, the VSR task is divided into three stages: deblurring, alignment, and reconstruction. In the deblurring stage, the current frame is pre-aligned with adjacent frames to obtain feature information highly related to the current frame, and the details of the current frame are enhanced to achieve more feature information extraction in the initial stage. In the alignment stage, the highly correlated information in adjacent frames is used to further strengthen the feature information in the current frame by performing a secondary alignment operation on the input features. In the reconstruction stage, raw low-resolution frames are aggregated to provide more feature information at the end of the network. In this paper, we use Multi-Layer Perceptron (MLP) instead of the traditional convolution operation to construct a feature extraction module, and also perform a secondary alignment of the generated feature information to refine the image features to obtain better video frame reconstruction results. The experimental results show that the proposed algorithm achieves a higher accuracy of video frame sequence reconstruction on a variety of publicly available datasets while achieving a lower number of network parameters and a more coherent video sequence reconstruction performance.
Get Citation
Copy Citation Text
Sen WANG, Yang ZHU, Yinhui ZHANG, Qingjian WANG, Zifen HE. Multi-stage frame alignment video super- resolution network[J]. Optics and Precision Engineering, 2023, 31(16): 2430
Category: Information Sciences
Received: Dec. 14, 2022
Accepted: --
Published Online: Sep. 5, 2023
The Author Email: ZHANG Yinhui (zhangyinhui@kust.edu.cn)