Optics and Precision Engineering, Volume. 32, Issue 16, 2564(2024)

Lightweight video super-resolution based on hybrid spatio-temporal convolution

Zhenping XIA1...3,*, Hao CHEN1, Yuning ZHANG2,4, Cheng CHENG1,3, and Fuyuan HU13 |Show fewer author(s)
Author Affiliations
  • 1School of Electronic & Information Engineering, Suzhou University of Science and Technology, Suzhou25009, China
  • 2Display R&D Centre, School of Electronic Science & Engineering, Southeast University, Nanjing10096, China
  • 3Jiangsu Industrial Intelligent and Low-carbon Technology Engineering Center, Suzhou215009, China
  • 4Shi-Cheng Laboratory for Information Display and Visualization, Nanjing210013, China
  • show less

    Addressing the issue of high computational complexity and limited extraction of spatio-temporal features in 3D convolutional neural networks for video super-resolution tasks, this paper introduced a novel lightweight video super-resolution reconstruction network based on hybrid spatio-temporal convolution. Firstly, a hybrid spatio-temporal convolution-based module was proposed to realize the enhancement of the spatio-temporal feature extraction capability of the network as well as reduction of the computational complexity. Then, a similarity-based selective feature fusion module was proposed to further enhance the extraction capability of relevant features. Lastly, a motion compensation module based on the attention mechanism was designed to mitigate the effects of erroneous feature fusion to a certain extent. The experimental results demonstrate that the proposed network can achieve a favorable balance between video super-resolution performance and network complexity, and the 4-fold super-resolution reaches 8 frames per second on the benchmark dataset SPMCS-11. The proposed network meets the requirements for fast and accurate reasoning operations on edge devices.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Zhenping XIA, Hao CHEN, Yuning ZHANG, Cheng CHENG, Fuyuan HU. Lightweight video super-resolution based on hybrid spatio-temporal convolution[J]. Optics and Precision Engineering, 2024, 32(16): 2564

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Mar. 28, 2024

    Accepted: --

    Published Online: Nov. 18, 2024

    The Author Email: XIA Zhenping (xzp@usts.edu.cn)

    DOI:10.37188/OPE.20243216.2564

    Topics