Laser & Optoelectronics Progress, Volume. 61, Issue 4, 0437002(2024)

Micro-Video Event Detection Based on Deep Dynamic Semantic Correlation

Peiguang Jing*, Xiaoyi Song, and Yuting Su
Author Affiliations
  • School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
  • show less

    Nowadays, micro-video event detection exhibits great potential for various applications. As for event detection, previous studies usually ignore the importance of keyframes and mostly focus on the exploration of explicit attributes of events. They neglect the exploration of latent semantic representations and their relationships. Aiming at the above problems, a deep dynamic semantic correlation method is proposed for micro-video event detection. First, the frame importance evaluation module is designed to obtain more distinguishing scores of keyframes, in which the joint structure of variational autoencoder and generative adversarial network can strengthen the importance of information to the greatest extent. Then, the intrinsic correlations between keyframes and the corresponding features are cooperated through a keyframe-guided self-attention mechanism. Finally, the hidden event attribute correlation module based on dynamic graph convolution is designed to learn latent semantics and the corresponding correlation patterns of events. The obtained latent semantic-aware representations are used for final micro-video event detection. Experiments performed on the public datasets and the newly constructed micro-video event detection dataset demonstrate the effectiveness of the proposed method.

    Tools

    Get Citation

    Copy Citation Text

    Peiguang Jing, Xiaoyi Song, Yuting Su. Micro-Video Event Detection Based on Deep Dynamic Semantic Correlation[J]. Laser & Optoelectronics Progress, 2024, 61(4): 0437002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Digital Image Processing

    Received: Mar. 30, 2023

    Accepted: Jun. 1, 2023

    Published Online: Feb. 26, 2024

    The Author Email: Jing Peiguang (pgjing@tju.edu.cn)

    DOI:10.3788/LOP230994

    Topics