Chinese Journal of Liquid Crystals and Displays, Volume. 40, Issue 6, 867(2025)

3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception

Jia LIU1, Nanxuan HUANG1, Dapeng CHEN1、*, and Lina WEI2
Author Affiliations
  • 1School of Automation, Nanjing University of Information Science & Technology, Nanjing 210044, China
  • 2School of Computer and Computing Science, Hangzhou City University, Hangzhou 310015, China
  • show less

    Existing 3D hand mesh reconstruction methods face multiple challenges, especially when dealing with occlusion and highly flexible hand poses, which lead to issues such as missing geometric information and topological errors. To enable accurate and efficient 3D hand reconstruction under occlusion, this paper proposes a two-stage network framework for real-time and efficient reconstruction of hand 3D meshes from monocular RGB images. In the first stage, the adaptive occlusion recovery module is designed by introducing learnable attention weight masks and region consistency loss within the attention mechanism. This module targets occluded regions and adaptively recovers information, significantly enhancing feature representation ability under occlusion. In the second stage, the paper combines static topology modeling and dynamic pose perception, as well as feature information exchange between bidirectional graph convolutions and a novel joint rotation-aware attention. This results in the topology-pose bidirectional perception module, which achieves complementary enhancement of static and dynamic features, improving the ability to capture fine-grained joint details. The proposed method is evaluated through qualitative and quantitative experiments on the FreiHAND and InterHand2.6M datasets, compared with state-of-the-art methods. Experimental results show that on the FreiHAND dataset, the proposed method achieves the PA-MPVPE reduction to 6.1 mm with an inference speed of 39 FPS, and on the InterHand2.6M dataset, the MPJPE of the proposed method is reduced to 8.07 mm, and the MPVPE is reduced to 8.22 mm. The proposed approach meets the requirements for robust occlusion handling, real-time performance, and accurate pose estimation in 3D hand reconstruction.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Jia LIU, Nanxuan HUANG, Dapeng CHEN, Lina WEI. 3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception[J]. Chinese Journal of Liquid Crystals and Displays, 2025, 40(6): 867

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Dec. 9, 2024

    Accepted: --

    Published Online: Jul. 14, 2025

    The Author Email: Dapeng CHEN (dpchen@nuist.edu.cn)

    DOI:10.37188/CJLCD.2024-0345

    Topics