Chinese Journal of Liquid Crystals and Displays, Volume. 40, Issue 6, 867(2025)
3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception
Existing 3D hand mesh reconstruction methods face multiple challenges, especially when dealing with occlusion and highly flexible hand poses, which lead to issues such as missing geometric information and topological errors. To enable accurate and efficient 3D hand reconstruction under occlusion, this paper proposes a two-stage network framework for real-time and efficient reconstruction of hand 3D meshes from monocular RGB images. In the first stage, the adaptive occlusion recovery module is designed by introducing learnable attention weight masks and region consistency loss within the attention mechanism. This module targets occluded regions and adaptively recovers information, significantly enhancing feature representation ability under occlusion. In the second stage, the paper combines static topology modeling and dynamic pose perception, as well as feature information exchange between bidirectional graph convolutions and a novel joint rotation-aware attention. This results in the topology-pose bidirectional perception module, which achieves complementary enhancement of static and dynamic features, improving the ability to capture fine-grained joint details. The proposed method is evaluated through qualitative and quantitative experiments on the FreiHAND and InterHand2.6M datasets, compared with state-of-the-art methods. Experimental results show that on the FreiHAND dataset, the proposed method achieves the PA-MPVPE reduction to 6.1 mm with an inference speed of 39 FPS, and on the InterHand2.6M dataset, the MPJPE of the proposed method is reduced to 8.07 mm, and the MPVPE is reduced to 8.22 mm. The proposed approach meets the requirements for robust occlusion handling, real-time performance, and accurate pose estimation in 3D hand reconstruction.
Get Citation
Copy Citation Text
Jia LIU, Nanxuan HUANG, Dapeng CHEN, Lina WEI. 3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception[J]. Chinese Journal of Liquid Crystals and Displays, 2025, 40(6): 867
Category:
Received: Dec. 9, 2024
Accepted: --
Published Online: Jul. 14, 2025
The Author Email: Dapeng CHEN (dpchen@nuist.edu.cn)