For the 3D key point pose estimation error caused by the high degree of freedom problem and structural similarity problem of the hand, this paper proposes a novel 3D hand skeleton pose regression framework for joint identification, detection, and pose estimation. The framework firstly adopts a YOLOv3-based detector to obtain the position of hands, then a cascade pose estimation network is designed to get initial hand poses with 2D and 3D pose supervisions. Finally, considering the natural constrains in hand graph connection, we present progressive GCN module to further refine the initial hand pose from coarse to fine. This paper compares PCK metrics and AUC metrics with the state-of-the-art approaches under different public benchmarks, and the proposed method achieves the highest AUC metrics on different test sets, with an average AUC accuracy of 92.9%. The experiments illustrate that the proposed method is able to effectively and robustly predict 3D hand pose from monocular image, performing well in both test sets and in the wild.