Most of the existing human pose estimation algorithms have designed complex network structures to obtain high accuracy but lead to low speed. The YOLO-Pose algorithm has taken advantages of state-of-the-art object detection algorithm and obtained higher accuracy and speed, but it still has the problems of missed detection and false detection. In this paper, a new lightweight human pose estimation algorithm is proposed according to the characteristics of non-rigidness of human poses and the distribution diversity of human landmarks so as to improve the YOLO-Pose algorithm. Firstly, the lightweight channel and spatial attention network (LCSA-Net) are designed to enhance the feature extraction capability. Secondly, a distance-based adaptive weighting strategy is presented to calculate the regression loss of human landmarks during model training so as to enhance the regression ability of the model to long-distance human landmarks. The experimental results on the COCO 2017 human pose dataset indicate that both of the improved strategies can effectively promote the performance of human pose estimation compared with the baseline model, and achieves improvement of 2% mAP, 1.5% AP50 and 1.7% AR.