In dense scenes, occlusion of the pedestrian body and varying pedestrian scales are the main reasons for the degradation of pedestrian detection accuracy. Since the head region of pedestrians is less occluded, it can be used to assist detection. In this regard, this paper improves the Faster R-CNN algorithm and proposes a multi-scale pedestrian detection algorithm based on the joint head and overall information. First, a recursive multi-scale feature extraction network incorporating the coordinate attention mechanism is designed for obtaining rich and detailed multi-scale feature information and enhancing the sensitivity of the network to small-scale pedestrian locations. Then, the region proposal network is utilized to simultaneously generate pedestrian head and body proposals, and a head and body detection branch is constructed for joint detection. Finally, an adaptive joint non-maximum suppression algorithm is proposed so that the detection boxes with severe overlap are not over-suppressed and the false detection boxes generated by the two detection branches are screened out simultaneously to further enhance the accuracy of pedestrian detection. The experimental results show that compared with the baseline algorithm, the proposed algorithm improves the average precision by 2.9% on the CrowdHuman dataset and reduces the log-average miss detection rate by 4%, and reduces the log-average miss detection rate by 2.4% and 2.2% on the two small-scale subsets of the TJU-DHD-pedestrian dataset, which verifies the effectiveness of the proposed algorithm in the dense scene.