In order to solve the problem that convolutional neural network detection of pedestrians was slow, and did not meet the real-time requirement when performing continuous pedestrian detection, pedestrian detection algorithm of history-based region with convolutional neural network was used. Current image was detected by using the detection result in the previous image. The detection process was optimized, and the detection result of the previous image was used as reference information for extracting region proposals of the current image. Convolution feature of the current image was filtered by using the gray value difference map of the current image and the previous image to reduce the sliding window searching area. The results of Caltech pedestrian detection data set show that the algorithm combined with historical information is 2.5 times faster than the advanced algorithm, and the detection accuracy is increased by 1.5%. The algorithm implements real-time pedestrian detection, and the designed network can effectively detect small target pedestrians.