In recent years, deep learning algorithm develops rapidly and is widely used in the task of target detection. However, real-time target detection cannot be carried out on devices with limited memory and computing power. To solve this problem, a fast pedestrian detection method is proposed in the surveillance system with limited memory and processing unit. Firstly, aiming at the problem of low detection efficiency caused by extracting high-dimensional pedestrian features in general pedestrian detection, an improved directional gradient histogram (HOG) and Sobel edge local binary pattern (Sobel LBP) are fused as features. Secondly, a model compression technique based on teacher-student framework is proposed, which is applied to random forest (RF) classifier without deep network, because the compressed deep network still needs a lot of memory to process parameter multiplication. Students' random forest (S-RF) (born again random forest, BARF) is trained to imitate the performance of teachers' random forest by using the soft target of teachers' random forest output. Then the pedestrian detection is carried out by BARF classifier, and finally the pedestrian detection is carried out by sliding window method. In experiments, the proposed method achieved up to a 2.05 times faster speed and a 5.39 times higher compression rate than T-RF and its detection performance is also ideal.