Objects intruding railway clearance pose great threat to normal railway operations. Identifying intruding pedestrians within the railway clearance limit was of great significance to ensure the safety of railway operations. The existing railway intrusion detection system only detected the intrusion, but did not distinguish whether it was a true alarm of pedestrian intrusion or false alarm caused by light interferences. To reduce false alarms, a training and test set of the alarm image samples were established. A pedestrian classification algorithm based on improved deep convolutional network, trained with combined features of HOG and high-level Alex was then proposed. First, an improved AlexNet deep convolutional neural network model was introduced to extract high-level Alex features by automatic learning; the extracted features were then combined with HOG features to form the combined features of Alex-HOG. Finally, the combined features were used to train the classification network. Experiments on the test set show that the proposed method has a high recognition accuracy of 98.46% in 3.78 s for 1 498 test image samples. The improvements in both accuracy and real-time performance will greatly reduce the false alarm rate of the railway intrusion detection system.