The traditional method based on the histogram of oriented gradients and Support Vector Machine causes large amount of computation. To deal with the problem, a novel method called the contour feature of head-shoulders combined with neural network is proposed. The head-shoulder model is relatively stable and the contour feature can be used as a basis for human identification. There are two main parts in the paper. Firstly, the head-shoulder model was extracted by edge detection and mean shift algorithm. Then Fourier descriptors with PCA dimensionality reduction were calculated according to contours of the head-shoulder model. Combined with neural network classifier, the initial human identification was completed. Secondly, several models of human head-shoulders from aim pictures which have been identified as non-person with RGB hair mode and the mean-shift algorithm were clustered and classified them again. The experiment result shows that, the detection accuracy and speed are improved compared with the conventional algorithms,and it performances well when shelters occur.