In view of the low detection accuracy caused by color bias in traditional face detection, the deep learning method realizes face detection by training a large amount of data, resulting in high hardware requirements. A simple convolutional neural network is proposed for face and non-face recognition, and then white balance algorithm is used to solve the problem of color cast. Skin-Color detection was realized by combining YCgCr color space with K-means clustering. Finally, face detection is realized on the basis of skin color detection. Its accuracy is about 3% higher than the traditional face detection method, and its speed is about twice faster than the face detection based on deep learning.