Aiming at the defects of the traditional image recognition algorithm, which has poor precision and low accuracy in fatigue driving detection, an effective evaluation method of fatigue driving detection using face image data is proposed. Through real-time acquisition of the vehicle drivers face image, the face image was preprocessed first, the face area in the image was detected with the help of Dlib and the feature points of the face image were marked, then the eye-aspect-ratio (EAR)-based method was used to recognize the fatigue feature of the human eyes in the image, the mouth-aspect-ratio (MAR)-based method was used to recognize the fatigue feature of the mouth in the image, and finally the support vector machine (SVM) was applied to combine the two features for fatigue driving detection. Experimental results show that the method can locate the feature points accurately, and the recognition rate of fatigue detection reaches 84.29%, which can effectively identify the fatigue state.