To deal with the robust face recognition problem, a mixed model based on Hidden Markov Model (HMM) and Support Vector Machine (SVM) was proposed for HMM has good ability for time sequence modeling and SVM has excellent ability for classifying. Firstly, a sequence of overlapping sub-images was extracted from face image by using Discrete Cosine Transform (DCT) and Singular Value Decomposition (SVD). Then, the sequence which was extracted from training images was modeled by using HMM, and the output probability of each HMM for the training sequence had been considered as the input vector of SVM for its training. Finally, the output probability of each HMM for the testing sequences had been considered as the input vector of SVM for its testing. Experimental results on ORL and Yale face database demonstrate that the effectiveness and robustness of the proposed algorithm are better than traditional signal HMM and SVM algorithm.