To achieve a robust and precise face tacking, the color information, gradient direction information and spatial structure information of face are fully exploited. The eyes, nose and mouth patches are employed as tracking regions from human face. The dominant features in these patches are extracted as the basis for tracking. Markov random fields are used to build the spatial constraints between these patches, and a robust tracking algorithm is realized. Experimental results show that, compared with several typical tracking algorithms, the proposed algorithm has well performance in robustness and precision.