The method of extracting features of model, which was the core of tracking, was employed to evaluate the fidelity of infrared simulation model. By analyzing the kernel of tracking process, the image of a partial field around the center of target was chosen to evaluate the target model. Based on the existing evaluating methods of the infrared simulation model, a typical object-tracking algorithm was adopted to design the experiment. Following theoretical analysis, the corresponding experiment was then carried out. Result shows that the similar target model is more than 2 times as much as the dissimilar model based on the key features extracting method, and the difference between them can even get to 10 times. Therefore, the key feature extraction algorithm is a practical and feasible method to verify the validity of the target model. This evaluation method can provide theoretical guidance for the evaluation of model fidelity in the field of infrared target detection.