A model with radial basis functions based on surface slope (RBF-BS) has advantages in terms of its characterizations and optimal system design. However, using this model can lead to some problems such as low optimization efficiency and difficulties in conducting tolerance analysis owing to the use of too many basis functions in RBF-BS. Thus, a method of achieving the optimal design and performing tolerance analysis with this model was proposed. This study combined global and local optimizations to develop a new optimization design method based on the local characteristics of the RBF-BS model. The linear relationship between the coefficient of basis function and freeform surface sag was determined by the method of mathematical statistics, and the initial range of tolerance of coefficients was directly determined. The reasonable tolerance value of freeform surface sag was acquired by large-sample statistics. When the proposed method of achieving the optimized design and performing tolerance analysis was applied to a head-mounted display system with a freeform surface using the RBF-BS model, the results indicate that the average modulation transfer function in the entire field view exceeds 0.3, and the largest distortion is 3.45%, which are consistent with the system design target. Based on the results of tolerance analysis, the experimental system integration successfully achieves the display of picture. The proposed method of achieving the optimized design and performing tolerance analysis provides valuable information about RBF-BS and other local freeform surface models.