Soil mid-infrared (MIR) can provide a rapid, non-polluting, and cost-efficient method for estimating soil properties, such as soil organic carbon (SOC). Although there is a wide interest in using the soil spectral library (SSL) for soil analysis at various scales, the SSL with a general calibration often produces poor predictions at local scales. Therefore, developing methods to ‘localize’ the spectroscopic modelling is a reliable way to improve the use of SSL. In this study, we proposed a new approach that aims to rapidly build the optimal local model from the SSL by calculating the spectral similarity and developing the local calibration, in order to further improve the prediction accuracy. The distance matrix was constructed by three distance algorithms, namely Euclidean distance, Mahalanobis distance, and Cosine distance, which were compared and used to measure the similarity between the local samples and the SSL. The capacity curve, which was taken from the distance matrix, was used with a method called “continuum-removal” to find the feature points. Partial least-squares regression was used to build the spectroscopic models for SOC estimation. We found that for all three distance algorithms combined with the continuum-removal, the local calibration derived from the first feature point gave us a good idea of how accurate the prediction would be. The Mahalanobis distance can effectively develop the optimal local calibration from the MIR SSL, which not only achieved the best accuracy (R² = 0.764, RMSE = 1.021%) but also used the least number of samples from SSL (14% SSL). On local scales, the approach we proposed can significantly improve both the analytical cost and the accuracy of the soil MIR technique.