Terahertz spectrum is sensitive to the change of the nonlocal molecular vibration mode. Accordingly, the spectral waveform is susceptible to variety of physical and chemical factors, which will lead to peak changes, frequency shifts, and even deformation of the overall waveform. Component analysis and material identification from the correspondence between the fixed peak features and materials will prone to cause errors or mistakes. Therefore, to solve this problem, we proposed a method based on Kernel Optimization Relevance Vector Machine (KO-RVM), which extracts global graphic features to distinct from the local features extraction method. And we use Support Vector Regression (SVR) algorithm as comparison. The result shows that, when basis functions’ parameters of RVM are optimized with expectation-maximization algorithm, it will be suitable for feature extraction of terahertz transmission spectrum. The spectrum can be sparsely represented, and the amount of extracted graphic features is substantially reduced. Reconstruction models based on these features are capable of retaining the overall spectral characteristics, and fitting results for each band are more consistent, while the extracted spectrum features can be used as basis of similarity measurement and the common characteristics investigation between different materials.