Soil organic matter (SOM) content is a key index of soil quality and plays an important role in the global carbon cycle system. Rapid and accurate estimation and spatial mapping of SOM content are significant for soil carbon pool estimation, crop growth monitoring, cultivated land planning, and management. It is time-consuming and difficult to use traditional methods to monitor regional SOM content, and it is a reasonable and effective method to establish an SOM estimation model based on hyperspectral remote sensing images. However, the SOM content estimation model for hyperspectral remote sensing images has some problems, such as spectral data redundancy, low feature extraction accuracy, and weak generalization ability of a small sample model. In this paper, a total of 67 soil samples were collected in Huangzhong County, Qinghai Province. The ZY1-02D hyperspectral remote sensing image was obtained and preprocessed to obtain pixel spectral data of the sample points. The fractional-order differential transform (FOD) method explored the sensitive bands with a response relationship with SOM content. With 0.2 as a step, the correlation threshold method was used to compare and analyze different order differential processing data mining capabilities. The stable competitive adaptive reweighted sampling algorithm (sCARS) removes hyperspectral redundant data to obtain the modeling feature bands. Random forest (RF), extreme gradient lifting tree, extreme learning machine, and ridge regression machine learning are selected as modeling algorithms. The SOM estimation model is constructed using the spectral data of the full band and the characteristic band as input variables. The results show that the FOD transform can greatly improve the correlation between the band and the SOM content compared with the integer order, and more subtle spectral bands with a response relationship with SOM content can be mined. The 0.8th-order differential transform has the best effect, and the maximum correlation coefficient is increased by 0.546. Compared with full-band spectral data, the estimation accuracy of the model constructed with the sCARS feature extraction method is greatly improved, indicating that sCARS can effectively improve the quality of modeling data and the model's prediction accuracy. In the modeling algorithm, RF performance is the best, with $R_p^2$ (determination coefficient) reaching 0.766 and RPD reaching 1.86, which is about 7.58% higher than the $R_p^2$ of the full-band modeling result. Regional SOM content estimation mapping was realized based on FOD-sCARS and RF. This study further verifies that space-borne hyperspectral remote sensing images are a reliable way to achieve regional SOM estimation mapping, and the research results can provide a new idea for estimating regional SOM content and provide data support for mapping spatial distribution map of SOM content using space-borne hyperspectral remote sensing images.