The rural buildings are the most important disaster recipient in the earthquake disaster， it has significant meaning in the fields of earthquake resistance and hazardous reduction to obtain the information like the type and distribution of it. Based on GF-2 high-resolution remote sensing data， the ESP （Estimation of Scale Parameter） algorithm and Seath（Seperability and thresholds） algorithm are used to determine the optimal image segmentation scale and construct the optimal feature learning space. The decision tree classification method and random forest machine learning classification method were chosen to extract and classify rural building structures in Xiangnan Town， Gansu Province，in early May 2021. Unmanned aerial survey and on-site investigation data were used to verify and refine the accuracy of the classification results. The results show that： ①Both methods can better identify brick-concrete buildings with uniform spatial distribution， large area and bright color， but for civil buildings with chaotic distribution and relatively concentrated， gray color and small area （brick-wood buildings） are difficult to effectively identify their boundary contours and accurately classify them. ② The accuracy and Kappa coefficient of the two methods for building classification in the study area are 84.42%， 86.82% and 0.701 5， 0.759 1， respectively， and the random forest-based method has less misclassification and missing phenomenon when extracting building information. Therefore， the random forest method is more suitable for rural building classification.