Building height information extraction is one of the hot issues in urban spatial information acquisition by high-resolution satellite remote sensing. An object-oriented building height retrieval method based on the rules of the information is proposed in this work in order to extract building height information accurately. Firstly, the building shadow extraction algorithm based on rule information was established according to the building shadow geometry, texture and spectral characteristics, and building shadow information was extracted. Then the length of the shadow was calculated by computing the average value of pixels, and the building height was inverted by using the spatial model of satellite, the sun, the building and its shadow. Two building height retrieval experiments was carried out based on GF-2 satellite remote sensing data of the two experimental area of Yangzhou (Jiajia garden and Shanwan community). Through the actual height measurement of 105 buildings, the retrieval results are verified and the overall accuracy is 96.5%. It shows that the method has high precision in the aspect of city building height information extraction.