In order to meet the needs of building shadow detection in high resolution remote sensing images, we study the shadow detection method based on shadow probability constraint by principal component transformation and spectral feature extraction in hue, saturation, and intensity (HSI) space. Based on the results of principal component transformation and the difference of the spectral characteristics of ground objects in HSI space, we eliminate the influence of dark objects and detect the shadow of buildings in the water using shadow probability. Compared with traditional methods, the proposed method avoids the false detection and missed detection caused by the similar spectral characteristics of water bodies and buildings. Experimental results based on Jilin No.1 images show that the false detection rate and missed detection rate of the proposed method are less than 6%, the overall classification accuracy and Kappa coefficient are higher than 0.9, the impact of water on shadow detection results is reduced, and the overall effect of image shadow detection is improved.