A global and local deep feature based (GLDFB) bag-of-visual-words (BoVW) model is proposed. The high-level features extracted from the deep convolutional neural network are reorganized and encoded by the BoVW model and the fusion features are classified by the support vector machine. The features from the convolutional layer containing the local details and the fully-connected layer containing the global information of scenes are fully used and thus the efficient expressions of the remote sensing image scenes are formed. The experimental results on two remote sensing image scene datasets with different scales show that, compared with the existing methods, the proposed method possesses unique advantages in the representation ability and the classification accuracy of high-level features.