To solve the difficult problem of the classification of high-resolution remote sensing images having large intraclass differences and small interclass differences, a hybrid structure using the advantages of convolutional neural networks and a Transformer in deep learning is proposed herein. Feature clustering is carried out for each channel along the horizontal and vertical directions using two attention mechanisms with spatial location information for the features extracted from the convolutional layer. This reduces the redundant mapping of remote sensing scene features and enables the network to extract more information relevant to the task object. Then, the captured feature maps are processed via encoding operations using the Transformer encoder structure to enable the allocation of greater weights to the regions of interest in the feature maps. The experimental results show that the proposed method reduces number of model parameters and increases the classification accuracy compared with the existing deep learning-based remote sensing image classification methods, achieving the highest average classification accuracy of 98.95%, 96.00%, and 95.01% on the remote sensing image classification datasets of AID, NWPU-RESISC45, and VGoogle, respectively.