Aiming at the problems of traditional remote sensing landslide recognition methods based on convolutional neural network, such as too many model parameters, insufficient attention in areas of interest, and difficulty in capturing long-term (global) dependency, an automatic landslide recognition algorithm of remote sensing images based on improved self-attention and convolution block attention is proposed. The algorithm is based on the encoder-decoder target recognition framework. In order to enhance the model’s attention to local features of landslide areas, the convolution block attention mechanism is applied to the extraction of shallow features, and the landslide target feature association information is obtained from the spatial and channel dimensions. The improved self-attention mechanism is applied to the extraction of deep features, so that the model can capture global feature information within and between feature maps, which effectively distinguishes landslide targets from background areas. Experimental results show that the landslide recognition precision of this method is 96.81%, and the average accuracy of pixel segmentation is 90.11%. The proposed method can effectively improve the accuracy of landslide identification while keeping the model lightweight in comparison with FCN, DeeplabV3+and other algorithms.