To improve the effect and classification accuracy of semantic segmentation of remote sensing images, a two-channel image feature extraction network combining with ResNet18 pre-training model is designed. Images with multiple features are combined, and the combined feature map has stronger ability to express features. At the same time, batch normalization layer and maximum pooling with location index are adopted to optimize the network structure and improve the classification accuracy of surface object. The accuracy and Kappa coefficient of this method are compared with those of other neural network methods by experiments. The results show that the proposed network structure achieves an overall accuracy of 90.68% when the number of data samples is small, and the Kappa coefficient reaches 0.8595. Compared with other methods, the proposed algorithm achieves better semantic segmentation effect, and greatly reduces the overall training time.