In order to achieve accurate extraction of buildings from high-resolution remote sensing images, an automated extraction method is proposed within the DeeplabV3+ framework. Firstly, SENet154 with SE module is selected as the backbone network to enhance the model's ability to extract image feature information. After the SENet154 network, the image can obtain 6 different feature maps. Then, the proposed Multi Feature Fusion Network (MFF) is used to fuse 5 low scale feature data. This process can fully utilize the advantages of different scale features in local detail and semantic information representation, achieve the comprehensive use of high and low scale features, and improve the segmentation accuracy of the model. Then, the highest scale and highest dimension feature maps are input into the ASPP module, and dilated convolution is used to expand the receptive field and enhance semantic features in depth. Finally, in the Decoder part, the fused features are combined with the multi-scale semantic information obtained from the ASPP module to obtain fine-grained extraction results of buildings. The proposed method is evaluated on publicly available high-resolution remote sensing building datasets through experiments on the effectiveness of the SE module, the combination of different backbone networks and multi-feature fusion, and comparisons with various commonly used methods. The method achieves precision, recall, and F1 scores of over 94% and an IoU score close to 90%. When using the proposed method, the accuracy and robustness of building extraction are significantly improved, and the performance is more outstanding.