In remote sensing images， roads， trees， and shadows in the background easily interfere with buildings； this usually leads to unclear segmentation boundaries. To address this issue， a Resnet network integrating fractal geometry features is proposed. Based on the coding–decoding framework and considering the Resnet network as a backbone network， the proposed algorithm introduces an atrous spatial pyramid pooling module （FD-ASPP） integrating fractal a priori in the coding stage， which can use the fractal dimension to capture the fractal features of remote sensing images and enhance the geometric feature description ability of the Resnet network. In the decoding stage， a deep separable convolution attention fusion mechanism （DSCAF） is proposed to effectively integrate high-level and low-level features to obtain richer semantic information and location details of remote sensing images. Experiments on the WHU remote sensing image dataset show that the accuracy precision rate is 0.944 8， the recall rate is 0.946 2， the F1 score is 0.945 5， and the average cross merge ratio mIoU is 0.941 5. Compared with existing remote sensing semantic segmentation algorithms for buildings， such as FCN， Segnet， Deeplab V3， U-net， SETR， and AlignSeg， the proposed method achieves better segmentation accuracy； effectively overcomes the interference of roads， trees， shadows and other factors； and obtains a clearer building boundary.