Traditional semantic segmentation networks often suffer from limited narrow receptive fields and single-branch network architectures, which inadequately capture the spatial relationships, boundaries, and contextual nuances of complex remote sensing scenes. These limitations can result in decreased segmentation accuracy and blurred boundaries. To address these challenges, we propose a multi-level branch cross-scale fusion network (MBCFNet) for remote sensing image semantic segmentation. First, the network employs a multi-level branch structure comprising a shallow Swin Transformer, spatial branch, semantic branch, and boundary branch, where each branch specializes in extracting specific feature levels. A cross-scale fusion module is then incorporated to effectively integrate multi-scale features from each branch, thereby enhancing the model's ability to comprehensively represent remote sensing landforms. Finally, a multi-scale decoding module with an expanded receptive field is introduced to transmit feature information across scales, effectively improving the network's adaptation to complex remote sensing scenes. The proposed MBCFNet achieves mean intersection over union scores of 86.93%, 84.51%, and 74.55% on the Vaihingen, Potsdam, and Uavid datasets, respectively, outperforming advanced semantic segmentation models, such as Mit-B2, ST-UNet, and GLOTS. The experimental results demonstrate the high segmentation accuracy and generalization capability of MBCFNet for remote sensing image semantic segmentation tasks.