Brain tumor magnetic resonance imaging (MRI) segmentation is an important step in the diagnosis and treatment of brain tumors. In this paper, a brain tumor MRI segmentation algorithm that integrates multi-scale features is proposed to address the low segmentation accuracy caused by the limited receptive field size of the U-Net network structure and the gap in contextual information. Firstly, a multi-scale aggregation module (MAM) is designed to replace the conventional convolutional layers in the original U-Net network. This increases the depth and width of the network to capture detailed boundary information of the feature maps. Secondly, the context atrous spatial pyramid module (CASP) is used in the skip connection instead of direct concatenation operation. This expands the network's receptive field and enhances the extraction ability of lesions at different scale sizes. Finally, a multi-level aggregation attention module (MAA) is designed at the bottom of the U-shaped network. This module enables the network model to focus on effective features in the image segmentation region and exclude background noise. The improved algorithm is experimentally validated on the Cancer Genome Atlas (TCGA) (brain tumor data) database. The results demonstrate that the proposed algorithm achieves the following metrics: mean intersection over union (mIoU) of 91.39%, Dice coefficient of 92.81%, sensitivity of 89.14%, specificity of 99.95%, and accuracy of 95.78%.