To further enhance the segmentation accuracy of deep learning semantic segmentation method on complex street images, this paper proposes a semantic segmentation network (MDFNet) incorporating multi-dimensional features based on PointRend network of street image. Firstly, the algorithm builds a target area enhancement module to optimize the feature extraction sub-network, which self-adaptively refines the intermediate feature map in each convolutional block of the deep network. Thus, the module enhances the fine extraction of multi-dimensional feature information of complex street images. Secondly, the paper introduces feature pyramid grid during feature fusion. The module uses different convolutional kernels to process street images of different scales. Thus, it obtains more comprehensively the different resolution features of various targets in complex street images. Finally, we use the double decoder to recover the details of the image in more detail to obtain the pixel-by-pixel classification results. The experimental results show that the network in this paper has higher segmentation accuracy on the Cityscapes dataset compared with other excellent networks such as DeepLabV3 and SegFormer. The mean intersection over union reaches 80.11% and an improvement of more than 3.51% compared to other networks. The method provides better understanding of images of complex street scenes.