Herein, a V-shaped pyramid bilateral feature fusion network (VPBF-Net) is proposed to address small-scale target missing segmentation, inaccurate edge segmentation, and inefficient fusion of deep and shallow feature information in current semantic segmentation networks. In the encoding stage, a V-shaped atrous spatial pyramid pooling (VASPP) module adopts multiple-parallel-branch interactive connection structures to enhance the information exchange between the local semantic information of each branch. In addition, multibranch feature hierarchical fusion is adopted to reduce grid artifact effects. Furthermore, a coordinate attention module is used to assign weights to the extracted deep semantic information, enhancing the network's attention to the segmentation target. In the decoding stage, a bilateral attention feature aggregation module is designed to guide shallow feature fusion through multiscale deep semantic information, thereby capturing different-scaled shallow feature representations and achieving more efficient deep and shallow feature fusion. Experiments are conducted on the PASCAL VOC 2012 dataset and Cityscapes dataset, the proposed method achieves average intersection to union ratios of 83.25% and 77.21%, respectively, indicating advanced results. Compared with other methods, the proposed method can more accurately perform small-scale object segmentation, alleviating missed segmentation and misclassification.