Deep learning-based algorithms of land cover classification for multispectral satellite remote sensing imagery classification typically utilize RGB band data while overlooking other bands such as NIR, and the networks’ feature extraction and application expansion capabilities need improvement. Regarding this issue, this paper proposes a land cover classification method multispectral satellite remote sensing imagery classification method based on the improved YOLOv5, called VN-YOLOv5-Seg. This method jointly utilizes RGB and NIR band data as inputs, adopts YOLOv5 object detection network as the backbone network, and employs the ProtoNet network as the segmentation head to convert object detection into pixel-level land cover classification tasks. The GID-15 dataset is used for experiments to verify the effectiveness of this method, with RGB band and RGB+NIR band as network inputs. Comparative analyses are conducted between VN-YOLOv5-Seg and other land cover classification networks. Experimental results demonstrate that by adding the NIR band to the RGB band, the mean Intersection over Union (mIoU) is improved by 2.5%. Compared to the FCN segmentation head, the mIoU is improved by 8.1%. Compared to PSPNet, DeepLabV3, and U-Net methods, the mIoU is increased by 2.6%, 1.2%, and 1.4% respectively. These results fully validate the effectiveness of the method and the necessity of introducing more band information for land cover classification.