To address the problems of existing medical image segmentation models, such as high computational complexity and large number of parameters, which make it difficult to deploy the models into real-time medical-aided diagnosis systems, and the existing lightweight models, such as the degradation of segmentation performance due to parameter reduction, an improved lightweight U-Net segmentation model is proposed. The model consists of three main components: encoder, decoder and hop connection. First, the encoder uses a multi-scale fusion module formed by the combination of standard convolution and depth-separable convolution, based on which a bottleneck layer structure is introduced to enhance the learning ability of the neural network, and a lightweight cross-level partial network module is designed as a feature extractor for feature extraction of the input image using an aggregation method. Secondly, the lightweight module is continued to be used in the decoder to further optimize the model, reduce the computational complexity of the model and the number of parameters, and produce better segmentation effects. Finally, the fusion of feature information at different resolutions between the encoder and decoder is achieved by means of hopping connections. Experiments were carried out on abdominal organ CHAOS and Chest X-ray data sets. The results showed that the number of parameters and computational complexity of the improved U-Net segmentation model were reduced to varying degrees. When the number of parameters was only 1.28M, the DSC was 87.53% and 95.85%, respectively. The IOU values are 85.25% and 92.21%, respectively, and the segmentation performance is not inferior to that of other networks.