In view of the complex and changeable morphological structure and scale information of retinal vessels, an U-shaped retinal vessel segmentation algorithm based on the adaptive morphological structure and scale information is proposed. First, the gray image of retina is obtained by synthetically analyzing the three-channel frequency information of the image with two-dimensional K-L (Karhunen-Loeve) transform, and the contrast information between the vessel and the background is enhanced by multi-scale morphological filtering. Then the preprocessed image is trained end-to-end by using the U-shaped segmentation model, and the data is enhanced by local information entropy sampling. The dense deformable convolution structure of the network coding part captures the multi-scale information and shape structure of the image effectively according to the informations of the upper and lower feature layers, and the pyramid-shaped multi-scale dilated convolution at the bottom enlarges the local receptive field. At the same time, introducing deconvolution layer with attention mechanism in decoding phase, which effectively combines the bottom and top feature mappings, can solve the problems of weight dispersion and image texture loss. Finally, the final segmentation result is obtained by using the SoftMax activation function. This approach achieves average accuracies of 97.48% and 96.83% and specificities of 98.83% and 97.75% on the DRIVE (Digital Retinal Images for Vessel Extraction) and STARE (Structured Analysis of the Retina) datasets respectively, which is better than the existing algorithms.