Diabetic retinopathy is a serious and common complication of diabetes. Herein, we propose a new feature fusion network model to improve the accuracy of the diagnosis for the severity of diabetic retinopathy and provide a basis for its precise drug treatment. A lightweight network, EfficientNet-B0, was used to extract layer information from fundus images, and high-level elements were combined with three dilated convolutions with various dilation rates to obtain multiscale features. The multiscale channel attention module (MS-CAM) was introduced to weigh high- and bottom-level features, which were then fused to form final feature representations and thereby complete the classification of the diabetic retinopathy. Experimental results show the classification accuracy of the proposed model is 85.25%; hence, the network is appropriate for practical applications. Furthermore, the model can play an auxiliary role for clinical diagnosis and can effectively prevent further deterioration in diabetic retinopathy.