At present, the retinal optical coherence tomography (OCT) image classification method based on convolutional neural network (CNN) has the problem of unclear identification of small-scale lesion areas, which leads to the difficulty in diagnosing the dry and wet aspects of age-related macular degeneration (AMD), and judging the activity of choroidal neovascularization (CNV), but correct judgment of lesion type is crucial for ophthalmologists to formulate treatment plans. Therefore, a CNN model MobileX-ViT based on the self-attention mechanism is proposed, which combines the traditional convolution layers and self-attention module, and simultaneously extracts the feature information of the shallow network and obtains the global information of the image to improve the performance of the model. Experiments have proved that compared with the classic CNN classification models Inception-V3, ResNet-50, VGG-16 and MobileNeXt, the classification accuracy of the proposed model is increased by 5.6%, 5.3%, 4.5% and 2.8% respectively. The effectiveness of the model is proved, and it provides a new method to solve the problem of unclear identification of small-scale lesion areas in the current classification of retinal OCT images.