A lightweight convolutional neural network, AGNet-improved GhostNet-50, is proposed for ship classification using self-made ship datasets while ensuring a desirable classification accuracy with a small model size. First, a Ghost module that integrates asymmetric convolution is proposed to improve the feature extraction capability of the AGNet convolution process. Next, combined with the bottleneck structure, an asymmetric Ghost bottleneck module is designed, further reducing the computational cost while maintaining the expression ability of the model. Finally, a 1×1 convolution layer in GhostNet-50 is removed to reduce the parameter redundancy of the overall model. The proposed method was compared from multiple aspects using evaluation indexes such as classification accuracy, parameters, and computational and inference speeds. Based on the experimental results, the accuracy of the AGNet model in the test set of 33 categories reaches 93.87%, and the number of model parameters is only 0.72×10⁶. Compared with GhostNet-50, the AGNet model is compressed in terms of size by 46.67%, and the accuracy is improved by 2.93 percentage points. The experimental results show that AGNet can achieve better classification performance with a smaller model size and can be better applied to ship classification tasks.