In this paper, to achieve accurate automatic classification of breast cancer histopathological images, an improved convolutional neural network is proposed, and two different convolutional structures are introduced in order to improve the accuracy of histopathological image recognition by the network. Based on using deep residual network (ResNeXt) as basic network, octave convolution (OctConv) is used to replace the traditional convolutional layer to reduce the redundant features in the feature map during feature extraction stage and improve the effect of detailed feature extraction. Heterogeneous convolution (HetConv) is introduced to replace part of the traditional convolutional layers in the network, reducing model training parameters. To overcome the problem of over-fitting due to the small number of data samples, an effective data enhancement method based on the idea of image block is adopted. The experimental results demonstrate that the accuracy of the network on the four classification tasks of the network at the image level reaches 91.25%, indicating that the designed network model has a higher recognition rate and a better real-time performance.