To address the problem that word vectors or character vectors are difficult to reconstruct the two-dimensional structure in mathematical expressions when using generative adversarial networks to generate images, the task of generating images with handwritten mathematical expressions is converted into a style conversion problem from printed mathematical expressions to handwritten mathematical expressions, and a self-constructed dataset with handwritten style categorization is used to train the style conversion model. In order to solve the problem of incomplete content, distorted details and low quality of images generated by CycleGAN network, a multi-scale discriminative cyclic consistency generative adversarial network MD-CycleGAN is designed, which introduces the CBAM attention mechanism to compensate for the loss of information in the downsampling link, introduces the ACON activation function instead of the ReLU activation function, and controls the network through adaptive learning nonlinearity degree of each layer. The experimental results show that the data enhancement method based on generative adversarial network in this paper can effectively reduce the degree of model overfitting. This study provides a new method for automatic recognition of handwritten mathematical expression images, which overcomes the data annotation problem and the model generalization problem, and has the potential for a wide range of applications, including the fields of mathematics education, scientific document processing, and mathematical search engines.