To address the problem of poor reconstructed image quality of traditional ghost imaging in handwritten digit recognition, this paper proposes a quality optimization method for ghost imaging based on the advantageous fast data generation in generative adversarial networks. The proposed method can improve the reconstruction quality of ghost images at a low sampling rate. Furthermore, the method concretely comprised the following steps: initially, a barrel detector collected the light intensity of the handwritten digital image irradiated by a series of scattering spots to obtain the total light intensity value; subsequently, a deep convolutional generative adversarial network applicable to the principle of ghost imaging was built, and the light intensity value was used as an input to train the model; finally, comparative analyses were performed with the traditional ghost imaging method and u-net network to verify the effectiveness and validity of the proposed method. The experimental results show that the reconstructed image obtained using the proposed method is considerably superior to the comparison methods. Additionally, at sampling rates of 0.0625 and 0.25, the peak signal-to-noise ratio and structural similarity of reconstructed image are 18.9%/51.9% and 38.29%/42.35% higher than those obtained using the u-net network, respectively.