To investigate the feasibility of diffraction neural network to perform multi-task image classification recognition, a diffraction neural network system is designed and built. The system uses a spatial light modulator (SLM) to modulate the phase and amplitude weights of the diffraction neural network and the optical full connection of the network layers. A CMOS camera is adopted to realize the optical nonlinear activation of the output of each diffraction layer in the diffraction neural network and discriminate the output image recognition results. The designed system model achieves 94.1% and 92.1% accuracy in MNIST and Fashion-MNIST image classification recognition. Finally, by building optical path system, optical experiments have 91% and 81.7% accuracy respectively, which verifies that the designed diffraction neural network system can meet the requirements of various image classification and recognition applications, and provides a new idea for the design and construction of diffraction networks.