In optical communication networks, faulty data affects the quality of data transmission. Therefore, a deep learning based mathematical model for data transmission in optical communication networks was designed. Convolutional neural networks were used to extract the characteristics of data transmission in optical communication networks, and data categories were output through fully connected layers and Softmax classifiers. To prevent overfitting, L1 regularization penalty term and optimization activation function are introduced to achieve parameter sparsity. At the same time, the convolutional neural network is connected to the input layer of the deep confidence network, and feedback fine-tuning of the classification results is performed using reverse training. The experimental results show that the data classification error of the model is less than 1.7%, and the output data is highly consistent with the actual data, effectively improving the data transmission quality of the optical communication network.