To improve the sensitivity and cost-efficiency of a fiber bending sensor and to increase its linear range, a method based on a deep neural network was proposed to classify different bending angles and directions of plastic fiber. Plastic fiber with side throw sensitization processing was used to collect speckle images of different bending angles at the output end of the fiber. Data set one was made with five types of bending angle and data set two contained seven types of bending angle. After the pretreatment of image data, a multilayer convolution neural network was used to analyze the speckle image. The convolution and pooling provided speckle image features. A softmax classification was used for classification accuracy. Finally, the effect of two different convolutions on the classification of the neural network model was compared. The results show that the classification accuracy reaches 96% when the angle interval of fiber bending in the data set one is 5°. The theoretical and practical analysis results show that the scheme has a high recognition rate. Moreover, the realization of this method is expected to provide a new type of simple and efficient fiber bending sensor.