Identifying the freezing injury of tea plants is the basis of evaluating the stress resistance of tea plants and guiding the overwintering management of tea plantations. The traditional method of tea plant freezing injury identification observes the number and degree of freezing injury of tea leaves manually, which has some disadvantages, such as low accuracy, low efficiency, strong subjectivity and so on. A framework for evaluating freezing injury of tea trees based on deep learning, wavelet transform, and visible spectrum is proposed. Firstly, we collected 1000 crown images of frozen tea trees, divided them into the training sets and test sets according to 4∶1, and labeled the frozen leaves in the training set images. Secondly, we use the Faster R-CNN network to identify and extract tea plants frozen leaves, select three feature extractors: AlexNet, VGG19 and ResNet50 respectively, and select the feature extractor with the highest robustness as the backbone network. Then, the extracted image of frozen leaves of tea plants is enhanced by wavelet transform, and one low-frequency image and three high-frequency images are obtained. Then the images processed by wavelet transform and those not processed by wavelet transform are input into VGG16, SVM, AlexNet, ResNet50 and other networks to classify the frozen leaves, and the classification performance of the four networks is compared. Finally, according to the number of frozen damaged leaves, the degree of frozen damaged leaves and the weight coefficient of leaves with different degrees of frozen damage, the freezing degree of tea plants is scored to evaluate the overall freezing degree of tea plants. The results show that: (1) the Faster R-CNN model based on ResNet50 has the best performance in extracting frozen leaves of tea plants, with a precision rate of 93.33% and a recall rate of 92.57%, which is higher than the recognition performance of VGG19 and AlexNet as the backbone network, which can ensure that most frozen leaves can be extracted and provide a basis for further classification of the degree of freezing damage of leaves. (2) The overall accuracy of the VGG16 model in classifying leaves with different degrees of freezing injury is 89%, which is higher than that of other models (SVM, AlexNet, ResNet50), which shows that the vgg16 model has high robustness. (3) Compared with the frozen leaves without wavelet transform, the overall classification accuracy of the model can be improved by 2%~6%. It shows that wavelet transforms enhancement technology can improve the accuracy of the network. Therefore, this experimental framework can accurately and efficiently classify the frozen leaves of tea plants, which is of great value for evaluating the degree of freezing injury of tea plantations and provides technical support for the overwintering protection of tea plantations in the north.