The rapid identification of lithology holds significant fundamental geological research significance as well as engineering application value. Traditional lithology recognition primarily depends on the image features of rocks. However, confusion tends to arise when identifying rocks with similar appearances. Consequently, relevant studies further utilize spectral features to reflect the compositional information of rocks. Nevertheless, spectral testing usually demands sample preparation and belongs to the category of destructive testing. This article proposes an intelligent lithology recognition method based on transfer learning of rock hyperspectral images, taking advantage of the integrated imaging hyperspectral technology and the non-destructive, non-contact imaging characteristics. Firstly, the hyperspectral data of the rock region of interest are normalized, and dimensionality reduction is performed to reduce the redundancy of spectral data. Then, a rock hyperspectral image transfer learning model is constructed using a 3D ResNet network, and three-dimensional information is extracted through a residual network. The transfer learning method is reused to train the model by loading pre-trained weights, thereby achieving intelligent recognition of lithology. In this article, the confusion matrix, accuracy (ACC), precision (P), recall (R), and F1 values (F1) are used as evaluation indicators for model accuracy. A comparative analysis is conducted on ResNet101 and ResNet18/34/50 models. The results indicate that the ResNet-101 migration model has the highest accuracy in the test set, reaching 98.29%. The highest P can reach 98.32%, the highest R can reach 98.29%, and the highest F1 can reach 98.31%. The accuracy of ResNet-101 in identifying rock spectral data is over 90% (except for chlorite schist), and most results can even reach 100%. Compared to ResNet18/34/50, ResNet101 has higher recognition accuracy and better stability for identifying each type of rock. In addition, this method was employed to predict the lithology of sampled tunnel site rocks pixel by pixel, verifying the good robustness and generalization performance of the proposed lithology intelligent identification method, which can be used for rapid and intelligent lithology identification in engineering fields like geology, logging, transportation, and water conservancy.