The inspection and identification of human biological tissues, such as nails, play an essential role in investigating several criminal cases. To quickly and nondestructively identify nail tissues extracted from crime scenes, this paper proposes a nondestructive identification and gender characterization method of human nails based on molecular spectroscopic analysis and machine learning. We establish various classification prediction models by collecting 120 infrared spectroscopy data of different gender nail samples of the same age group. Using principal component analysis technology, dimensionality reduction is used to extract 3 principal components, and the samples are interactively verified. The recognition effects of Fisher discriminant function, multilayer perceptron, and back propagation (BP) neural network model are also compared. The experimental results show that the classification and recognition rate of the multilayer perceptron model can reach 91.4%, which is better than the Fisher discriminant analysis model. The BP neural network model based on the particle swarm optimization algorithm has the best classification effect, with a recognition rate of 97.7%.