A single mode-multimode-single mode (SMS) optical fiber structure is adopted, and a pattern recognition classification method is proposed based on the combination of short-time Fourier transform (STFT) and convolutional neural network (CNN) to deal with the intrusion signals which are applied on the multimode fiber. The proposed method initially performs STFT on the intrusion signal to obtain the time-frequency map and subsequently creates a training set and a test set. Further, the training set is input into three network models for training, and a reasonable network model is selected according to the engineering application index. Finally, the identification result of the intrusion signal is made to the test set through the network model; furthermore, the validity and real-time performance of the method are verified using four intrusion signals. The results denote that the proposed method can effectively identify artificial and non-human intrusion signals; in addition, the robustness of this method can be verified by increasing the types and quantities of intrusion signals with noises, thereby reducing the alarm failure and false alarm rate of the intrusion signals and improving the application value of the SMS fiber structure in perimeter defense area pattern recognition.