To meet the requirements of intrusion detection with high recognition rate and low false alarm rate in specific areas such as airports and oil depots, a target recognition method based on optical fiber sensing and infrared video was proposed. Among them, the distributed optical fiber vibration sensor based on MCSVM ADMZI (asymmetric dual Mach-Zehnder interferometer) was used in the optical fiber sensing part, which combined the empirical mode decomposition (EMD) and the kurtosis feature with the MCSVM to improve the recognition rate. The infrared recognition part improved the clarity of the gray difference image through the wavelet transform. The intrusion detection was realized by pattern comparison algorithm. The field experiment results show that this method can identify four common intrusion events (climbing fence, tapping cable, cutting fence, shaking fence). The average recognition rate is over 92.5%, and the false alarm rate is 0.9%. Compared with the traditional single sensor scheme, this method has a great improvement in the system performance such as false alarm rate and false alarm rate, and can meet the practical application requirements.