As a novel distributed sensing system, the distributed optical fiber vibration system (DOFVS) has been widely applied in recent years due to its advantages of real time, high accuracy, and strong robustness. DOFVS has many application fields, such as structural health monitoring, pipeline leak detection, and perimeter security. In recent years, DOFVS performances such as spatial resolution, monitoring distance, and accuracy have been improved with the demodulation algorithm development and system structure optimization. Meanwhile, with the development of technologies such as deep learning and artificial intelligence, DOFVS also gradually becomes intelligent. To achieve accurate automatic pattern recognition of vibration signals, we combine the DOFVS with an unmanned aerial vehicle (UAV) video monitoring system. The proposed system employs convolutional neural networks to realize pattern recognition in optical signals and video signals simultaneously. Our scheme increases the number of recognizable sensing events and improves recognition accuracy, expanding the intelligent application scenarios of DOFVS.

We propose a multi-dimensional sensing event recognition scheme based on convolutional neural networks, combining the DMZI-based DOFVS and a UAV video monitoring system. The proposed scheme adopts Resnet 50 as the feature extraction backbone network to extract features of the optical signals and video signals. The optical signals are transformed from 1D time-domain signals to 2D time-frequency signals by short-time Fourier transform. The 2D time-frequency images are then segmented based on power distribution to reduce image noise, and the images are fed into a 2D Resnet 50 network to obtain the confidence of the recognized sensing events. The 3D video signals are fed into a SlowFast model with a 3D Resnet 50 as the feature extraction network to obtain the confidence of the recognized sensing events for video signals. Finally, the confidence vectors obtained from both optical and video signals are multiplied and normalized, and the event with the highest confidence is output as the final judgment event. To verify the feasibility of the proposed method, we conduct experiments to recognize nine types of sensing events, and the average recognition accuracy and system response time of the proposed scheme are obtained.

The proposed scheme overcomes the limitation of recognizing multiple events when only recognizing optical signals. The employed dataset consists of two parts: one is the 2D time-frequency images corresponding to optical signals with 1800 images for each sensing event, and the other is video data obtained from UAV with 140 segments of 20 s videos for each intrusion event (Table 3). Both parts are divided into training, validation, and testing sets in an 8∶1∶1 ratio. To validate the feasibility and effectiveness of the proposed solution, we compare the results of recognizing optical signals alone, results of video signals alone, and the fused recognition results (Table 4). Optical signals achieve high recognition accuracy on events with more obvious time-frequency features, such as climbing, cutting, and pulling. However, the events with similar features have low accuracy, such as crashing, kicking, and waggling. Similarly, the accuracy of UAV video signals for events such as climbing, knocking hard, and pulling is low. When optical signal recognition and video signal recognition are applied separately, neither of them achieves sound pattern recognition results. After confidence fusion, the proposed method achieves 99.58% recognition accuracy for nine sensing events in the testing set. Moreover, the recognition of optical signals and video signals can be performed simultaneously, and the system response time can meet the real-time detection needs.

We propose a multi-dimensional DOFVS pattern recognition scheme based on convolutional neural networks (CNNs), which combines two models including a 2D time-frequency signal recognition model based on the Resnet 50 and a 3D video signal recognition model based on the SlowFast model. This scheme not only expands the features of the optical signal by time-frequency transformation but also automatically extracts and classifies features using CNNs. The impact of low robustness of manual feature extraction schemes can be reduced. Meanwhile, the 3D video signal recognition is combined with optical signal recognition to enable the detection of nine types of events including climbing, crashing, cutting, kicking, knocking hard, knocking lightly, pulling, waggling, and no intrusion. The effectiveness of the proposed scheme is verified via experiments, which demonstrate that the average accuracy of the nine events is 99.58% and the recognition time is 0.16 s to achieve real-time synchronous response to event changes. Compared with traditional single optical signal recognition, the proposed scheme greatly expands the event types that can be recognized in the DOFVS field. Therefore, this scheme will further improve the DOFVS stability and reliability in practical engineering applications in the future.