Vision-based three-dimensional (3D) trajectory and velocity measurements of high-speed moving targets have an important role in structural health monitoring^[1], robotics^[2,3], object tracking^[4,5], experimental mechanics^[6,7], and other fields. Since trajectory and velocity calculations require finding corresponding features between adjacent frames of a continuous image sequence, their accuracy depends heavily on the frame rate of the camera^[8]. High-speed cameras have disadvantages such as high-transmission bandwidth requirements, high cost, and short recording time. As frame-based cameras record the movement of an object over a period of exposure time, if the exposure time is too long relative to the speed of the target’s movement, the image will have a motion blur effect, which needs to be reduced by using a very short exposure time and a flash synchronized with the camera^[1,6,7]. For the measurement of high-speed rotating objects, researchers have relied on optical de-rotator to eliminate motion blur caused by high-speed rotation^[9], but this approach requires high hardware requirement and requires precise co-axiality between the rotational axis of the optical de-rotator and the rotating object, as well as precise matching of their rotational speeds^[10,11]. When 3D information about the object needs to be measured, multiple high-speed cameras are required to form stereo vision^[12,13]. Time synchronization between multiple high-speed cameras is a challenging task. Wang et al. used a galvanometer scanning system to generate a reference signal to correct the camera start-up time difference^[13]. Yu et al. used a single high-speed camera combined with reflectors or prisms to form stereo vision^[14,15], which avoids the problem of temporal synchronization between multiple cameras.

Conventional cameras use frame-based imaging sensors such as complementary metal oxide semiconductors (CMOS) and charge-coupled devices (CCD). A frame-based sensor has the same exposure time for all pixels at the same moment (global shutter) or over a period of time (roll-up shutter) to produce a frame of the entire field of view. However, this frame-based imaging data is very inefficient and redundant in target tracking tasks, where we are only interested in the pixels where the moving target is located. Event camera is an emerging vision sensor that generates ‘events’ by asynchronously detecting the change of illumination intensity at each pixel. The event camera outputs pixel coordinates and time when there is a change in illumination intensity of a pixel that exceeds a set threshold. The output is with low redundancy as data is only generated at the pixel locations where the illumination intensity changes. The advent of event cameras dates back to 1992 when Mahowald^[16] proposed a new type of vision sensor called the ‘Silicon Retina.’ In the following decades, more and more commercial event cameras have been developed^[17]. Among them, three types of cameras are widely accepted and used: dynamic vision sensor (DVS), asynchronous time-based image sensor (ATIS), and dynamic and active pixel vision sensor (DAVIS). DVS is an event-only camera, whereas ATIS and DAVIS can output events as well as gray-scale information. Figure 1 shows a comparison of the principle of imaging a moving object with an event camera and a frame-based camera. Compared with the data recorded by the frame-based camera, the data recorded by the event camera has better continuity in the time axis and less data redundancy. Characteristics of event cameras make them highly advantageous for target tracking and have gained a lot of attention in recent years. Saner et al. used one event camera for two-dimensional (2D) tracking of high-speed targets^[18], and Zhou et al. utilized two event cameras to form stereo vision for 3D reconstruction of objects^[19], with the problem of temporal synchronization when using the multi-event camera approach.

To address problems of camera synchronization difficulties, data redundancy, and motion blur in high-speed target 3D trajectory measurement, in this work, we propose a single-event-camera stereo vision method using a four-mirror adaptor. We have used the proposed method to measure the 3D trajectory and velocity of a particle flight process, and the results show that the method is very suitable. In addition, we have applied the proposed method to the measurement of rotating objects, and the results show that the proposed method can monitor the operational state of high-speed rotating objects at a very low hardware cost. The proposed method provides a new approach for vibration measurement of rotating objects and external monitoring of rotational angles^[3].

The event camera used in this Letter is the CeleX5-MP (1280×800 pixels) from CelePixel Inc., which has various modes such as gray scale, event, and optical flow. The gray-scale mode is the same as a conventional camera, which allows us to calibrate the imaging parameters using conventional calibration methods. A single-view image provides only 2D information about the object’s motion and requires the object’s trajectory to be perpendicular to the camera’s optical axis, which is a great limitation of the application range of event cameras. In order to obtain the 3D trajectory of an object, multi-view measurements are required. Accurate temporal synchronization between multiple event cameras is very difficult, so we used a four-mirror-based monocular stereo vision approach^[20]. As shown in Fig. 2, four mirrors are mounted in front of the lens to divide the image into two parts. The left and right parts of the sensor have a difference in viewing angle and form stereo vision.

Before the 3D reconstruction^[21–23], the stereo vision system needs to be calibrated. The process of obtaining the imaging parameters and the relative position relationship between the two views of stereo vision is called stereo calibration. We use the pinhole imaging model to describe the projection of a point in the world coordinate system onto the camera target surface, expressed in homogeneous coordinates as α[xiyi1]=[fxfscx0fycy001][R11R12R13TxR21R22R23TyR31R32R33Tz][XwYwZw1],(1)where (cx,cy) is a principal point, usually the image center; (fx,fy) represents the image distance expressed in pixels; fs is the non-perpendicular angle for the camera sensor array; α is a nonzero scale factor; R|T is the transformation between the world and principal point coordinate systems; (Xw,Yw,Zw)T are the coordinates for a point in the world coordinate system, and (xi,yi)T are the coordinates for the projection point in pixels. We define the transformation from the left to right virtual camera principal point coordinate system as Rh|Th [Fig. 3(a)]. We used the plane model calibration method to obtain Rh|Th, which requires the monocular stereo vision system to take a set of calibration board images [Fig. 3(b)]. The world coordinate system (Xw,Yw,Zw)T was defined to coincide with the principal point coordinate system for the left view of the monocular stereo vision system. Therefore, R becomes a unit matrix and T a zero vector for the left virtual camera in Eq. (1), and R|T is the transformation from the principal coordinate system of the left virtual camera to that of the right virtual camera. Thus, Eq. (1) can be simplified to {xil=cxl+fxlXwZw+fslYwZwyil=cyl+fylYwZwxir=cxr+fxr(R11hXw+R12hYw+R13hZw+txh)R31hXw+R32hYw+R33hZw+tzh+fsr(R21hXw+R22hYw+R23hZw+tyh)R31hXw+R32hYw+R33hZw+tzhyir=cyr+fyr(R21hXw+R22hYw+R23hZw+tyh)R31hXw+R32hYw+R33hZw+tzh.(2)

Solving Eq. (2) provides the world coordinates for points to be measured from pixel coordinates in the left and right views of the monocular stereo vision system.

Since we can obtain the 3D coordinates of the object to be measured at each moment, we can obtain the displacement vector by making the difference between the coordinates of adjacent moments. The derivative of the displacement vector with respect to time can obtain the velocity vector of the object to be measured at this time, and the mode of the velocity vector is the velocity at this time. The proposed measurement system is, in principle, a conventional stereo vision method, but the camera is replaced by an event camera, and the estimation of the measurement accuracy of in-plane and out-of-plane displacements follows the estimation method of the conventional stereo vision.

The event camera used in this Letter costs about 16,000 renminbi (RMB) and has a temporal resolution of 10−5s. Conventional high-speed cameras with the same time resolution usually cost more than 500,000 RMB. We utilize the 3D measurement capability of the proposed measurement method to measure the 3D trajectory and velocity of a small steel ball hitting the wall and rebounding (Fig. 4). Since there is no guarantee that the object’s motion trajectory is perpendicular to the camera’s optical axis, single-view measurement cannot deal with this situation. We can observe the deceleration of the small ball due to the impact and acceleration after the rebound in Fig. 4(b).

Vibration measurements of rotating components have been of great interest to the engineering field^[24]. In recent years, thanks to the development of the laser Doppler vibrometer (LDV), researchers can use an optical de-rotator combined with scanning LDV to obtain surface vibrations of rotating components^[10], but both the optical de-rotator and scanning LDVs are very expensive. For in-plane deformation measurements, some researchers have used an optical de-rotator combined with a camera to measure the in-plane deformation of rotating components^[11], but this method cannot measure the off-plane deformation, which is the most important part of the rotating component deformation.

We apply the proposed method to the vibration measurement of a rotating disc. The proposed method can measure the 3D trajectory of the marker on the rotating disc, and its vibration information can be obtained by the analysis of the 3D trajectory. The experimental setup is illustrated in Fig. 5. A brushless motor for an unmanned aerial vehicle was used to drive a disc made of acrylic (thickness: 2 mm, diameter: 20 cm). The rotating disc is black with a marker dot painted in white. The marker is about 38 mm from the center of rotation. When the disc rotates, only the movement of the marker points will cause events, so the redundancy of the data will be expected to be very low. Figure 6 shows the event data due to rotation after filtering, from which it can be observed that the speed of the disc is about 75 r/s.

Fitting the event data can obtain an expression for the pixel coordinates of the event over time. Combined with the stereo vision calibration results, the 3D coordinates of the marker point can be obtained by 3D reconstruction. From the 3D coordinate information of the marker point, we can get the trajectory and velocity of the marker point during the rotation [Fig. 7(a)]. Extracting the off-plane components of the 3D trajectory of the marker point [Fig. 7(b)] and performing Fourier analysis on them, the spectral distribution can be obtained [Fig. 7(c)]. From the spectrum results, it can be seen that the vibration of the rotating disc mainly consists of the rotational frequency as well as its harmonic frequency response.

In summary, we proposed a single-event-camera-based 3D trajectory measurement method for high-speed moving targets. We have used the proposed method to measure the 3D trajectory and velocity of a particle flight process, and the results show that the method is very suitable. In addition, we have applied the proposed method to the measurement of rotating objects, and the results demonstrate that the proposed method can monitor the operational state of high-speed rotating objects at a very low hardware cost. The method proposed in this Letter is intended to be an alternative to expensive high-speed cameras or scanning LDVs in some application scenarios. As demand for event cameras increases in fields such as autopilot and robotics, it is foreseen that the performance of event cameras such as resolution, signal-to-noise ratio, and bandwidth will advance rapidly. It is meaningful to investigate new event-camera-based measurement methods to achieve a breakthrough of the defects of the old methods. The application of the event camera in the field of experimental mechanics will be greatly expanded with the improvement of their performance.