In rail transportation, detail inspection of rails and trains is required to ensure safe operation. The surface is measured when the train is running or rail inspection is performed on a moving vehicle to improve the detection speed and efficiency. Three-dimensional (3D) shape measurement is also needed because of the rich information, and the point cloud density should be high enough to find minor defects. Dynamic line-scan point cloud measurement based on line-scan cameras shows great potential to meet the above requirements. The line-scan cameras can capture one-dimensional (1D) images at ultra-high frequencies and resolutions, and high-density point clouds can be easily obtained in motion. However, the complex perturbance in motion represented by multi-degree-of-freedom deviations and vibrations tends to introduce errors into dynamic line-scan point clouds and poses serious challenges to point cloud correction. We report a dynamic line-scan point cloud correction method based on two-dimensional (2D) image reference. A two-step correction based on global transformation and optical flow analysis of reprojection images is adopted to achieve an accurate and reliable correlation between point cloud profiles and 2D image reference under complex perturbance. The pose of each point cloud profile is optimized, and correction compensation based on the low-noise reference according to both images and point cloud information is designed to achieve comprehensive correction. We hope the proposed method can provide effective support for point cloud-based detail analysis and inspection of rail transportation.

First, a point cloud correction based on global transformation is adopted. As a preliminary correction method, it can quickly eliminate the influence of obvious motion deviation and improve the reliability of subsequent fine correction. Meanwhile, we perform the global geometric transformation on the reprojection image of the point cloud, making it as consistent as possible with an undistorted 2D reference image captured by an area-scan camera. The image geometry transformation includes translation in two directions, scaling in two directions, image rotation, and shearing in two directions. The gradient descent method is adopted to obtain the global image transformation, and the pixel deviations between the reprojection image and the 2D reference image are calculated by the transformation matrix. According to the imaging model and the point cloud pose perturbation model, the point cloud correction vector is calculated for every point cloud profile. Second, correction based on optical flow analysis of reprojection images is adopted. The Demons algorithm based on the optical flow field features high operation speed and high registration accuracy and is not susceptible to the distorted features of the reprojection image. Therefore, the non-rigid image deformation can be modeled via optical flow analysis, and the pixel deviations between the reprojection image and the 2D reference image can be obtained. Similar to the first step, the pixel deviations are converted to point cloud correction parameters, and iterations are employed to improve fine correction quality. Third, correction compensation based on low-noise reference is implemented. The continuous surfaces are identified according to both images and point cloud information, and the low-noise reference is obtained by fitting. The point cloud accuracy is further improved under the constraint of the low-noise reference. This method is easy to implement, and the texture of rails and trains is enough for effective correction without the need for additional speckle spraying.

The measurement results of the rail sample are utilized to verify the proposed method. An accurate two-ball model is adopted as a detailed feature to assist in accuracy evaluation. In the verification experiment of method effectiveness, both qualitative and quantitative accuracy evaluations demonstrate that the point cloud deformation is reduced with improved reconstruction accuracy (Figs. 10-13). The improvement can be seen in the texture of the point cloud and the 3D point cloud shape. In the precision comparison experiment, the proposed correction method has better control of local detail reconstruction accuracy than the previous methods based on pose estimation using attached sensors (Figs. 14-16). Meanwhile, the proposed method is easy to implement and is not costly. In the ablation experiment, each of the three correction processes has proven to be of great importance (Fig. 17).

We propose a dynamic line-scan point cloud correction method based on 2D image reference. With the help of 2D reference images, the initial correction based on the global transformation of reprojection images is adopted to eliminate the influence of obvious motion deviations, and fine correction based on optical flow analysis of reprojection images is utilized to correct small irregular deformations. A correction compensation method based on low-noise reference is designed to realize better 3D reconstruction. Through quantitative and qualitative evaluations, the proposed method can realize effective and reliable point cloud correction under complex perturbance scenes. The comparison experiment further verifies the performance of the method, and the ablation experiment validates the importance of each correction process. Therefore, the proposed method has potential applications in 3D reconstruction and detail inspection of rail transportation.