In order to improve the accuracy of road unevenness detection by vehicle-mounted lidar in outdoor scenes, the road environment information was extracted and segmented by the network structure of random down-sampling and local feature aggregation. Random sampling method was added in the segmentation process to improve the computing efficiency of high point cloud information. To solve the problem of the loss of key features in the segmentation process of road environment information, local feature aggregator was added to increase the acceptance domain of each 3-D point cloud to retain geometric details. The results show that the proposed algorithm can accurately identify the road environment information, and the recognition accuracy of convex hull, pit, and road able area reaches 71.87%, 82.71%, and 93.01% respectively, which is significantly improved compared with the traditional convolution neural network. This study can efficiently extract the information of road roughness and road able area. Thus, the active safety and ride comfort of the vehicle are improved.