With the rapid development of high-speed railway and urban rail transit systems, research on traffic safety technology is becoming increasingly urgent. The 3D point cloud of railway line environment generated by applying laser scanning technology can achieve accurate perception and monitoring of operating environments. In this paper, the three-dimensional point cloud data of railway scenes is taken as the research object, and a large-scale point cloud semantic segmentation dataset for railway scenes is constructed for the first time. The existing point cloud semantic segmentation models are mainly applicable to small-scale scenes, and large scenic point clouds need to be segmented first. However, three-dimensional point cloud data of railway line environments have the characteristics of high data acquisition frequency and large data scale. Therefore, a large-scale point cloud semantic segmentation method for semantic perception of railway scenes is proposed in this paper. During the coding stage, an adaptive local feature fusion module based on self-attention is proposed in the encoding stage, which can better aggregate local features of different scales and solve the problem of category imbalance. In the decoding stage, an up-sampling method guided by high-dimensional semantic information is proposed to compensate for the information loss caused by large-scale down-sampling in the coding stage. The proposed method achieves excellent segmentation performance on both railway scene datasets and public indoor datasets.