Multispectral LiDAR can directly and effectively obtain point clouds containing spectral information, and it has become a new trend of LiDAR imaging technology. The point clouds obtained by this new multispectral LiDAR have more spectral and color information, which puts forward higher requirements on the data quality of point clouds; therefore, point cloud denoising becomes the key to improve the data quality. The traditional monochromatic point cloud denoising algorithm mainly uses spatial information to remove noise, but it is not suitable for multispectral LiDAR point clouds. In this paper, a multispectral LiDAR point cloud denoising algorithm based on color clustering is proposed. First, the point clouds containing color information are inverted according to the spectral information obtained by the multispectral LiDAR. Then, the point clouds are clustered by color difference. After the clustering, the density of noise points in each cluster is lower than that of the real point. Finally, the noise is identified and removed in each cluster. The results show that the proposed algorithm can effectively remove the noise from the multispectral LiDAR point cloud with a ground object identification accuracy of above 95%.