Light detection and ranging (LiDAR) offers the advantages of high-ranging accuracy, negligible environmental influence, and all-day operation, making it suitable for unmanned ship obstacle detection. Since LiDAR point cloud is nearly dense and far sparse, the grid's size has a direct effect on the accuracy of obstacle detection based on a grid map. In this study, a variable size grid map with a linear increase of grid size is established. The grid is divided and clustered using the height difference discrimination method and eight-neighborhood connected component marking approach, and the obstacle information is extracted using a box model, yielding a more accurate obstacle detection result. The proposed method can effectively solve the problems of grid division and poor clustering effect in traditional approaches for the detection of small and medium fishing boats near shore, according to the results of a real ship experiment conducted at sea that compared the processing results of the proposed method with the traditional approach. The proposed method provides more accurate and real-time detection of obstacles at sea, as well as data support for unmanned ship obstacle avoidance.