The key parameters of the neutral beam, such as beam uniformity and beam divergence angle, can be obtained by analyzing the infrared images generated by the beam bombarding the target surface. Due to the camera setup angle, the IR images show geometric distortion, which affects the accurate analysis of the beam parameters. Therefore, so the images should be corrected for the distortion. The traditional Hough transform, canny lines algorithm, and probabilistic Hough transform methods are not effective in detecting straight lines on this image, and there is a problem of detecting discontinuous and incorrect line segments. In this paper, Sobel filter is used to sharpen the infrared image in the horizontal and vertical directions respectively and line segment feature is detected by line segment detector (LSD) algorithm. Then complete straight lines are obtained in the image by clustering and fitting the line segments according to the geometric and angular relationships between them. Vanishing points is calculated by the intersection points of the lines. Finally, the corrected image is obtained based on the perspective relationship. Experiment results prove that this method can achieve automatic and effective correction of neutral beam infrared images and it lays a basis for obtaining key parameters of the beam.