In order to meet the demand for all-day observation in near-Earth space by star sensors, star sensors based on short-wave infrared (SWIR) detectors have been widely used. However, noise such as stripe non-uniformity and defective pixels are present in SWIR images, which degrade the quality of the images. To suppress the influence of the above noise, a noise suppression algorithm for SWIR star image is proposed in this paper. Firstly, the two-point method is used to make a preliminary non-uniformity correction to reduce the non-uniformity between pixels in the same column. Then, the influence of the stripe non-uniformity noise is removed by an improved column-Top-hat algorithm. Finally, the star point is extracted by the connected domain algorithm based on center point diffusion. The experimental results show that the proposed algorithm can suppress the influence of noise well within two magnitudes, and the extraction error of the star points can be kept within 0.3 pixels, i.e., 2". Meanwhile, the algorithm is designed for a single-frame images, which is easy to implement and can satisfy the requirements of real-time processing.