Star sensor provides high-accuracy attitude information to satellite. With the continuous development of satellite internet platform technology in the national defense and civil fields, the star sensor is required to develop towards the small volume, light weight and high precision. Therefore, the optical system of star sensor should be designed in accordance with the application requirements. The relationship between color temperature and irradiance of sixth-magnitude star in the same integral band was established by using blackbody radiation law, thus confirming the effective aperture. Based on the Gaussian model of star point, it was proposed to realize the small aperture and high sensitivity detection by increasing the energy concentration of the optical system. The working field of view of star sensor was determined on the basis of Monte Carlo method. The technology requirements of stray light suppression was confirmed according to sensitivity. The optical system was designed based on the above mentioned verification results. The energy concentration of 3 pixels×3 pixels is better than 95%. In the range of ?40 ℃ to 60 ℃, the centroid shift of star point is less than 0.40 μm. At the stray light suppression angle of 32°, the extinction ratio of the system is 3.0E-8. In the last part, the rationality of the optical system design was verified through the accuracy calibration of centroid in laboratory, test of accuracy and sensitivity in out-field stargazing, and stray light test. The test results show that the calibration accuracy of the star sensor is 1.45″, the accuracy of out-field stargazing is 4.2″ (3σ), the limiting sensitivity is 6.04 magnitude star, and the mean value of the gray level of stray light background is 36.51.