For the evaluation of star tracker accuracy on board, a method was proposed using differenced quaternion, sliding window and optical axis included angle to analyse noise equivalent angle, total error of star trackers respectively, based on several kinds of data download from PuJiang Satellite-1 on board. The on board test result shows the optical axis direction accuracy of the star tracker reached 15″ (3σ), meeting the requirement of AOCS, and the three-axis stability reached 0.01°. The method can be used for star tracker accuracy evaluation on board or real sky experiment on ground.