The Risley-prism system has a great application prospect in target tracking with the aim of both large field of view and high precision due to its extension of the field of view. Although increasing the prism vertex angle enlarges the magnification of the field of view, it also expands the influence of the system assembly error on the target pointing accuracy. Considering the low pointing accuracy of the Risley-prism system with a large vertex angle, this paper proposes a method of correcting the pointing error of the Risley-prism system based on the particle swarm algorithm. A Risley-prism imaging system is built with a prism vertex angle of 14.85° and a prism refractive index of 1.515. A mathematical model of an experimental prototype based on assembly error analysis is built by using pointing test results of the ideal model. Parameter identification of the setting values of the prism vertex angle and the prism refractive index in the reverse solution algorithm is conducted. Finally, a pointing test is carried out on the experimental prototype. The experimental results demonstrate that the proposed method effectively improves the pointing accuracy of the Risley-prism system with a large prism vertex angle. When the method is applied, the maximum pointing error decreases by 52.4%, and the average pointing error reduces by 43.3%. The root-mean-square error decreases by 44.0%, and the fitting radius of the least squares method decreases by 44.7%.