In order to study the effects of different rogue particles on the scratches, and establish the relationship between the rogue particle size and the scratch topography, experiments have been carried out. During the process of phosphate neodymium polishing, three different sizes of corundum, cerium oxide agglomerate and polished powder are introduced as rogue particles. Data of scratch topography is analyzed statistically. The results show that brittle scratches, plastic scratches and mixed scratches all exist, among which brittle scratches account for a large proportion; scratches caused by cerium oxide agglomerate and polished powder are shallow and few, while scratches caused by corundum are deep and many; the scratch density and scratch length caused by corundum of different sizes increase with the increasing of particle size; distribution of scratch widths caused by different sizes of corundum is similar to the distribution of the size of rogue particle, shown as Gaussian distribution. Using the model proposed by T. Suratwala, mechanism of scratch generating during phosphate neodymium glass polishing is investigated. It is illustrated that the width and length of scratches increase with the increasing of rogue particle size.