Conformal vibration polishing is an effective mid-spatial frequency (MSF) error smoothing process method for complex surfaces, and it has broad application prospects in the processing of optical elements in many major optical engineering projects. In this study, a finite element based simulation method is proposed to address the issues of unclear polishing mechanism and unclear process rules in vibration polishing of viscoelastic tools. Furthermore, the influence mechanism of flexible layer material properties, polishing pressure, and vibration frequency on mid-spatial frequency error smoothing in vibration polishing is examined. Additionally, the influence law on the mid-spatial frequency error smoothing ability is obtained. Through simulation analysis and experimental verification, the following conclusions are drawn: the mid-spatial frequency error smoothing ability of conformal vibration polishing initially increases and then decreases with the increase in the elastic modulus of the viscoelastic material, increases with the increase in polishing pressure, and initially stabilizes and then increases with the increase in vibration frequency. When the maximum elastic modulus of the viscoelastic body is within 9?12 MPa, the polishing pressure is between 800?1100 Pa, and the vibration frequency is between 20?30 Hz. Furthermore, the mid-spatial frequency error smoothing effect of vibration polishing is optimal, and it also has exhibits low-spatial frequency conformal ability.