CVD diamond with high surface flatness has very promising applications in microtools, optical windows, semiconductors and other fields. However, the high hardness, high wear resistance and chemical inertness of CVD diamond make the high-precision polishing of CVD diamond a great challenge. To address the difficulty of diamond polishing, nanosecond laser composite ion beam etching technology was used to polish CVD diamond, and an experimental study on laser polishing of polycrystalline CVD diamond was carried out to investigate the effects of laser power and scanning speed on the ablation depth and surface roughness of CVD diamond, based on which ion beam etching and polishing of CVD diamond after laser polishing was carried out, and the changes in the surface roughness of diamond under ion beam polishing were investigated. The surface roughness change was investigated, and the surface composition of diamond after ion beam polishing was analyzed. An excellent polishing effect of reducing the surface roughness of CVD diamond samples from 6.687 μm to 0.804 μm was realized by the described method. This study provides a diamond polishing method designed to have applications in fields such as infrared windows, micro-milling machines, and aerospace.