Surface quality of the workpiece is critical for part reliability, quality and service life. Although various vision-based target detection frameworks have been widely applied to industrial surface defect detection scenarios, surface defect detection of ultra-precise machining workpieces is still challenging due to the influence of face shape and the confounding nature between defects. Therefore, we propose a frequency-embedded two-branch parametric prediction network to predict the filtering parameters and filter out the profile information to make the defect features more significant. After pre-processing based on intelligent type surface analysis, a cascaded regional neural network-based perceptual field enhancement defect detection network is proposed. It replaces the deformable convolution intervals into the convolution module of the EfficientNet, which effectively improves the feature extraction capability of the backbone network. Then, the feature map is reselected to form a new feature pyramid network to improve the efficiency and further improve the network performance. In addition, the filter parameter dataset ultra precision polishing (UPP-CLS) with filter parameter labelling information and the defect detection dataset UPP-DET with defect category and location are constructed. The model achieves 85.36% accuracy on UPP-CLS, which is 3 to 5 percentage points higher than that of the existing networks, and 0.862 average precision on UPP-DET, which is 5.3%?7.8% higher than that of the existing networks. The overall performance of the model is better than the mainstream network architecture. The source code and dataset will be available at https://gitee.com/zihaodl/detect_app.