The hybrid of laser additive and subtractive manufacturing technolog, which involves selective laser melting (SLM) and femtosecond laser subtractive manufacturing, has been considered as an effective process to achieve near-net-net forming of complex and fine structures. In the SLM process, due to the movement of the molten pool and the overlap of the molten channel, the surface of the SLM formed part has a certain periodic structure, which has a great influence on the subsequent femtosecond laser subtractive manufacturing. In this study, the surface structure of the SLMed Ti-6Al-4V workpiece was measured first, and the femtosecond laser subtractive manufacturing experiment was designed. Through experiments and a two-dimensional numerical model, the evolution of the surface morphology in the process of femtosecond laser subtractive manufacturing is studied, and the surface roughness value is predicted. The simulation results are very close to the experimental results, and the error is only 7.63%. In addition, the model is used to study the surface movement speed and processing depth of the positive/negative defocus position: both the surface movement speed and processing depth of the negative defocus position are greater than the positive defocus position, revealing the mechanism of surface roughness reduction during the femtosecond laser subtraction process.