ObjectiveThe nitrogen-vacancy (NV) center in diamond, with its advantages of high sensitivity and high resolution, has been widely used in the measurement of magnetic fields. And the microwave frequency modulation technique can overcome the limitations of traditional ODMR in achieving high-sensitivity real-time magnetic field detection, and by reducing 1/f noise, it demonstrates higher sensitivity for magnetic field measurements, making it a research hotspot. However, most of the studies lack simulation-based theoretical model validation, and the modulation parameters in experiments are mostly set based on empirical values, constrained by hardware conditions. Therefore, to enhance the universality of the modulation-demodulation model and accurately improve the magnetic field measurement sensitivity of NV centers, we combine simulation and experimental methods to optimize the modulation parameters in order to determine the optimal modulation frequency and frequency deviation under the best sensitivity conditions.MethodsWe optimized the modulation frequency and frequency deviation parameters in the simulation to achieve the best sensitivity, based on ODMR and demodulation signal models. By analyzing the variations in the slope of the demodulation signal, we identified the optimal modulation parameters. An experimental platform was then constructed to validate the simulation results. During the experiments, we measured the slope of the demodulation signal and its amplitude spectral density under various conditions. Finally, the magnetic field sensitivity was calculated using the sensitivity formula, and the optimal modulation parameters were determined by comparing experimental data with simulation results.Results and DiscussionsThe simulation results show that the optimal modulation frequency deviation corresponding to the 10 MHz half-peak width ODMR signal is 3.55 MHz. However, due to limitations of the microwave source equipment, the best modulation frequency deviation obtained in the experiment is 8 MHz. Nevertheless, the trend of the curve's rise phase is generally consistent. For the modulation frequency parameter, the simulation cannot directly obtain its optimal value, while the experimental results show that the best modulation frequency lies within the range of 500 Hz and 1 kHz to 2 kHz. Therefore, in practical applications, the optimal modulation parameters should be referenced based on experimental results.ConclusionsBy developing the ODMR and demodulation signal models, simulations and experiments identified the optimal modulation parameters for the NV magnetic sensing system, resulting in the best sensitivity. The optimal modulation frequency deviation is 8 MHz, with the modulation frequency ranging from 500 Hz to 2 kHz. This study investigates the relationship between noise spectral density, the demodulation signal ratio, and sensitivity under different modulation parameters. Compared to previous studies, it provides a more accurate explanation of the impact of modulation parameters on sensitivity. However, future simulations require further optimization to improve the generality of the simulation model.