Frequency modulated continuous wave (FMCW) laser ranging, which is critical to the laser industry, faces challenges in achieving high range accuracy due to the non-linearity of the laser frequency modulation. Traditional calibration methods based on iterative algorithms and photoelectric phase-locked loops are limited by their dependence on prior knowledge and often ignore to exploit the full potential of the data generated by the ranging system. In this paper, we introduce a novel deep learning approach using digital twin (DT) technology, demonstrating the potential in fields such as aerospace and optics. We propose a neural network (NN) model integrated with the soft actor-critic (SAC) control algorithm operating within a DT framework to effectively calibrate the nonlinear errors during the frequency-swept process. This innovative framework has significantly improved the performance of FMCW laser ranging, with the side mode suppression ratio (SMSR) of the beat signal increasing from 3.2 dB to 17.3 dB and the full width at half maximum (FWHM) decreasing from 15.82 kHz to 2.64 kHz. These improvements not only enhance the accuracy and quality of FMCW ranging system, but also pave the way for new research directions and insights in the broader fields of optical society.