Laser frequency stabilization is an essential technology in various applications, including optic communication, quantum metrology, and space-borne gravitational wave detection. Conventionally, the laser for use is frequency stabilized to an ultra-stable reference cavity. However, the frequency noise due to the cavity length noise of the reference cavity still limits the application of the ultra-high-precision measurement of space-time strain with a magnitude of the order of 10^-18-10^-20 in the frequency range of mHz-Hz. Before application in such high-precision measurements, extensive suppression of frequency noise is mandatory. The thermal noise of the reference cavity is typically a predominant source that necessitates reduction. Higher-order laser transverse modes, characterized by a larger transverse intensity distribution, yield a lower spatially averaged thermal noise. The integration of higher-order modes into frequency stabilization with ultra-stable cavities has not been exhaustively explored in the literature.

We first introduce a generalized noise model for frequency stabilization based on a reference cavity. Then we apply this model to higher-order mode reference cavities to scrutinize the influence of the mode transformation noise on the final frequency noise. By controlling a series of technical noises of the cavity such as vibration, temperature, and electronic noise, the thermal noise and shot noise emerge as the two dominant noise sources. According to the fluctuation dissipation theorem, we calculate and compare the thermal noises of higher-order Hermite-Gaussian (HG) and Laguerre-Gaussian (LG) modes, using parameters of a regular ultra-stable cavity. We also delve into the mode coupling efficiencies of different LG_p,0 modes based on the scheme of mode-mismatching for mode transformation. The shot noise, attributable to the limited mode coupling efficiency, is also taken into account. By compromising the thermal noise and shot noise, we propose some optimal mode orders for achieving minimal total noise.

According to the noise transfer model, the noise introduced by the mode transformation is non-negligible, particularly in the presence of a mode-filtering cavity (Figs.1 and 2). Consequently, we implement a simple mode transforming scheme based on the mode mismatching. The mode coupling efficiencies varying with mismatching parameters for higher-order LG modes are given. The thermal noise for both higher-order HG and LG modes is delineated, demonstrating a decrease in noise with an increasing mode order. Owing to the better spatial symmetry, the LG mode exhibits lower thermal noise for the mirror substrate compared to the HG mode at equivalent mode orders (Fig. 4). The frequency thermal noise across the entire reference cavity is calculated (Table 2). When the ULE substrate is changed into fused silica, the fundamental mode thermal noise is reduced from 0.096 Hz/Hz^1/2 to 0.029 Hz/Hz^1/2. With a fused silica substrate, the reduction rate of thermal noise of the LG_10,0 mode at 1 Hz is 16% compared to the fundamental mode. Considering the shot noise, the lowest total noise for the LG_p,0 mode in the range of 0≤p≤25 can reach 0.022 Hz/Hz^1/2 at 1 Hz, marking a 23% reduction compared to the total noise of the fundamental mode. More results involving different mode order, input optical power, and analyzing frequency are listed (Table 3).

We present a general noise transfer model for laser frequency and extend its application to higher-order mode-based frequency stabilization. The noise associated with mode transfer warrants careful consideration. The thermal noise decreases with increasing mode order. The thermal noise of LG mode is lower than HG mode under the same condition. The dissipation due to the limited mode transfer elevates the shot noise, a factor that should be contemplated for low-power injection and can be weighed against the reduced thermal noise of higher-order modes. The total noise is influenced by various parameters and can be optimized by considering the mode order.