In coherent detection system, the matching of signal beam and local oscillator and the beam characteristic have important effects on the detection efficiency. The light-intensity distributions of Gaussian beam and flattop beam are simulated in Matlab respectively, and the mathematical feature of the two spots are provided. Based on the acousto-optic deflection effect, the effect of spot size matching and signal beam parallel moving along light axis on the coherent detection efficiency is analyzed by building overlay model between Gaussian beam and rectangle beam. The results show that, with the assurance of beam collimation and equivalence in the total energy of two spots, we can get the optimal overlapping efficiency of 79.6% when the energy density of rectangle spot is 0.8 times as the central energy density of Gaussian spot. The non-collimation of the beams will reduce the detection efficiency when the signal beam is mixed with the local-oscillator beam on the optoelectronic sensor surface. But while rectangle beam is mixed with Gaussian beam, the variation of detection efficiency tends to become gentle when beams deviate from optic axis within ±0.5 mm.