Based on 4×4 matrix including meridian and latitude, the roundtrip matrix of nonplanar cavity was established. According to the matrix, the q parameters matrix was calculated, the complex astigmatism in some nonplanar cavity ring laser gyro (RLG) was analyzed, and the spot radius of Gaussian beam at any location was obtained theoretically. Then the spot radii of the outcavity Gaussian beam were measured by CCD, and the Gaussian beam waist was obtained by hyperbola fitting. The result was consistent with the theoretical one. According to spot radii, diaphragm radii and location, the diffraction loss was obtained. The designed diaphragm located at the Gaussian beam waist is circular, and the ratio of diaphragm radius to waist radius is about 2. The designed scheme guarantees the nonplanar cavity RLG works with single TEM00 mode.