The coupling and propagation characteristics of fundamental modes and high-order modes in the few mode fiber Bragg grating was numerically analyzed and simulated. It is indicated that the shifts of the Bragg wavelength corresponding to the coupling between the fundamental mode and the backward high-order mode is obviously larger than the shift of the Bragg wavelength corresponding to the coupling between the forward and backward fundamental modes. The few mode fiber Bragg grating was fabricated, and the variations of Bragg wavelengths corresponding to fundamental modes coupling and fundamental-high-order modes coupling with external refractive index and temperature were measured. The experimental results agree with the theoretical analysis. Therefore by the means of Bragg wavelength difference calculation, the temperature induced fluctuation is overcome, which presents a new ray to measure the external refractive index change of fiber Bragg gratings.