It is essential for the development of photonic crystal fiber based fiber grating devices to study the transmission spectral characteristics of the fiber grating in photonic crystal fiber. Based on the coupled mode theory and beam propagation correlation function mode-solver, the transmission spectrum of a typical fiber Bragg grating in photonic crystal fiber is studied. The transmission spectra of fiber Bragg grating in photonic crystal fiber and conventional fiber are compared. The influence of the structure change of the fiber cross section on the transmission spectrum is simulated, and derived qualitatively. It is found that the loss peaks caused by cladding mode resonances have similar strengths with those caused by the bidirectional core mode coupling, and the spacing of cladding mode resonances is larger than that in a conventional fiber. Meanwhile, the rule of effects of duty ratio of the air holes and the layer number of the air holes on the transmission spectrum is given.