Thermal effect of the gain fiber is one of the main factors limiting the power improvement of high power fiber amplifiers. Using traditional temperature measurement methods, we can only obtain the surface temperature of fiber while the core temperature cannot be detected. In this paper, the temperature of the gain fiber core in an all-fiber amplifier is measured by optical frequency domain reflectometry (OFDR). Firstly, the temperature measurement results by OFDR are calibrated and the measuring accuracy of the gain fiber temperature is verified when the amplifier is in operation. Then, the temperature distribution of the gain fiber core in the all-fiber amplifier is measured when the output power is 6 W and the results agree with the current theoretical results. The temperature measurement method proposed can provide a reference for the temperature monitoring in high power fiber lasers in the future.