Taking panda-type polarization-maintaining fiber as the object, the influence of drawing tension and heat treatment on its stress birefringence is systematically studied. In the multiphysics finite element modeling, we introduced the drawing tension during the fiber preparation process into the PMF photoelastic model, and we summarized the numerical law that the birefringence of the panda-type PMF decreases linearly with the increase of the drawing tension. We used white light interferometry to experimentally measure the effects of different heat treatment conditions on the birefringence of PMFs. The structural relaxation induced by boron-doped silica glass at high temperature causes the volumetric shrinkage of the Stress Applying Parts (SAP), which increases the birefringence. The deformation caused by the structural relaxation is obtained from the variation of the vacancy defect concentration of the glass network, and then the birefringence after the annealing of the fiber is calculated. The numerical simulation results are consistent with the experimental measurements. This work provides a theoretical basis for the design, preparation and birefringence control of PMF.