Boron is one of the most common acceptor elements in diamond that forms a shallow level at 0.37 eV above the valence band, and thus boron-doped diamond is considered to be an ideal p-type semiconductor material. For the chemical vapor deposition synthetic boron doped diamond, the distribution of boron impurities in the crystal is very inhomogeneous, and the Raman intensity is so dependent on the test position that the repeatability is poor. However, the boron content in the same growth sector of high temperature and high-pressure synthesized diamond changes slightly. In this paper, the photoluminescence properties of irradiated defects in high temperature and high pressure synthesized boron-doped diamond were investigated by low temperature photoluminescence spectroscopy, and the growth sector dependence of irradiated defects was interpreted by the theory of crystal growth.