A scheme for phase-matched passive-decoy-state quantum key distribution is proposed based on the heralded single photon source. In the scheme, both communication parties only need to generate one-intensity signals. The detection results of the third party are divided into four sets according to the response of the local detectors of both communication parties. These four sets not only serve as the signal state and the decoy state, but also jointly participate in parameter estimation and key generation. Thus, the difficulty of system implementation is reduced and the performance of the scheme is improved. The simulation results show that the maximum safe transmission distance of the phase-matched passive-decoy-state scheme can reach 552 km, and the performance is close to those of the existing phase-matched active-decoy-state schemes. Moreover, there is no need to generate decoy states actively. To some extent, this scheme overcomes the drawback that the phase-matched active-decoy-state scheme relies on the detection efficiency heavily and makes the performance more stable. As the data length decreases, the transmission performance of the scheme is declined. Even if the data length drops to 10 ⁷, the maximum safe transmission distance of the scheme can still reach 507 km.