Characterization of photodetectors (PDs) plays a crucial guiding role in device design and optimization. Therefore, there is an urgent need to develop measurement techniques capable of realizing the small-signal linear and large-signal nonlinear frequency responses of PDs, in order to comprehensively evaluate and understand the performance of the PDs. We propose a high-frequency characterization method for PDs based on pilot-tone modulation. When measuring the small-signal linear frequency response, pilot-tone tags are used for the decoupling of the electro-optical modulation frequency response, and the scalable high-frequency characterization of the small-signal frequency response of PDs is achieved through harmonic up-conversion of a narrowband electro-optical stimulus. When measuring the large-signal nonlinear frequency response (third-order intermodulation distortion), electro-optical modulation spurious is decoupled by means of the pilot-tone tags, thereby realizing modulation-spurious-free and frequency-swept characterization of the large-signal nonlinear frequency response of PDs. This provides a technical solution for the dynamic frequency response characterization of high-speed, high-power, and high-linearity PDs.