The asymmetry of gain and time delay in two input paths of the balanced detector will affect the performance of the optical analog-to-digital conversion (PADC) system based on balanced detection. A mathematical model of the PADC system based on balanced detection is built. The relationship between the effective number of bits (ENOB) of the system and the asymmetry of gain and time delay in two input paths of the balanced detector is given. The effect of asymmetry of gain and time delay on ENOB of the system is analyzed by simulation, and corresponding confirmatory experiments are carried out. The results show that ENOB of the system decreases with the increase of relative gain and relative time delay in two input paths under the same modulation depth. When the gain and the time delay in two input paths of the balanced detector are exactly the same, ENOB of the system decreases with the increase of modulation depth. When the gain and the time delay in two input paths of the balanced detector are different, ENOB of the system increases at first with the increase of modulation depth and then decreases.