The block-averaging reference phase estimating algorithm is able to reduce the variance of phase compensation noise effectively in the continuous-variable quantum key distribution. In order to optimize the performance of the block-averaging reference phase estimating algorithm, according to the mathematical model of phase drift variation, the mean square error characteristics of the algorithm are analyzed theoretically, and the optimal block length is obtained. Simulation results show that the variance of phase compensation noise can be reduced by 72% under the optimal phase estimation, so that the distance for generating secure key can be improved most significantly.