Large integer multiplication is the most time-consuming operation during encrypted data calculation. It is particularly important to improve large integer multiplier speed in machine learning based on fully homomorphic encryption. A design scheme of high speed 768 kbit large integer multiplier was proposed in this paper. The critical component 64k-point finite field number theory transform (NTT) was decomposed into 16-point NTT. And through dichotomy processing, the pipeline architecture of 16-point NTT was refined. To increase the speed of the multiplier, addition and shift were adopted to achieve the modular-subtraction unit, and data interaction was accomplished by using an efficient non-conflict address algorithm. The multiplier was deployed on the Altera Stratix-V FPGA development board. And the experimental results showed that the circuit had a working frequency of 169.23 MHz and took 0.317 ms to complete the large integer multiplication. Comparing with the state-of-the-art works, our speed performance was improved by 1.2 times to 7.3 times.