A high speed DDC system with 1, 2, 4, or 8 times optional decimation ratio was designed by analyzing the principle of the common digital down converter system. The mixers and filters of the system were optimized. A radix-4 Booth encoder and a 4-2 compressor were used in the mixers to shorten the critical path. A CSD coding based on the sharing of sub-expressions and Horner rule was utilized at the filters to reduce the hardware consumption. The digital down-conversion system was designed for a four-channel 560 MHz 14-bit time-interleaved analog-to-digital converter (TIADC), and the function verification was completed based on FPGA. The results showed that the SFDR of the I/Q signals was above 90 dB when the input signal frequency was 380 MHz and the decimation ratio was 8.