Considering that the standard (Particle Swarm Optimization) PSO algorithm has low iteration efficiency during later period and may trap to local optimum, Immune Clone (IC) principle is introduced into the PSO algorithm. The antibodies can be regarded as the particles. According to the degree of affinity, the clone selection, clone suppression, and high-frequency mutation are performed, which can enhance the diversity of particle swarm and the capability of global searching. The optimal compensation experiment is performed in the 40 Gb/s transmission system, in which the compensation time required was about 71 ms. The opening of signal eye diagram has been improved obviously after compensation. The experimental results demonstrate the effectiveness of the algorithm proposed.