To realize the on-axis tracking in an electro-optical tracking servo system, a nerve net Extreme Learning Machine (ELM) was adopted to obtain the velocity and acceleration of the target motion through learning, training and fusing data. Through algorithm optimization, the amount of computing is reduced by about 50 percent in the ELM system, and the period of computing is 3.5 ms, so as to meet the real-time of electro-optical tracking system. The fused target information was filtered through the 6 step Butterworth filter. The simulation result verifies that the predicted target velocity error is about ± 3 (°) / s at the peak of velocity when the target velocity is 50 (°) / s and the target acceleration is 30 (°) / s2. Finally, the electro-optical tracking system was used to track an optical dynamic target equipment. When the system is in a double-loop and on-axis control, tracking results show that the maximal system tracking error has decreased from 11.35′to 0.88′, and the random error decreased from 8.2″to 7.6″. As compared with other control methods, the proposed method has better real time performance and higher accuracy, and improves the system tracking precision significantly.