In order to reduce the amount of feature extraction network parameters and computation of effective convolution operator (ECO) tracking algorithm, the improved eco target tracking algorithm based on GhostNet was adopted. Firstly, the GhostNet network was used as the main feature extraction network to extract the convolution features of shallow and deep layers, and the global average pooling was adapted to downsampling convolution features to improve the image representation ability. Secondly, after interpolating the convolution feature with the manual feature, convolution calculation was performed with the current filter in the Fourier domain to realize the target localization. Finally, conjugate gradient algorithm was used to optimize the loss function of the sum of response error and penalty term to update the filter. Theoretical analysis and experimental verification were carried out on the proposed algorithm and OTB2015 and VOT2018 datasets, then the comparative experimental data of target tracking were obtained. The results show that compared with the ECO algorithm based on ResNet feature extraction network, the proposed algorithm can achieve higher precision tracking, the convolution feature extraction process reduces 95.75% of computation and 79.69% of parameters, and the tracking speed increases 160% at the same time. These results provide a reference for the research of lightweight target tracking algorithms.