In order to effectively solve the problem of insufficient training data base on studying the characteristics of infrared dim dots, an improved algorithm for tracking infrared dim dots build upon meta-learning was adopted. Firstly, the pre-training tracking model was used to apply the meta-learning to the convolutional neural network. The general representation of the target were obtained through the offline training on the static infrared image data set, accordingly to obtain the specific representation of the infrared point-like target by using the initial frame target position.The target motion model was predicted by kalman filter algorithm and the optimal search area was obtained. In addition, in order to solve the problem of target loss caused by occlusion, the re-detection mechanism was studied. Theoretical analysis and experimental verification were carried out, with tracking accuracy up to 90%. Concluded that this approach is capable of tracking the infrared dim dots more accurately than other tracking algorithms in the same data set. This research provides a reference for the application of machine learning algorithms in the tracking of infrared dim and small targets.