Current object tracking algorithms often adopt bounding box or segmentation mask to initialize the template. However，it is difficult to obtain them under limited operation time or harsh scenarios. An object tracking algorithm with point prompt is proposed，which only needs the coordinates of any image in the object segmentation mask to complete the template initialization. Firstly，SAM is adopted to perform zero-shot segmentation of the image object to obtain the segmentation mask，and the bounding rectangle is obtained and taken as the input of the ATOM tracking algorithm to complete the template initialization. The Gauss-Newton conjugate gradient method is adopted to quickly learn the template online to obtain the object locator，and the offline learning IoU prediction branch is applied to jointly complete the object tracking task. Finally，the simulation of the object tracking algorithm is completed according to the point prompt input. The simulation results show that: 1) The precision of the proposed algorithm reaches 79.7% and the AUC reaches 60.6% on the UAV123 dataset; 2) The tracking performance is better than that of the classic algorithms; and 3) The FPS reaches 87.4 frames per second，which meets the requirements of real-time tracking.