The optical remote sensing image sea surface ship target detection is susceptible to cloud, island, sea clutter, shadow and other complex factors interference. At the same time, due to the large width of satellite remote sensing images, if the real-time detection and hardware transplantation needs to be met, the amount of computation and portability of the algorithm should be considered. In order to meet the needs of practical engineering, this paper proposes an adaptive fast ship target location and detection method based on visual saliency under complex background. Firstly, the algorithm adaptively obtains the global optimal scale based on image gradient, and obtains the global significance region through the spectral residual significance model. For the regions with poor segmentation effect due to global threshold in significant regions (Complex regions), the local complex regions are screened out by designing contour and other shape features, and the saliency map is further calculated. Then, the processing results are fused with the original significance map to obtain the final extraction results of suspected regions. Finally, the candidate regions are further distinguished using support vector machine. The results show that the proposed algorithm can effectively detect ship target areas with different sizes and directions under complex background, and the detection accuracy of the algorithm is 91.4% and the recall rate is 91.2%, which is better than most of the similar algorithms and close to the accuracy of the deep learning algorithm. At the same time, in terms of algorithm volume, the calculation amount and the number of parameters of the algorithm in this paper are far lower than those of most deep learning frameworks, which is more suitable for hardware transplantation. Meanwhile, the algorithm has stronger mobility and is easy to modify and maintain.