To detect ship targets accurately, a new method to detect ship targets on sea surface was proposed based on multi-feature and multi-scale visual saliency. Firstly a scale-adaptive top-hat algorithm was used to suppress the interference of clouds and oil. Then, the double-quaternion images are constructed by using double-color spatial features and edge features to detect the saliency of ships. This method makes full use of the double quaternion images, so it can be operated at the same time in a number of channels, and can save operation time to guarantee the characteristics of different scale characteristics. Furthermore, the method also uses the character that the human eye focused on the different targets for image with different sized in implement of the up-down sampling to avoid the leak overlapping in image detection. When the last saliency map is obtained, the ships were segmented to ensure the target location by using the OTSU algorithm, and then the ship target was marked and extracted in the original image. The experiments were analyzed in the several sea conditions. Experimental results show that the algorithm eliminates the interference of cloud, fog and oil pollution and ship targets are detected accurately. With this algorithm, true rate iss 97.73%, and the false alarm rate as low as 3.37%. Compared to other frequency domain saliency detection algorithms in ship detection, this algorithm has obvious advantages.