In this paper, a fast and accurate ship target detection method wa proposed for ship detection in optical remote sensing image. The “coarse-to-fine” strategy was applied, which contains mainly three stages: the candidate regions extraction, building the candidate regions’ descriptor and the candidate regions discrimination by reducing the false alarms to confirm the real ship targets. In the first stage, first the initial saliency map was extracted by the maximum symmetric surround method, which was based on the visual attention mechanism, and updated according to the local similarity via a updating mechanism of cellular automata; then, the final saliency map was segmented by OTSU algorithm to obtain binary image; finally salient regions were extracted from the segmented binary image, and filtered roughly by the ship objectives' geometric features. In the second stage, a new descriptor, named edge-histogram of oriented gradient (E-HOG), was proposed to describe the ship target. The E-HOG feature was an improvement of the traditional HOG feature, based on the inherent characteristics of the ship targets. Compared to the traditional HOG feature, the E-HOG feature limited the statistical scale into the edge of the salient regions, for the purpose of reducing the influence of the variability of oriented gradient, and reducing computation complexity. On one hand, the descriptor could discriminate the ship objectives from others like cloud, islands and wave; on the other hand, the descriptor was insensitive to the size of the ship objectives, which reinforce the robustness of the approach. In the third stage, the AdaBoost classifier was used to confirm the real ship targets by eliminating the false alarms. We intercept 517 positive samples and 624 negative samples from the remote sensing images to train the AdaBoost classifier. The size of these training samples ranges from 20 pixel×10 pixel to 200 pixel×120 pixel, where the positive samples include different types of ship targets, and the negative samples include non-ship targets such as clouds, islands, coastlines, waves and sea floating objects. In this paper, the detection time is 2.386 0 s for the 1 024 pixel× 1 024 pixel remote sensing image, the recall rate is 97.4%, and the detection precision is 97.2%. Experiments demonstrated that the detection performance of the proposed method outperforms that of the state-of-the-art methods, and it can meet the actual engineering requirements in the detection time and detection precision.