The star/airborne optical remote sensing image has a wide field of view and a complex scene. It is easy to produce a large number of false alarms that are similar to the ship's target due to the impact of the shore construction and broken cloud, causing great interference to the ship's detection. Traditional marine ship detection algorithms are difficult to be effective extracting discriminative features that are conducive to detection, results in low detection rates and high false alarm rates for ships. In view of this, this paper proposes an optical ship target detection method combining hierarchical search and visual residual network from the perspective of low false alarm and low missed detection. Firstly, the land and sea area are segmented based on the texture integral map; secondly, the target candidate area is extracted by combining the multi-scale local structural features; then, the primary false alarm is removed by the layered removal strategy based on multi-dimensional visual features; finally, the visual residuals are built the network finely removes false alarms from suspected candidate areas to obtain the final detection result. Based on the GF2 remote sensing GF2 set, the algorithm proposed in this paper is tested and verified. The comprehensive detection rate of this algorithm is 92.0%, the false alarm rate is 12.58%, the average processing time is 0.5 s, the detection effect is good, the efficiency is high, and the adaptability to various scenes is good. It can achieve accurate and efficient detection and positioning of optical ships in complex environments.