Aiming at the problem of missing detection in the process of small target detection due to insufficient semantic information of shallow features, a multi-layer feature fusion improved single shot multi-box detector (SSD) method is proposed. Firstly, deepwise separable convolution (DSC) is added to the shallow network, and the shallow semantic information is strengthened by channel-by-channel convolution and point-by-point convolution. Then the features of deep network and shallow network are refined utilizing deconvolution and dilation convolution. Finally, the attention mechanism is added to the deep network to enhance the detection ability of small targets. Verified on VOC2007 and VOC2012 data sets, the average detection accuracy is improved by 5.56% compared with the benchmark algorithm and 4.25% compared with other advanced algorithms. The experimental results show that the proposed refined semantics and enhanced perception methods can achieve the purpose of improving the detection accuracy of small targets.