Aiming at the problems such as high complexity and high labeling cost of the traditional directed target detection algorithm based on rotating frame labeling, a weakly supervised directed target detection algorithm LSK-EFPN based on cross-space and multi-scale is proposed, which can infer the rotating frame information of the target by using the horizontal frame labeling information. The directed target detection in complex remote sensing scenarios is realized. In order to improve the network detection ability, the algorithm uses LSKNet network to extract the prior background features of input images, and adds a cross-space multi-scale attention module to capture cross-space feature regions. Finally, CIoU is used as a scale-constrained loss function to reconstruct the consistency loss. The experimental results show that the average accuracy of LSK-EFPN on DIOR data set of remote sensing scenes reaches 61.7%, which is 4.7% higher than H2RBox algorithm, providing a new technical solution for directed target detection scenes based on horizontal box marking.