Most existing remote sensing object detectors use detection heads with parallel branches, which leads to noticeable misalignment between the predictions of the two branches. To address this problem, this study propose a branch feature alignment network (BFA-Net). First, a branch alignment module (BAM) is used to enhance feature interaction between the classification and regression branches by learning branch alignment features. Simultaneously, the module dynamically adjusts classification features and sampling positions to achieve alignment between the two branches. Then, during the label assignment, the model dynamically evaluates the classification and regression quality of samples via the designed alignment index, and the sorting-based assignment strategy screens positive samples from coarse to fine to ensure that different scaled objects receive sufficient supervisory information. Extensive experiments on commonly used datasets such as DOTA-V1.0, DOTA-V1.5, and DIOR-R, show that the BFA-Net achieves mean average precision values of 75.36%, 68.26%, and 65.50%, respectively. The proposed method has obvious advantages over other advanced algorithms in terms of detection performance and detection efficiency.