In unstructured indoor scenes, motion blur and lighting changes affect the images captured by rapidly moving drones. This leads to certain limitations in the feature extraction and real-time processing of traditional simultaneous localization and mapping (SLAM) algorithms, which then affect the positioning accuracy and operational efficiency of the drones. Considering this, a visual-inertial SLAM algorithm that combines point and line features with step-by-step marginalization is proposed. A reverse optical flow module for point feature tracking is introduced to improve the accuracy of point feature tracking through the bidirectional verification and fusion of forward and reverse optical flows. Simultaneously, line encoding feature selection and homogenization modules are integrated into the line segment detection model, and directional thresholds and spatial distance constraints are employed for screening to improve the accuracy and robustness of line feature detection. The improved marginalization strategy is used to separately optimize the visual and the residual information of inertial measurement unit. This reduces the dimensionality of the information matrix and enhances the computational efficiency and resource utilization of the SLAM system. The experimental results demonstrate that the proposed method reduces the root mean square error of the absolute trajectory error by 15.79% and 7.99% compared with PL-VINS and EPL-VINS, respectively. This indicates a significant improvement in the accuracy of the unmanned aerial vehicle pose estimation.