To address the low accuracy of PCBA defect detection caused by the lack of 3D morphological information, an MGT-Fusion defect detection method incorporating both RGB texture and depth image features is proposed. The proposed method enhances traditional RGB texture image-based defect detection by integrating depth images to capture richer spatial and morphological details. Gate fusion module (GFM) and Transformer encoder fusion module (TFM) are designed to effectively fuse features from the two modalities. The GFM employs a dual-gated attention mechanism to perform shallow fusion and extract complementary features, while the TFM leverages a self-attention mechanism to capture global correlations and achieve deep fusion. To support the method, high-precision automatic optical inspection equipment based on a structured light phase-shift fringe technique is developed, enabling the acquisition of both depth and RGB images for constructing a comprehensive PCBA defect dataset. Experimental results demonstrate that the proposed method achieves a mean average precision of 99.89% on the dataset. Furthermore, comparative and ablation experiments are conducted to assess the individual contributions of the GFM and TFM, confirming the effectiveness and advancement of the overall approach. This method offers a valuable reference for improving surface defect detection in PCBA applications.