To address the challenge of balancing the computational accuracy and efficiency in adaptive finite element meshing, this study proposes a GTF-Net model based on the attention fusion mechanism. The model combines the graph attention network with the Transformer architecture, dynamically couples local geometric features with the global physical field through a multi-head cross-attention module, and enhances the representation of singular fields and complex boundaries. The verification of two case studies of waveguide transmission and Bessel equation shows that compared with the traditional Scikit-FEM (skFem) method, GTF-Net improves computational efficiency while reducing the standard deviation of gradient error by 85.9% and 23.8%, respectively. The results show that the model significantly improves the fit between mesh distribution and physical field changes through nonlinear feature mapping, providing a novel deep learning solution for adaptive mesh optimization in engineering calculations.