In the process of extracting buildings from high-resolution remote sensing imagery, convolutional neural networks (CNNs) struggle to balance global information, edge details, and underlying texture information, leading to blurred edges and incomplete results. To address this, we propose a dual-branch remote sensing building extraction network based on texture enhancement, named texture enhancement and Outlook attention U-shaped network (TEOA-UNet), which combines shallow CNNs with Transformer networks. First, to learn global contextual information while maintaining focus on local buildings, we introduce the Outlook attention mechanism. Then, to improve the model's perception of building edges, we utilize an edge aware module (EAM) to encourage the learning of edge information. Finally, to enhance the model's sensitivity to low-level textures, we put forward a texture enhancement branch based on a shallow convolutional network to strengthen the model's learning capability of low-level features. Experimental results on the Massachusetts, WHU, and Inria building datasets demonstrate that TEOA-UNet performs well in extracting buildings from remote sensing imagery with different resolutions and scenes, effectively improving the precision and completeness of building edge segmentation. The F1 score reached 88.54%, 95.22%, and 90.94% on the aforementioned datasets, respectively, which is a respective increase of 1.72 percentage points, 0.49 percentage points , and 0.23 percentage points over the Baseline model SDSC-UNet. These results indicate that TEOA-UNet possesses high extraction accuracy.