To address the challenges of inaccurate lesion region localization and blurred segmentation boundaries in colorectal polyp images, this paper proposes a colorectal polyp segmentation algorithm that integrates Transformer and dual graph convolutional network (dual-GCN) techniques. First, the Transformer encoder is utilized to parse multi-scale feature information within the images, establishing long-distance dependencies between pixels. Second, a dual graph convolution semantic detail injection (DGSDI) module is designed to integrate the spatial and structural information of deep features, enhancing boundary representation. Third, a local pyramid attention (LPA) module is employed to calculate global and local attention weights on shallow features, precisely localizing the lesion regions while suppressing irrelevant information. Finally, a dynamic feature fusion (DFF) module is introduced to adaptively aggregate multi-scale features, improving the handling of pathological images with large scale variations and irregular shapes. Experimental results on four public datasets, Kvasir, CVC-ClinicDB, CVC-ColonDB, and ETIS, show that the Dice coefficients are 0.924, 0.943, 0.817 and 0.813, respectively, and the mean intersection over union (mIoU) scores are 0.871, 0.896, 0.733 and 0.732, respectively, verifying the effectiveness of the proposed algorithm.