Metal artifact leads to reduced quality of computed tomography (CT) images, which can severely degrade segmentation accuracy. To address this problem, a segmentation network is proposed to resist metal artifact interference. This network uses composite connected dual-stream encoder structure with two backbones for feature extraction from disturbed and undisturbed CT images, respectively. The composite connection structure integrates the features extracted by the encoders on the two backbones. A Transformer-based focal self-attention (SA) mechanism block is developed to encode global multi-scale information. The training process of the network is optimized using hybrid loss and ancillary supervision. The experimental results show that this network on metal artifact data could reach 86.40%, 93.11% and 90.76% in average Dice coefficient, MIoU and Recall, respectively. The network has great anti-metal artifact interference effect in semantic segmentation for CT images, and achieves high segmentation accuracy without artifact reduction.