In order to improve the accuracy of image segmentation without changing the structure of original semantic segmentation models, an approach is proposed to train Semantic Segmentation models by using Generative Adversarial Network(SS-GAN). There are three steps related to this work: constructing the generative model of Fully Convolutional Network(FCN) structure to segment image preliminarily; constructing the adversarial model which can learn the high-order relationship between pixels and training it to improve the learning ability of generative model; adding the anti-loss to assist generative model training, encouraging generative network to learn the relationship between pixels independently. Experiments on Pattern Analysis, Statistical Modeling and Computational Learning (PASCAL VOC) and Cityscapes datasets show that the proposed method achieves better performance than the existing advanced methods, and improves Intersection over Union(IoU) by 1.56%/1.17% and 1.93%/ 1.55%, respectively.