The existing image generation methods based on cycle generative adversarial network have achieved excellent results in unpaired image to image translation tasks by introducing a separate generic attention module, but they also increase the model complexity and training time, and it is difficult to focus on all the details of key regions in the image, and there is still room for improvement in image generation results. To solve these problems, this paper proposes a dual special attention-mechanism guided cycle generative adversarial network architecture for unpaired image transformation translation (Dual-SAG-CycleGAN). Firstly, to improve the quality of the generated images and reduce the complexity of the network, a special attention module called SAG (Special Attention-mechanism Guided) is proposed to guide the generator. Then, to suppress the generation of extraneous noise by the generator, a discriminator based on special attention mechanism of CAM (Class Activation Mapping)is introduced. Finally, a cyclic consistency loss function for the background mask is introduced to guide the network to generate a more accurate mask map, which can better aid for image translation. Experiments demonstrate that compared with similar existing models, our proposed method can reduce the maximum number of parameters by 32.8%, speed of train 34.5% faster, and generate higher quality images with a minimum KID and FID of 1.13 and 57.54, respectively.