In order to alleviate the scarcity of image datain infrared small dimtarget detection, an image augmentation algorithm based on image-to-image translation is proposed. This method is a two-stage image generation algorithm. First, additional visible image is introduced, and the mapping between visible and infrared image is learned by U-GAT-IT model, converting the visible image into infrared background image. In order to solve the problem of overfitting in image translation, a channel regularization method is proposed to make the channel information of infrared and visible images consistent Then, an auto-encoder based on vision Transformer structure is designed to learn the distribution characteristics of infrared small targets and synthesize small targets on the obtained infrared background images in the way of masking and reconstructing. The method is trained and tested on SIATD data sets. The experimental results show that the proposed data augmentation algorithm can improve the detection indexes on three models to a certain extent, among which the AP index of the YOLOv3 model increases by 1.37%, which proves the effectiveness of the proposed data augmentation algorithm and can improve the performance of the target detection model in the infrared small dim target detection task.