An image conversion method based on generative adversarial networks is proposed in order to solve the problem of different image acquisition costs in different spectral segments. In the conversion process, the image outline does not change into the starting point in the range that can be distinguished by the naked eye. Firstly, the generator and discriminator are trained alternately through pairs of training data, and the loss function is optimized until the Nash equilibrium of the model is reached. Then the test data are utilized to detect the trained model, to check the conversion effect, and to evaluate the conversion effect from the subjective observation and objective calculation of the average absolute error and mean square error. Through the above process, the conversion between different spectral images is finally realized. Among them, the generator learns from U-Net architecture; the traditional convolution neural network architecture is used by the discriminator; and L1 loss function is increased to ensure the integrity of high and low frequency features before and after image conversion. In this paper, the conversion between infrared image and visible image is taken as an example to carry out the experiment. The results show that the conversion between infrared image and visible image can be well realized through the generative adversarial networks designed in this paper.