An image shadow removal algorithm based on minimum noise fraction (MNF) and a generative adversarial network (GAN) is proposed to improve the shadow removal effect. The algorithm takes GAN as its basic framework, introduces condition information into the generator and discriminator respectively, and adopts the multitask mode of end-to-end joint learning. The generative network adopts the encoding-decoding structure, and the discriminant network adopts the Markov discriminator structure. Additionally, the proposed algorithm uses MNF to restore the shade-free image after graying the noise-eliminating image with the shadowed image. Therefore, our network can focus on single feature embedding after the change in MNF instead of the traditional cross-task shared embedding. Experimental results indicate that the proposed algorithm can increase the mean structural similarity (SSIM) to 0.9780 and decrease the mean root mean square error (RMSE) to 9.8717 on the specified dataset. Both visual and statistic comparisons confirm that the proposed algorithm is better than other algorithms.