In infrared and visible image fusion, fused images often suffer from insufficient prominence of significant targets, inadequate expression of visible light information, edge blurring, and local information imbalance under uneven lighting conditions. To address these issues, an image fusion algorithm that combines attention mechanisms and equilibrium loss, termed the depthwise separable, squeeze-and-excitation, and equilibrium loss-based convolutional neural network (DSEL-CNN), is proposed. First, a depth-wise separable convolution is used to extract the image features. Subsequently, a fusion strategy is used to apply the squeeze-and-excitation attention mechanism to enhance the weight of effective information. Finally, an equilibrium composite loss function is utilized to calculate the loss of the fused image to ensure balanced information. A comparison of the fusion generative adversarial network (FusionGAN), DenseFuse, and four other fusion algorithms on the TNO and multi-spectral road scenarios (MSRS) public datasets showed that the proposed method achieved the highest improvements in mutual information (MI), visual information fidelity (VIF), and edge retention index (Q^abf) by 1.033, 0.083, and 0.069, respectively. Experimental results demonstrate that the proposed algorithm outperforms six commonly used fusion methods in terms of visual perception, information content, and edge and texture preservation in fused images.