The multiscale features extraction method in infrared and visible image fusion algorithms loses detail information. Existing fusion strategies also cannot balance the visual detail and infrared target features. Therefore, a fusion network via a dilated convolution and dual-attention mechanism (DCDAM) is proposed. First, the network extracts the original features from the image through a multiscale encoder. The encoder systematically aggregates the multiscale context information through dilated convolution instead of using downsampling operator. Second, a dual-attention mechanism is introduced into the fusion strategy, and the original features are input into the attention module for feature enhancement to obtain the attention features. The original and attention features were combined into the final fusion feature. The mechanism captured the typical information without losing details and suppressed the noise during the fusion process. Finally, the decoder used a full-scale jump connection and dense network to decode the fusion features and generate the fused image. The experimental results show that the DCDAM is better than other representative methods in qualitative and quantitative index evaluations and has a good visual effect.