Spectral processing holds immense significance in the research and application of optics. Various devices and instruments have been developed for specific tasks, including spectral filtering, shaping, analysis, and wavelength demultiplexing. However, advanced multitasking spectral processing equipment has been lacking. In this work, we have designed a diffraction neural network for spectral filtering, comprising a phase-modulated diffraction layer and a detection layer. During the training process, wavelength parameters were incorporated to achieve the processing of broadband signals. By designing a tailored loss function, we gained control over the output spectrum. Taking the broadband signal in the visible light band as an example, we achieved both single and dual passband spectral filtering, with adjustable central band widths and relative intensities. This research demonstrates that optical diffraction neural networks can effectively handle broadband spectra, laying the foundation for tackling more complex spectral processing tasks in the future.