Terahertz (THz) and infrared spectra of four kinds of lactose are studied. THz absorption spectra and the dispersion of refractive index of α-lactose, β-lactose, α-lactose monohydrate, D(+) lactose monohydrate are obtained in the frequency range of 0.2～2.6 THz at room temperature by using of THz time-domain spectroscopy (THz-TDS). The experimental results show that there are the significant absorption peaks for these substances in this frequency range. These absorption peaks show a large difference in the presence or absence of crystallization water for the same lactose. This difference is expected to be used to identify these substances. In addition, the absorption spectrum of four lactose is also measured by the Fourier transform infrared spectroscopy (FTIR) within 1.8 and 10 THz. The spectral characteristics between 0.2 and 10 THz of four lactose are simulated based on the single molecular model by using of Gauss 03 software. Both of simulation and experimental results fit well at the position of absorption peaks. At the same time, the spectral characteristics of four lactose are analyzed based on the density functional theory (DFT). The mechanism of molecular motion shown from the experimental results is presented and analyzed.