The concentration of Mn and Si in different kinds of steels was determined by laser-induced breakdown spectroscopy (LIBS). The multivariate quadratic nonlinear function was adopted for calibration. Samples including common alloy steels, stainless steels and carbon tool steels were analyzed. The matrix effect was serious because of large difference in compositions of different kinds of steels and strong line overlaps in steel spectra. Therefore, the common calibration methods that only use one analytical line to calibrate the complex chemical compositions of alloy steels will lose much information. The multivariate calibration methods, however, can utilize more information of spectra, successfully reduce the matrix effect and improve the measurement repeatability and accuracy of LIBS. Compared with the common calibration method based on one analytical line, the relative standard deviation was reduced from above 20% to below 10%, and the accuracy was increased by more than 5 times for Mn and more than 6 times for Si.