Using a pump laser coupling the hyperfine transition of D2 line in a cesium cell, the optical pumping spectra are demonstrated by detecting the absorption of the co-propagational probe beam with the identified frequency. The peaks or dips are related to the optical pumping where the pump laser and probe laser coupling different hyperfine transitions with a frequency difference. Employing a locking loop and the optical pumping spectrum, a scheme for frequency offset locking of two diode lasers with the reference frequency based on the hyperfine splitting is presented. The frequency offsets can be the intervals between the hyperfine levels of the 62P3/2 excited states or the 62S1/2 ground states from hundreds megahertz to 9 GHz.