The nonlinear absorption properties of borate glass doped with CdS0.2Se0.8 nanocrystal (RG665) were investigated using Z-scan and pump-probe techniques with femtosecond pulses. It was found that RG665 showed strong nonlinear absorption under 130 fs laser pulses at 800 nm wavelength. By theoretically analyzing the experimental results, the nonlinear absorption of RG665 included two parts: two-photon absorption and the excited state absorption induced by two-photon absorption at 800 nm wavelength. The two-photon absorption coefficient was obtained to be about 0.05 cm/GW, the excited state absorption cross section was σe=3×10-23 m2, and the electron lifetime on the low energy state and bottom in conduction band was 13 ps and 210 ps respectively. The research results certify that the borate glass doped with CdS0.2Se0.8 nanocrystal is a good kind of nonlinear optical material.