In this study, two kinds of strain-compensated MQW SESAM structures with a total thickness of 150 and 300 nm of InGaAs absorber layer are designed. The epitaxial material of MQW and DBR is grown by MOCVD. The features of epitaxial material are characterized by photoluminescence spectrometer, high-resolution X-ray diffractometer, and spectrophotometer; its growth parameters are optimized based on the characterization results. Further, epitaxial materials of SESAM are grown and characterized using the same test based on MQW and DBR epitaxial growth parameters. The two SESAMs are applied to the linear cavity Yb-doped fiber laser, and stable mode-locking is achieved. The experimental results show that SESAM with the thickness of the InGaAs absorber layer of 300 nm is more likely to achieve stable mode-locking and obtain pulse output with narrow pulse width.