The single longitudinal mode (SLM) operation characteristics of a compound ring cavity fiber laser based on cascaded fiber rings formed by single and dual optical coupler (OC) was investigated. By analyzing the resonating pass-band properties of cascaded fiber rings with respectively single and dual OC, a method to optimize the resonating pass-band properties of the dual-OC fiber ring for achieving stable SLM operation was given. With a compound ring cavity erbium-doped fiber laser based on such cascaded fiber rings, the optimization method for achieving stable SLM operation was verified experimentally. For the length of the main ring, the single-OC ring and dual-OC ring are respectively 6.8, 0.45 and 0.56 m, the operation for SLM with the fiber laser is observed to be most steadily when the coupling ratio of the two OCs in the dual-OC fiber ring is chosen to be 0.7. The experimental results are in agreement with the theoretical analysis.