Angular velocity measurement error exists in swaying motion Fiber Optic Gyroscope (FOG), which constrains FOG strap-down system’s performance under severe angular motion. Based on automatic control theory and FOG’s closed-loop control scheme, the reason for FOG’s swaying error was analyzed and found out that the angular acceleration is the main factor for FOG’s swaying error. Then a simplified FOG’s swaying error model was built up and the compensation algorithm was proposed. Finally, according to the equivalent input principal, it is experimented to test and compensate FOG’s swaying error under different swaying frequencies. Test results show that the compensation algorithm has the capability of reducing FOG’s swaying error by an order of magnitude.