In the free space coherent optical communication system of satellite-ground link,atmosphere turbulence and tracking error will affect the mixing efficiency and the bit error rate.According to the mixing efficiency formula,both the relationship between system mixing efficiency and atmosphere turbulence intensity and the relationship between system mixing efficiency and tracking error were derived and simulatied.The results show that the mixing efficiency will decrease with the increasing of tracking error and atmosphere turbulence intensity.When the tracking error is greater than 12 μrad or turbulence intensity exceeds 10-16,the mixing efficiency is close to 0.An optical mixing efficiency model and a bit error rate model were built in atmosphere turbulence with tracking error,and the homodyne efficiency with different conditions was calculated.Atmosphere is the significant factor of mixing efficiency when tracking error is less than 2 μrad.Then,the tracking error will be as the leading factor with it increasing.According to the reference values of traditional coherent communication system,the maximum of mixing efficiency is about 67.8%,and the system receiving sensitivity is 17 photon/bit when the error rate is 10-9.