To compensate for atmospheric turbulence-induced wavefront distortion in coherent free-space optical communication, we develop an adaptive optical system based on the improved simulated annealing algorithm. The system seeks to optimize mixing efficiency and reduce the bit error rate, ultimately enhancing overall system performance. First, we describe the structure of a coherent optical communication system without a wavefront adaptive optics component, focusing on key parameters such as mixing efficiency and bit error rate. Next, the paper implements a detailed explanation of the working principles of the improved simulated annealing algorithm and its application in adaptive optical systems. To validate the proposed algorithm's effectiveness, numerical simulations are performed and compared against traditional algorithms. Finally, real-world data is collected from an experimental platform to further assess the algorithm's performance. Experimental results demonstrate that, in comparison to the standard simulated annealing algorithm, the improved simulated annealing algorithm reduces the iteration count by 50%, decreases the bit error rate to 10^-9, and increases the mixing efficiency to 0.9. Overall, the improved simulated annealing algorithm effectively reduces the iteration count in traditional adaptive optical systems, enhances wavefront correction accuracy, and satisfies communication system requirements.