Aiming at the problem that the light emitting diode (LED) limits the transmission rate in the existing visible light communication (VLC) system. According to the signal-to-noise ratio(SNR) and the set bit error rate(BER) of the receiver, this paper proposes a VLC system that adaptively selects the modulation mode, modulation order, channel estimation algorithm and coding mode, and carries out simulation research with Monte Carlo method. The simulation results show that the VLC system adopts adaptive modulation, channel estimation and coding, which can make full use of the spectrum resources and maximize the transmission rate and channel capacity under a certain SNR and BER.