A vortex beam can effectively improve the channel capacity of the communication system, however in the atmospheric environment, the optical communication channel is influenced obviously by the atmospheric turbulence. Thus it is of great significance to study the transmission characteristics of a vortex beam in the atmospheric turbulence. The Kolmogorov spectral model is widely used to describe the atmospheric turbulence previously, but the further researches show that the atmospheric turbulence also has the non-Kolmogorov spectral characteristics. Therefore, the investigation of the non-Kolmogorov spectral transmission characteristics for a vortex beam is conducted. Based on the numerical methods, the influence of outer- and inner-scale of turbulence, generalized exponential factor, refractive-index structure constant on the spiral spectral distributions, topological detection probabilities and others of a Laguerre-Gaussian beam after different transmission distances is investigated. The simulation results show that the topological detection probability is closely related with the above parameters. Finally, a numerical method for calculating the scintillation index is proposed, and the influence of turbulence on bit-error-rate (BER) is figured out. The results show that when the beam is propagated up to 1000 m even in a weak turbulence, the BER is still difficult to meet the communication requirements, and thus a further phase correction is quite necessary.