Aiming at the low utilization of spectrum resources in optical network resource allocation, an improved deep Q network(DQN) reinforcement learning algorithm is proposed. Based on the ε-greedy strategy, the algorithm sets the loss function according to the difference between the action value function and the state value function, and constantly adjusts the ε value to change the exploration rate of the agent. In this way, the optimal action value function is realized, and the routing and spectrum allocation problems are solved well. In addition, different experience pool sampling methods are used to improve the convergence speed of iterative training. The simulation results show that the improved DQN reinforcement learning algorithm can not only make the elastic optical network training model converge quickly, but also improve the spectrum resource utilization by 10.09%, reduce the blocking rate by 12.41% and reduce the average access delay by 1.27 ms compared with DQN algorithm when the traffic volume is 300 Erlang.