Initialize replay memory D to capacity C, random weights ωevalua of evaluate network, target network ωtarget=ωevalua.

for episode=1, 2, …, N, do

Update FSO network environment as well as source node and destination node.

Source node generates data transfer task and agent gets the initial state St.

for t=1, 2, …, T, do

Obtain valid action space V(St) according to Eq. (13).

Select action using decay greedy strategy and obtain instant reward Rt according to Eq. (11).

Execute action at in FSO network environment as well as get the next state St+1 and dDone.

Store the experience tupleSt,at,Rt,St+1,dDone in D.

Update εt+1=εt-ηnet and St=St+1.

if number of samples≥C, do

Sample random mini batch of transitions from D.

Obtain Vevalua(St,at) and compute Vtarget(St,at) according to Eq. (15).

Compute the loss according to Eq. (16) and perform a gradient descent step with respect to ωevalua.

Reset ωtarget according to Eq. (17) in every step.

end if

end for

end for