Free space optical (FSO) networks, with their large bandwidth, unlicensed spectrum, and high data rates, have become a promising technology for future broadband wireless networks. These networks provide an appealing alternative to traditional radio frequency (RF) and optical fiber networks, especially in environments where conventional communication technologies encounter difficulties, such as in remote or disaster areas. However, the data transmission of FSO networks is prone to atmospheric conditions like rain, fog, and snow, which can lead to link attenuation or even communication interruption. In addition, the problem of dead nodes in the network, where nodes lose communication capability due to equipment failure or energy depletion, also challenges the connectivity and reliability of the FSO network. Traditional routing protocols and heuristic routing algorithms have achieved some success in optical fiber networks and wireless radio frequency networks. However, they do not effectively adapt to the rapid changes in link communication quality and node instability in the dynamic FSO network environment and have certain limitations. Due to the characteristics of FSO networks and the drawbacks of existing routing algorithms, a new routing algorithm needs to be proposed to handle the dynamic and complex network environment. This new routing algorithm should have stronger adaptability and can dynamically adjust the routing strategy according to the real-time link quality and node state.

We present the deep Q network with mask (DQNM) algorithm, a reliable routing solution based on deep reinforcement learning (DRL) specifically designed for FSO networks. The proposed algorithm utilizes the capabilities of DRL to learn the complex relationships between environmental states (such as weather conditions and node health) and corresponding routing actions. This enables the algorithm to optimize the routing decisions and ensure the most reliable transmission path in dynamic and uncertain network conditions. The algorithm’s reward function is designed to be related to the link margin, a critical parameter in FSO communication systems that reflects the quality of the communication link. Through experiments, we verify the effectiveness of link margin as a reliable metric for assessing the transmission performance of FSO links. One of the remarkable features of the DQNM algorithm is the incorporation of the action mask mechanism. This mechanism allows the algorithm to avoid selecting links that are either too unstable due to adverse weather conditions or those connected to dead nodes. By masking out invalid or ineffective actions during the training process, the algorithm avoids wasting resources on redundant actions, thereby improving training efficiency. This action masking process enhances the robustness of the learning process as it reduces the influence of poor-quality links and dead nodes on the overall performance of the algorithm. Eventually, this approach leads to a more reliable and efficient routing solution for FSO networks, especially under challenging environmental and operational conditions.

Simulation results show that in the dynamic and uncertain FSO network environment, the deep Q network (DQN) algorithm requires about 12000 training iterations to gradually converge. In contrast, the proposed DQNM algorithm converges in around 5000 iterations, achieving an approximately 58.3% improvement in convergence speed. When the two algorithms converge, the data fluctuation amplitude of the DQNM algorithm is smaller than that of DQN algorithm, and it has higher routing cumulative reward performance. This improvement is due to the presence of many ineffective actions during the training of the DQN algorithm, which degrades its overall performance. In comparison, the DQNM algorithm mitigates the impact of adverse link conditions and dead nodes through the incorporation of an action masking mechanism, thereby reducing redundant training and enhancing both learning efficiency and robustness. This, in turn, leads to a significant increase in the reliability of FSO network transmission. Furthermore, the DQNM algorithm attains packet delivery rates of 96.9%, 91.3%, and 84.5% under rain, snow, and fog conditions, respectively, showing improvements of 2.9, 5.3, 6.9 percentage points over the DQN algorithm. Under the same conditions, the DQNM algorithm also exhibits reductions in average energy consumption and average transmission delay by 10.5%, 17.4%, and 16.6%, as well as 9.2%, 15.0%, and 15.0%, respectively, when compared to the DQN algorithm. Additionally, when dead nodes are present in the network, the packet delivery rate of the DQNM algorithm reaches 88.2%, which is 3.8 percentage points and 9.2 percentage points higher than those achieved by the DQN and ant colony algorithms, respectively.

In the context of the complex FSO network environment, we propose a reliable routing algorithm based on DRL. The algorithm takes advantage of the capabilities of DRL to learn the functional mapping between environmental states and actions, thereby facilitating reliable routing in FSO networks. While traditional DRL algorithms can adapt to changes in the state space, they often involve a large number of ineffective actions during training, which can undermine the overall training performance. To solve this problem, the DQNM algorithm is introduced. By incorporating an action mask mechanism to reduce the impact of harsh link environments and dead nodes, the algorithm decreases training redundancy, enhances learning efficiency, and improves robustness. As a result, the proposed DQNM algorithm considerably enhances the reliability of data transmission in FSO networks.