The optical endoscope imaging technology based on fiber bundle has the advantages of strong flexibility, no radiation and ease of integration. However, the structural characteristics of the fiber bundle inherently cause the honeycomb grid artefacts. In addition, the endoscope microlens introduces image distortion and low numerical aperture. As a result, there is a substantial decrease in the image quality for such endoscopic systems. In order to address this challenge, tremendous efforts have been made and a number of algorithms have been developed, such as spatial/frequency domain filtering, interpolation, etc., to successfully eliminate the image grid artefacts introduced by fiber bundle. Nevertheless, the image quality has not been substantially improved especially in terms of reduced spatial resolution and image distortion. In recent years, the image recognition and enhancement capabilities of Deep Learning (DL) have been significantly improved and thus, DL has been explored to apply for the reconstruction of endoscopic images, for example, using U2-Net, Attention U-net, and GARNN models. While the image quality could be improved to some extent, it remains challenging in the restoration of fine details. More importantly, until now, there has been no research on the utilization of DL to eliminate image distortion introduced by microlens in the endoscopic image. In order to address this challenge, we develop a new open source algorithm AtG-DeepLab V3+ by effectively integrating the self-attention mechanism and the DeepLab V3+ network for the enhancement processing of endoscopic images. This developed model adapts the coding-decoding network as a whole architecture and uses the ResNet101 network to extract features. Encoding context information by probing incoming features at multiple rates and multiple valid horizons or merging operations allows high-level abstract features to be extracted and clearer objects to be captured by progressively recovering spatial information. The self-attention gate integrated with the model decoder can effectively suppress the activation of irrelevant regions, better realize the screening and extraction of important spatial features of images, and achieve the distortion correction and high detail restoration of the endoscope image. In addition, we develop an optical endoscopy imaging system to collect test target images and biological tissue images for model training and testing. The training results of test target images demonstrate that the developed AtG-DeepLab V3+ model can effectively correct the intrapexial image distortion and remove the honeycomb grid structure artefacts. The results of local detail magnification and comparison illustrate that this model has a good performance in improving the image resolution limit and detail feature extraction compared with other current models (i.e., U2-Net, Attention U-net and GARNN). The training and testing results of biological tissue images exhibit that the fine detailed features of human tissue can be extracted by the AtG-DeepLab V3+ model, and the distortion of the pattern can be effectively eliminated. The recovered images show a clearer tissue texture and richer fine details, which are highly similar to the real images. Compared with the existing U²-Net, Attention U-net and GARNN algorithms, the AtG-DeepLab V3+ model can achieve a much higher peak signal-to-noise ratio (increased by 66.4%, 51.9%, 154.6%), structural similarity (increased by 55.6%, 45.9%, 231.5%), demonstrating a significant improved enhancement effect of endoscopic image. We believe that this algorithm can provide a new efficient processing scheme for optical endoscopy image processing.