During the space-based remote sensing long-distance imaging process, the impact of the atmospheric environment leads to a decline in the image quality captured by the remote imaging system, specifically manifested as increased image blur and geometric distortion. This degradation has a particularly negative impact on computer vision tasks like object recognition and detection, significantly reducing processing accuracy and efficiency.

In recent years, there has been a lot of focus on the impact of atmospheric turbulence and aerosols on image quality. Many methods have been proposed to address the blurring caused by turbulence. Carhart et al. improved the frame selection method using short-exposure images^[1]. Potvin et al. simulated realistic turbulent imagery using a point spread function (PSF)^[2]. Valley et al. proposed a modulation transfer function (MTF) with tilt correction to predict Strehl ratio and resolution degradation^[3]. Burckel et al. enhanced the simulation accuracy of phase screens using logarithmic sampling of filtered white noise, reducing memory requirements^[4]. Reinhardt et al. proposed an algorithm to extract atmospheric turbulence and extinction parameters from passive images^[5]. Hunt et al. used statistical methods to describe resolution variations caused by turbulence^[6]. Schwartzman et al. developed a two-dimensional (2D) image deformation method to render turbulence distortion^[7]. Wu et al. developed an image degradation model incorporating turbulence and aerosols, optimizing the atmospheric MTF curve^[8]. Sadot et al. studied the influence of aerosols on MTF, finding that aerosols dominate at high spatial frequencies^[9]. Hu et al. built a PSF model that incorporates aerosol properties to analyze the blurring effect on remote sensing images^[10].

Deep learning has also made significant progress in image processing, especially through the use of convolutional neural networks for feature extraction. Researchers have proposed various deep learning-based methods to mitigate atmospheric turbulence^[11–14]. Barbastathis et al. used deep learning and computational imaging to address ill-posed problems^[15]. Chimitt et al. simulated turbulence quickly using Zernike coefficients^[16]. Hoffmire et al. proposed a restoration method for long-distance turbulence-induced deformation^[17]. Cheng et al. used a generative adversarial network (GAN) to simulate and restore turbulent images^[18]. Jin et al. developed TSR-WGAN, which generated high-quality image sequences without assumptions and performed exceptionally well^[19].

First, in this paper, we propose a new time-variant physical degradation model for remote sensing image atmospheric impact. This model accounts for both the turbulence short-exposure function and the single scattering aerosol function to simulate image blur effects. Additionally, we introduce the Schwartzman distortion vector field to model the distortion effects caused by atmospheric turbulence. Furthermore, we develop a multi-scale feature fusion and attention-based generative adversarial network (MSFFA-GAN) to optimize constraints on deep neural networks for atmospheric parameters in the physical degradation model. This network effectively restores images degraded by atmospheric temporal physical processes through multi-scale fusion. The encoder of the generator is built upon the Inception-ResNet-v2 architecture, incorporating multi-scale feature extraction units that work in tandem to enhance the encoder’s functionality. The decoder, in turn, aligns features from multiple scales to a common resolution through up-sampling while applying spatial weights to different levels of the feature maps during the fusion process. The discriminator employs a Markov discriminator (PatchGAN) structure to further enhance the network’s performance in distinguishing between real and restored images. Finally, by utilizing the time-varying physical degradation model of atmospheric impact, we generate synthetic image datasets degraded by turbulence and aerosols. These datasets are then input into the network for adversarial training, allowing the generator and discriminator to collaboratively enhance the image restoration performance. To validate the model, experimental imaging of real atmospheric turbulence and aerosols was conducted in the Jingyue region of Changchun City. During this process, real-time monitoring of vital atmospheric parameters, such as the atmospheric structure constant and aerosol extinction coefficient, was performed under actual atmospheric turbulence conditions. The resulting dataset was used to evaluate the restoration capability of the proposed network on real-world images degraded by atmospheric turbulence. The overall structure block diagram is shown in Fig. 1.

Atmospheric turbulence and aerosols were the main contributors to the degradation of imaging quality in remote sensing systems. The effects of turbulence and aerosols on imaging quality can be modeled as I(x,y)=C{H[O(x,y)]}.(1)where H represents the image blur effect, C represents the image jitter effect, I represents the degraded image, and O represents the original image. The image blur can be modeled as H(ρ)=Hatm(ρ)Hdif(ρ){F[F(x,y)]}.(2)where Hatm(ρ) and Hdif(ρ) represent the optical transfer functions of atmospheric turbulence and the optical system, respectively, and are expressed as Hatm(ρ)=exp{−3.44(λfρr0)5/3[1−α(λfρD)1/3]},(3)Hdif(ρ)={2π[cos−1(ρρc)−ρρc1−(ρρc)2]ρ≤ρc,0ρ>ρc,(4)where ρ=u2+v2 represents the spatial frequency, λ represents the wavelength, f represents the telescope focal length, and D represents the telescope aperture diameter. When α=0, the above equation represents the long-exposure atmospheric optical transfer function. When α=1, it represents the atmospheric optical transfer function under short-exposure conditions. ρc=1λfn represents the optical cutoff frequency, fn=f/D represents the F-number, and r0 represents the atmospheric coherence length, given by the following equation: r0=[0.423(2πλ)2(secψ)∫0LCn2(h)dh]−3/5.(5)where ψ represents the zenith angle and Cn2 denotes the atmospheric refractive index structure constant.

F(x,y) represents the point spread function of aerosol particles, which is expressed as F(x,y)=ω04πμ∫0τμ′r′2e−τ(1/μ′−1/μ)P(cosγ)dτ,(6)where ω0 represents the single scattering albedo, μ=cosθ, μ′=cosθ′, θ is the zenith angle, and θ′ is the angle between the scattering light propagation direction and the vertical direction. P(cosγ) is the scattering phase function, which describes the scattering intensity distribution of particles at different angles. According to the Mie scattering theory, the scattering phase function of aerosol particles can be expressed as P(cosγ)=2πx2σs(S1S1*+S2S2*),(7)where σs represents the scattering cross-sectional area, S1* and S2* denote the complex conjugates. τ represents the aerosol optical thickness, which describes the degree of attenuation of light radiation during transmission through the aerosol medium. It can be calculated by integrating the aerosol extinction coefficient over direction, τ(z)=∫0zKe(z′)dz′.(8)where Ke represents the aerosol extinction coefficient and z represents the distance in the vertical direction.

Based on physical principles, Schwartzman’s method directly generates a 2D random deformation vector field for image distortion. C(v)=E[e(p)Te(p+v)].(9)where p and p+v are the distortion motion vectors of two pixels, and e represents the distortion motion vector function.

Using this physical model, the generation of short-exposure atmospheric turbulence and aerosol-degraded images was simulated, with the simulation parameters provided in Table 1.

Atmospheric turbulence leads to independent effects of image blur and jitter, which can be calculated separately. With a fixed wavelength and a focal length, various values for the atmospheric refractive index structure constant and aerosol optical thickness were set, and the target image was input into the simulation model. It was observed that under strong turbulence, the resulting degraded image became nearly indistinguishable, while under weaker turbulence, the image remained discernible but exhibited reduced contrast and blurriness, as shown in Fig. 2. Furthermore, turbulence-induced distortion was observed in the target image. A physical model based on a 2D random deformation vector field was used to simulate these distortions under varying turbulence intensities, as shown in Fig. 3.

To investigate the impact of atmospheric aerosols on long-range imaging, simulations were conducted to observe the degradation effects of different aerosol optical thickness values on the image, as shown in Fig. 4.

Variations in aerosol optical thickness strongly affect the range of influence of the atmospheric aerosol PSF, as we can observe from the calculated 2D spatial distribution map. As the aerosol optical thickness increases, the number of pixels affected by the PSF increases, leading to a greater degradation of image quality in the spatial domain.

To validate the effectiveness of the atmospheric turbulence and aerosol image degradation simulation model, we conducted field trials on July 30 and August 1, 2024, at a 60-m-high rooftop platform in the National Science Park of Jingyue District, Changchun. In support of the field trials, Henghui Optoelectronics provided extensive instrumentation, range sponsorship, and coordination of the measurement program. The trials employed a NexStar 8SE telescope paired with a MER2-230-168U3C camera for image acquisition of rooftop targets at 3.3 km distance and streetlights at 8.1 km distance. Simultaneous atmospheric parameter monitoring was implemented using the HMTM-2 temperature fluctuation meter and the MSLR-200 dual-band aerosol lidar. Figure 5 shows the specific setup of the field trials.

During data acquisition, the temperature fluctuation meter continuously recorded the atmospheric refractive index structure constant at 5-s intervals. The lidar system executed sector scans in 10° increments across zenith angles from 10° to 90°, while conducting detection every minute at 15-m vertical intervals. To mitigate solar interference, each observation angle maintained 24-h continuous monitoring, ensuring continuous acquisition of aerosol extinction coefficients.

As illustrated in Fig. 6, turbulence intensity demonstrates distinct diurnal patterns, with periodic peak values occurring during daytime hours from 11:00 to 15:00 that correlate with substantial deterioration of imaging quality. Conversely, nocturnal observations from 20:00 to 07:00 exhibit markedly reduced turbulence intensity, resulting in minimal degradation of the imaging system performance. Figure 7(a) reveals that under clear-sky conditions on July 30, the atmosphere displays homogeneous distribution characteristics accompanied by a stable vertical profile of near-surface aerosol extinction coefficients. However, the observed distribution of thin clouds within the altitude layer from 7.5 to 13.5 km resulted in an aerosol extinction coefficient peak reaching approximately 10km−1. Figure 7(c) shows that under cloudy conditions on August 1, there are larger fluctuations in the extinction coefficient within the 0.4–0.8 km altitude layer, with the enhancement of cloud coverage being the primary contributing factor. Comparative analysis of aerosol extinction coefficient temporal variations at 0.06, 0.3, and 0.6 km altitude levels in Figs. 7(b) and 7(d) quantitatively confirms altitude-dependent aerosol distribution characteristics.

By comparing the actual captured images with the simulated images, the performance of the model can be qualitatively analyzed. When performing the simulation, the simulation parameters must correspond to the atmospheric parameters of the actual captured images. Based on the capture time of the actual images, along with the corresponding atmospheric refractive index structure constant and aerosol extinction coefficient, the simulation parameters are set as outlined in Table 2.

When the reference target is a streetlight situated 8.1 km from the observation point, simulations are performed based on the atmospheric refractive index structure parameters and the aerosol optical thickness corresponding to real degraded images at various times, as shown in Fig. 8.

Figure 8(b) shows an image captured on July 30, 2024, at 13:19:18. The relevant atmospheric parameters are as follows: the atmospheric refractive index structure constant is 9.503×10−14m−2/3, and the aerosol optical thickness is 0.248. It can be seen that in the areas marked by the red, orange, and yellow boxes, strong turbulence causes significant distortion of traffic signs, windows, and prohibition signs in the image. In addition to distortion, turbulence and aerosols also cause blurring of the image.

Figure 9(b) shows an image captured on August 1, 2024, at 13:44:46. The atmospheric refractive index structure constant is 5.936×10−14m−2/3, and the aerosol optical thickness is 0.188. As illustrated in the figure, the white line within the orange box is visibly distorted, and the image shows noticeable blurring.

To perform a more quantitative analysis of the effects of atmospheric turbulence and aerosols on image quality, this work compares the quality of clear images, actual degraded images obtained from tests, and simulated degraded images. The comparison focuses on parameters closely related to image quality, such as resolution and contrast, with particular emphasis on resolution as an indicator to evaluate the degradation of images before and after atmospheric effects.

Figure 10 compares the clarity and contrast of clear, actual degraded, and simulated degraded images under four sets of different atmospheric parameters. The results show that, compared to the clear images, both the clarity and contrast of the actual and simulated degraded images decrease, with a consistent trend observed. When the atmospheric parameters are Cn2=2.299×10−14m−2/3 and τ=0.248, the difference in clarity between real and simulated images is minimal. When the atmospheric parameters are Cn2=9.464×10−16m−2/3 and τ=0.200, the difference in contrast between them is the smallest. It is concluded that this physical model can effectively simulate the degradation process of images under actual atmospheric conditions.

Figure 11 illustrates the proposed generator network structure, which utilizes the pre-trained Inception-ResNet-V2^[20] model as the backbone of the encoder, combined with a multi-scale feature extraction (MSFE) unit. These two components collaboratively form the functional framework of the encoder. Following down-sampling, the encoder produces feature maps at different scales. High-level feature maps exhibit a larger receptive field and stronger semantic representation but with lower resolution. In contrast, low-level feature maps have a smaller receptive field and higher resolution, though they provide weaker semantic representation. To reconcile this, a 3×3 convolutional up-sampling layer is applied to rescale the feature maps to a uniform size. The fused feature maps undergo further up-sampling, and the generated output image is then passed to the discriminator for adversarial training against the initial image.

Figure 12 illustrates how the MSFE unit incorporates multi-scale feature fusion and attention mechanisms. By employing a multi-branch convolutional layer structure with varying receptive field sizes, the multi-scale feature fusion stage effectively handles the diverse feature information contained within images exhibiting different levels of degradation. An attention mechanism module is added to highlight important image parts and improve learning ability. The attention mechanisms refine feature maps by emphasizing relevant channels and spatial regions, leading to better data understanding and representation. To preserve crucial information and prevent vanishing gradients, a residual connection merges the input feature map with the processed feature map from the MSFE unit.

The attention mechanism module comprises channel and spatial attention mechanisms. The channel attention mechanism module compresses global spatial information into channel descriptors by generating channel statistics through global average pooling^[21]. This descriptor embeds the global distribution of the channel feature responses, allowing information from the network’s global receptive field to be utilized by all layers, Fc=1H×W∑i=1H∑j=1WF(c,i,j),(10)where F(c,i,j) represents the value of the input feature F∈RC×H×W at channel c and position (i,j), while the output Fc represents the one-dimensional feature corresponding to channel c.

To utilize the aggregated information, an excitation operation is applied to fully capture the channel dependencies. A simple gating mechanism with a sigmoid activation function is employed, where the embedding is used as input to generate a set of modulation weights for each channel, s=σ(W2δ(W1Fc)),(11)where δ(·) denotes the ReLU activation function, and σ(·) represents the Sigmoid activation function. To constrain model complexity and improve generalization, we parameter the gating mechanism by introducing a bottleneck with two fully connected layers around the non-linearity, specifically a dimensionality reduction layer with a reduction ratio r denoted as W1∈RCr×C and W2∈RC×Cr. Finally, an element-wise multiplication is performed between the output of the gating mechanism and the input feature F, producing the output of the channel attention, Fscale=F⊙s.(12)

Unlike the channel attention mechanism, the spatial attention mechanism focuses on the “spatial locations” of the relevant information in the feature map^[22]. First, global max pooling and global average pooling are applied along the channel axis to the input feature F∈RC×H×W, producing the 2D features Favg∈RH×W×1 and Fmax∈RH×W×1, respectively, Favg=1C∑c=1CF,(13)Fmax=maxcF.(14)

After stacking the obtained 2D features, they are passed through a 7×7 convolutional layer to capture contextual information in the spatial domain. The output is then passed through a Sigmoid activation function to generate the spatial attention weight matrix Ms(F). Finally, an element-wise multiplication is performed between the attention weights and the input feature F, assigning different levels of importance to different spatial locations, thereby enhancing the model’s ability to understand and represent the input data. Ms(F)=σ(W1[Favg;Fmax]),(15)Fscale=F⊙Ms(F).(16)

The discriminator adopts a Markov discriminator structure^[23], as shown in Fig. 13. PatchGAN maps the input image to an N×N matrix and uses a fixed-size local receptive field (referred to as a “patch”) to assess different regions of the image. Since the size of the local receptive field is fixed, the discriminator does not need to know the dimensions of the entire image, making PatchGAN highly flexible and capable of handling images of varying sizes. Xi,j represents the discriminator’s output for a small patch of the input image, indicating the probability that the patch is a true sample, with values between 0 and 1. The final output of the discriminator is obtained by averaging all the Xi,j values.

Selecting an appropriate loss function is critical for optimizing neural network performance. In standard GAN frameworks, the discriminator typically employs a classifier with Sigmoid cross-entropy loss, which may lead to vanishing gradient issues during training. To address this, the least-squares loss variant modifies the discriminator’s objective to penalize the squared difference between its predictions and target values (1 for real data and 0 for generated samples), LD=12Ex∼Pdata(x)[(D(x)−1)2]+12Ez∼Pz(z){D[G(z)]2}.(17)

The generator aims to minimize the discrepancy between the discriminator’s prediction for its output and the ideal value of 1, LG=12Ez∼Pz(z){D[G(z)]−1}2.(18)

Content loss combines mean squared error loss (LMSE) and perceptual loss (LVGG). The mean square error measures the average squared difference, pixel by pixel, between the original image and the image corrected by the network. A pre-trained VGG19 network calculates the perceptual loss. Both the generated and clear images are processed by the VGG19 network, and their corresponding feature maps are used to compute the Euclidean distance, LMSE=1CHW∑c=1C∑x=1H∑y=1W‖IGCHW−IRCHW‖2,(19)LVGG=1N∑i=1N[ϕi(IR)−ϕi(IG)]2.(20)where C represents the number of input channels of the image, and W and H denote the width and height of the image, respectively. N is the number of feature layers, IR represents the clear image, IG represents the image reconstructed by the generator, and ϕi(·) denotes the feature representation at the ith layer of the pre-trained neural network. Therefore, the total loss function is composed of the adversarial loss and the content loss, L=αLGAN+βLMSE+γLVGG.(21)where α, β, and γ are weight coefficients, set to 0.01, 0.5, and 0.01, respectively.

The dataset used for training and testing our network architecture is described at the beginning of this section. Following that, the parameters needed for training are outlined. After outlining our experimental design, we benchmark our approach against various leading methods.

A large dataset is crucial for training GANs to maintain stability and achieve optimal performance. To overcome the challenge of obtaining clear images in real-world conditions, this study utilizes simulations to create the necessary dataset. As shown in Fig. 14, 137 clear images were selected from the RS_C11 remote sensing dataset^[24], and used to synthesize degraded images using the time-varying physical degradation model for atmospheric effects. Blurry images with varying levels of degradation were produced by altering parameters, including the atmospheric refractive index structure constant, the atmospheric coherence length, and the aerosol optical thickness. A total of 2466 training images were generated, with 2088 used for training and 378 reserved for testing the network’s performance.

The training was conducted using the PyTorch 1.13.0 deep learning framework and an NVIDIA GeForce RTX 4090 GPU. The training process employed the Adam optimizer, a method that incorporates momentum and RMSProp. It adjusts the learning rate for each parameter by estimating both the first and second moments of the gradients, enabling more efficient network training. The two hyperparameters of the Adam optimizer were set as follows: β1=0.5 and β2=0.999, with a batch size of 8. The initial learning rates for the generator and discriminator were set to 0.001 and 0.005, respectively, and were reduced by a factor of 10 every 25 epochs. In total, the learning rate was decreased 4 times, and training was terminated after 100 epochs. The training process took 3.59 h.

After the model training was completed, this study applied the generator network to degraded image restoration tasks. To validate the performance advantages of the proposed algorithm, we compared it with various advanced image restoration models including VDSR^[25], MemNet^[26], MSICF^[27], and EDSR^[28]. The experiments employed the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) as objective evaluation metrics to quantify image reconstruction quality. Additionally, the computational efficiency of each algorithm was evaluated based on the number of model parameters, floating-point operations (FLOPs), and single inference time. The experimental results shown in Table 3 comprehensively demonstrate the performance differences and comparative characteristics of various methods.

As shown in the quantitative comparison in Table 3, the MSFFA-GAN achieves the best multi-dimensional balance in image restoration tasks. The proposed method demonstrates superior restoration capabilities, with a PSNR of 22.65 dB and an SSIM of 81.42%, both significantly outperforming other comparative methods. In terms of computational efficiency, the MSFFA-GAN requires only 122.66 s to complete processing, which is much faster than the EDSR, the MSICF, and the MemNet, while maintaining excellent restoration quality. The model achieves an optimal balance between performance and efficiency, with 323.75M parameters and 706.76G FLOPs. It avoids the high computational cost of the EDSR while outperforming lightweight models like the VDSR and the MemNet. These advantages are attributed to the multi-scale feature fusion strategy, which allows the model to analyze features across various scales and is particularly beneficial for handling objects of diverse sizes and complexities. Additionally, the integrated channel attention mechanism captures the interactions between the feature channels while the spatial attention mechanism determines the prominence of different parts of the image. The synergy between these attention mechanisms enables a more thorough understanding and utilization of image features, allowing the model to better present these features.

Figure 15 compares the restored images generated by different algorithms. Figure 15(a) displays the original clear image as the reference for restoration. Figures 15(c) to 15(f) present the results processed by the VDSR, the MemNet, the MSICF, and the EDSR, respectively, while Fig. 15(g) shows the output of the proposed MSFFA-GAN. The proposed algorithm significantly improves the restoration of the blurred images, producing outputs with enhanced clarity and richer edge details that closely resemble the original image. The objective metrics listed in Table 3 confirm the subjectively perceived visual improvements.

To systematically evaluate model stability and robustness, we performed 25, 50, 75, and 100 independent trials on the test set. The means and standard deviations of both the PSNR and the SSIM were computed for each trial configuration, with comprehensive results summarized in Table 4.

Experimental analysis reveals that the computation time increases linearly from 0.68 to 2.90 h as the number of experiments increases. Both quality metrics exhibit remarkable stability: the PSNR maintains 22.614–22.620 dB (σ=0.03dB) and the SSIM remains at 0.814 (σ=0.03%), demonstrating exceptional model robustness and reproducibility.

To test the algorithm’s ability to restore images impacted by real atmospheric effects, we gathered a dataset of degraded images captured under actual atmospheric conditions using a NexStar 8SE telescope and a MER2-230-168U3C camera. The images, influenced by atmospheric distortion, served as input for the network’s evaluation. The results are shown in Fig. 16.

Based on the comparison between the restored and degraded images, there is a significant improvement in overall clarity, with better preservation and presentation of local details. The experimental results further validate the restoration performance of the proposed model, demonstrating its feasibility and effectiveness in image restoration.

In this paper, we propose a model that analyzes how the physical degradation of atmospheric effects evolves over time. This time-variant model is not limited to processes caused by atmospheric turbulence and aerosols in short-exposure conditions. It can also model distortion effects under a range of atmospheric turbulence scenarios. Meanwhile, we conducted field trials to get the dataset and briefly analyzed important parameters, such as the atmospheric refractive index structure constant, the aerosol extinction coefficient, and the vector field. Comparing simulated and real degraded images in an experimental analysis confirmed the model’s effectiveness.

Moreover, in this work, we present a generative adversarial network called MSFFA-GAN that can be coupled with the time-varying physical model mentioned above. This framework leverages receptive fields of different sizes for multi-scale spatial feature extraction, while a channel attention mechanism dynamically adjusts channel weights in feature maps. Through a spatial attention mechanism, the model can prioritize important spatial features, allowing for more effective restoration of degraded images. Compared to images degraded by turbulence, the proposed network enhances the PSNR by 5.05 dB and the SSIM by 4.43%. Notably, the proposed model applies to moderate and weak atmospheric turbulence conditions. When the atmospheric refractive index structure constant falls within the range of 10−15–10−14m−2/3 and the aerosol optical thickness is configured with specific parametric combinations of 0.048, 0.148, and 0.300, it enables the generation of image datasets exhibiting varying degrees of degradation. Additionally, the model training relies on the residential area subset of the RS-C11 dataset, which demonstrates robust adaptability to remote sensing images in this specific scenario.

In future research, we plan to incorporate more diversified datasets that encompass various scenarios and environmental conditions of remote sensing imagery. This strategic enhancement aims to improve the model’s generalization capacity and strengthen its applicability across broader operational scenarios.