Existing deep learning-based methods for fluorescence image super-resolution can be broadly classified into two categories: those guided by peak signal-to-noise ratio (PSNR) and those guided by perceptual considerations. The former tends to produce excessively smoothed prediction results while the latter mitigates the over smoothing issue considerably; however, both categories overlook the ill-posed nature of the super-resolution task. This study proposes a fluorescence image super-resolution method based on flow models capable of reconstructing multiple realistic super-resolution images that align with the ill-posed nature of super-resolution tasks. Moreover, microscopy imaging is conducted in continuous time sequences naturally containing temporal information. However, current methods often focus solely on individual image frames for super-resolution reconstruction, completely disregarding the temporal information between adjacent frames. Additionally, structures in similar biological samples exhibit a certain degree of similarity, and fluorescence images collected possess internal self-similarity in the spatial domain. To fully leverage the temporal and spatial information present in fluorescence images, this study proposes a frequency- and spatial-domain joint attention module. This module aims to focus on features that significantly contribute to the prediction results, obtaining more accurate reconstruction outcomes.Similar to most supervised learning methods, our approach has a limitation in that it requires labeled paired image sets for training the network model. Generalization performance may significantly decline when applying the model to a test set with a distribution different from the training set. Acquiring labeled paired training data is not always feasible in practical applications. Therefore, future work may need to address the challenge of cross-dataset super-resolution reconstruction, considering optimization strategies and network improvements from a domain adaptation perspective.

This study introduces a flow-model-based multi-frame dual-domain attention flow network. Given a low-resolution image, the network learns the distribution of super-resolution images using flow models, enabling the reconstruction of multiple realistic super-resolution images to address the underdetermined nature of super-resolution tasks. Additionally, as the imaging process is typically continuous, the acquired raw data from a microscope has temporal relationships with adjacent frames. However, existing deep learning-based fluorescence image super-resolution methods often neglect the temporal priors present in multiple input frames over time. Moreover, biological sample structures exhibit internal self-similarity. Therefore, by constructing a frequency-domain and spatial-domain joint attention module, the network is guided to focus extensively on features that contribute significantly to the prediction results, further enhancing the network’s performance. The proposed flow-model-based multi-frame dual-domain attention flow network consists of a flow model and a frequency-domain and spatial-domain joint attention module. The flow model, composed of multiple reversible modules, facilitates a reversible mapping between the target and latent space distribution. The frequency-domain and spatial-domain joint attention module achieves conditional feature extraction and includes a set of frequency- and spatial-domain attention blocks. These blocks comprise Fourier channel attention blocks, spatial attention blocks, and convolutional layers, serving to extract temporal, spatial, and aggregated features from the fluorescence image, respectively. Furthermore, the study employs skip connections to enable feature reuse and prevent gradient vanishing.

This study demonstrates the importance of temporal information by comparing the proposed method, a multi-frame dual-domain attention flow network (MDAFN), with a single-frame dual-domain attention flow network (SDAFN). Quantitative evaluation metrics include PSNR and learned perceptual image patch similarity (LPIPS). Experimental results indicate that the MDAFN outperforms the SDAFN, and the indexes of PSNR and LPIPS on the three data sets are shown in Table 1. Moreover, visually, the images reconstructed using the MDAFN exhibit improvement over those generated using the SDAFN (Figs. 7?9). Finally, a comparison between the proposed method and state-of-the-art super-resolution reconstruction methods is presented. The results indicate that when the standard deviation of hyperparameters is set to zero, the PSNR of the super-resolved images obtained using the proposed method is comparable or even superior to that obtained using other methods. For LPIPS, the proposed method outperforms other methods. When the standard deviation is greater than zero, the LPIPS obtained using the proposed method is further decreased across the three datasets (Table 2). The reconstructed results using the proposed method visually resemble the ground-truth images more closely. In contrast, other methods generate over-smoothed, signal-depleted, or artificially enhanced reconstructions with poorer subjective quality (Figs. 10?13).

This study proposes an MDAFN for high-quality super-resolution reconstruction of fluorescence images. Unlike conventional neural networks that directly learn deterministic mapping functions, our approach can predict various possible super-resolution images for a given low-resolution wide-field image, addressing the underdetermined nature of the one-to-many relationship in super-resolution tasks. Additionally, considering the high internal self-similarity of structures in similar live cell samples in both temporal and spatial dimensions of fluorescence images, we further introduce a frequency- and spatial-domain joint attention module based on multi-temporal input. This module aims to focus more on features contributing significantly to the prediction results, yielding more accurate predictions. Experimental results demonstrate that the proposed method outperforms other methods in terms of super-resolution image quality.