The advantages of structured illumination microscopy (SIM) as a live-cell imaging modality are high temporal resolution, minimal photon damage, and low photobleaching.1^–8 However, observing nanostructures in thick cells is challenging for conventional two-dimensional (2D)-SIM because the image quality is degraded by the out-of-focus background. This originates from the well-known “missing cone” of the optical transfer function (OTF).9^,10 Specifically, such out-of-focus information introduces significant background fluorescence and periodic honeycomb artifacts in the superresolved images, resulting in degraded image resolution and poor contrast.11

To fill in the missing cone of the OTF, several three-dimensional (3D) super-resolution (SR)-SIM approaches, including the three-beam SIM2 and I5S,12 were developed to enhance the spatial resolution in three dimensions. Owing to the faster imaging speed and simpler implementation, the former has become a standard, widely used approach in many commercial 3D-SIM setups. However, as it requires 15 raw frames to reconstruct a single SR image, three-beam SIM requires a significantly higher photon budget and has a lower imaging speed than the conventional 2D-SIM.

Separately, optically sectioned, super-resolution structured illumination microscopy (OS-SR-SIM) was developed to compensate for the missing cone via an empirically optimized approximation, known as OTF attenuation.13^,14 This approach removes out-of-focus information contained within the missing cones of the first-order passbands without decreasing the signal-to-noise ratio in the final image, whereas other approaches suppress out-of-focus information by simply removing the zero-order information component.15 In contrast to three-beam SIM, OS-SR-SIM requires only nine frames to reconstruct an SR image, resulting in a faster reconstruction speed and lower light dosage.7^,16 More importantly, it shares an identical optical setup with conventional 2D-SIM, making it much easier and more convenient to implement. For these reasons, OS-SR-SIM is preferred where higher demands are placed on temporal resolution and photon budget over axial resolution, thereby enabling the dynamics of organelles in relatively thick cells (with several microns thickness) to be studied.11^,16^,17

However, the reconstruction algorithm for OS-SR-SIM imposes a significant computing burden due to a complex workflow and a large number of calculations. This nullifies real-time imaging as 4 to 8 s is required to reconstruct a single SR image of 1024×1024pixels in size.17 In other words, the microscope operators must first use the widefield mode to navigate the promising field-of-view (FOV) for their investigations, then switch to the SR-SIM mode to acquire the raw images for SR-SIM, and wait for a long time to observe a single, SR image after dedicated postprocessing. The disjointed workflow of SIM setups inevitably impedes the widespread application of SR-SIM among biologists. To overcome this issue, a graphics processing unit (GPU)-enhanced approach was presented by using parallel computing tools to accelerate the SR reconstruction.18 While an overall increase in imaging speed was achieved, their high-speed reconstruction was limited to a raw image size less than 512×512pixels.

Alternative approaches focused on improving the speed of the reconstruction algorithms, especially from the perspective of spatial domain reconstruction (SDR).19^–22 The origin of the concept to reconstruct the SR image in real space dates back to the earliest work of Lukosz20 in the 1960s, which appeared as an early thought of SR-SIM. In 2001, So et al.21 comprehensively interpreted the principle of SR-SIM in the spatial domain and gave the explicit expression of the spatial domain interpretation of SR-SIM, which however was not further exploited as a reconstruction algorithm. In 2008, Somekh et al.22 established a new reconstruction method to facilitate quantitative noise analysis of SR-SIM, which yielded the SR image from the raw SIM images without the need to perform Fourier transforms. This was the first time that spatial domain interpretation appeared as a reconstruction method. Till 2021, we further exploited it as a rapid reconstruction algorithm termed SDR algorithm.23 In contrast to the mainstream reconstruction algorithms that were generally executed in Fourier space, the SDR method performed all calculations in the spatial domain, resulting in up to a sevenfold increase in image reconstruction speed. However, the previous algorithm did not properly address the out-of-focus background induced by the missing cone of the OTF.

In this paper, to address both the reconstruction speed and the missing cone problem, a rapid reconstruction algorithm termed joint space and frequency reconstruction (JSFR)-SIM was developed by combining spatial domain processing with OS-SR-SIM implemented in the frequency domain, resulting in the improved image reconstruction speed and suppression of the out-of-focus background in thick cells. Essentially, the JSFR-SIM method transfers the OTF compensation and the OTF attenuation in the conventional method to bandpass prefiltering before performing spatial domain processing, by engineering the effective point spread function (PSF) of the raw images. The execution time of the reconstruction is reduced to 10.2 ms for raw images with 512×512pixels in size, which is 80-fold faster than the widely used Wiener-SIM. Critically, the speed increase does not come at the expense of the image quality. To demonstrate this, real-time live-cell observation of COS-7 cells was performed that revealed insight into microtubule dynamics and rapid mitochondrial tubulation.

To date, most reconstruction algorithms for SR-SIM are based on the Wiener-SIM protocol developed by Gustafsson,1 Heintzmann and Cremer.24 Here, the acquired raw images are transformed to the Fourier domain and the zero-order component (lower frequency information) and ±first-order components (higher frequency information) of the specimen are extracted by solving a set of linear equations. Afterward, as a Wiener deconvolution procedure, each component is multiplied with the conjugated OTF (referred to as OTF compensation) and moved to their true positions. Eventually, the superresolved image is obtained by successively adding these components together, dividing their sum by the sum of the squares of the OTFs plus a small constant, and inversely transforming the spectrum back to real space.1 For thick samples, to suppress the background fluorescence and periodic honeycomb artifacts, the OTF attenuation strategy (hereinafter included in Wiener-SIM) is generally employed by recombining the shifted spectrum components with an empirical attenuation function, thereby enabling the optically-sectioned, super-resolved image to be obtained.14 Due to the effectiveness of this reconstruction protocol in enhancing the spatial resolution and improving sectioning capability, it is widely used in commercial SIM systems and open-source tools.17^,25^,26

However, the complex workflow makes the SR reconstruction extremely time-consuming, limiting the utility of SIM as a live-cell imaging modality. To address this issue, SDR was developed23 by clarifying the resolution enhancement of SR-SIM from the perspective of point spread function engineering, in which the overall workflow is simplified to multiplication and summation calculations in real space (Supplementary Note 1 in the Supplemental Materials). Eventually, the SDR method results in a sevenfold increase in the SR-image reconstruction speed. However, the reconstruction result of SDR is equivalent to a Wiener-free reconstruction protocol where the spectrum components O(k)H(k), O(k)H(k+k0), and O(k)H(k−k0) are overlapped without OTF compensation and an intermediate superresolved image, without deconvolution, is obtained by transforming the superimposed spectrum back to real space (see following discussions and Supplementary Note 1 in the Supplemental Materials). Thereafter, a routine Wiener deconvolution is implemented by treating the intermediate image as a new image acquired with a shrunken PSF.21^,23 As the out-of-focus background is not taken into consideration, this method is not suitable for SR imaging of thick specimens (Fig. S1 in the Supplemental Materials).

To address these issues, we develop a concise reconstruction protocol to rapidly obtain optically-sectioned, super-resolved images of thick specimens. We first explored the connection of the SDR method and the conventional Wiener-SIM by deducing another equivalent workflow implemented in real space (see Fig. S2 in the Supplemental Materials). Specifically, in one-dimensional cases, the intermediate results of these two different algorithms can be converted to each other by the following equations: $$\begin{gathered} R0(x)=F−1[O˜(k)·H˜(k)], \\ R1(x)=eiφ02F−1[O˜(k−ω)·H˜(k)]+e−iφ02F−1[O˜(k+ω)·H˜(k)], \\ R2(x)=eiφ02iF−1[O˜(k−ω)·H˜(k)]−e−iφ02iF−1[O˜(k+ω)·H˜(k)], \end{gathered}$$(1)and $$\begin{gathered} O˜(k)·H˜(k)=F[R0(x)], \\ O˜(k−ω)·H˜(k)=e−iφ0F[R1(x)+iR2(x)], \\ O˜(k+ω)·H˜(k)=eiφ0F[R1(x)−iR2(x)], \end{gathered}$$(2)where ω and φ0 respectively denote the angular frequency and the initial phase of the illumination field, the recombined raw images R0(x), R1(x), and R2(x) are linear combinations of the three raw images with specific coefficients, and O˜(k)·H˜(k), O˜(k−ω)·H˜(k), and O˜(k+ω)·H˜(k) represent the frequency components separated in frequency domain processing. These equations verify that the result of the SDR method is equivalent to the conventional Wiener-SIM workflow without OTF compensation and OTF attenuation. More detailed deductions and discussions about these equations can be found in Supplementary Note 1 in the Supplemental Materials.

Based on the above relations, we demonstrate that OTF compensation and OTF attenuation in conventional Wiener-SIM are equivalent to a preset filtering operation on the initial raw images (see details in Supplementary Note 2 in the Supplemental Materials), that is, Dd,i′(r)=Dd,i(r)⊗F−1{[1−a(k)]·H˜*(k)},(3)where Dd,i(r) denotes the raw image in dth direction (d=1,2,3), i is the index of phase-shifting (i=1,2,3), 1−a(k) is the attenuation function to suppress the fluorescence background and the honeycomb artifacts in Wiener-SIM, H˜*(k) is the complex conjugate of the OTF of the system, and F−1(*) denotes the inverse Fourier transform.

By replacing the initial raw images Dd,i(r) in the SDR method with the filtered raw images Dd,i′(r), an optically-sectioned, super-resolved image equivalent to that of Wiener-SIM can be obtained via the following equation (see the details in Supplementary Note 2 in the Supplemental Materials) IJSFR−SIM(r)=F−1{F[∑d=13∑i=13cd,i(r)Dd,i′(r)]×{∑d=13{[1−a(k)]|H˜(k)|2+[1−a(k+kd)]|H˜(k+kd)|2+[1−a(k−kd)]|H˜(k−kd)|2}+α2}−1},(4)where cd,i(r) is the constant coefficient function in the SDR method, and α denotes the empirical parameter for Wiener deconvolution.

As such, a simplified version of the Wiener-SIM protocol is obtained by combining spatial domain processing with the conventional workflow implemented in the frequency domain, termed JSFR-SIM (Fig. 1, top panel). The resulting protocol contains only four steps and most Fourier domain operations are replaced with simple multiplication and summation calculations in real space. As a preparation procedure, the parameters of the illumination field, including the wave vectors, the initial phases, and the modulation depth are first estimated by the cross-correlation method, with which the coefficient functions can be simultaneously calculated for later use. Among those parameters, the initial phases can also be calculated with other estimation methods such as image recombination transform27^,28 or auto-correlation reconstruction,29 which were specially designed for more complex circumstances. Then, nine filtered raw images are obtained by filtering the raw images with an attenuated OTF [1−a(k)]H˜*(k) as in Eq. (3). By using an approximated OTF calculated by the parameters of the detection objective and emission wavelength, the attenuated OTF has a doughnut shape (Fig. 1, top panel). Next, nine intermediate images are obtained by multiplying the precalculated coefficient functions with the filtered raw images. The final SR image is recovered by superimposing the intermediate images and completing the Wiener deconvolution.

All the experiments of this study were carried out on a custom-built SIM setup, which is constructed by employing a ferroelectric liquid crystal spatial light modulator (SLM) to generate the fringe illuminations (Fig. S3 in the Supplemental Materials). The illumination light from a solid-state laser of 405, 488, 561, and 638 nm (L4cc, Oxxius Inc., Lannion, France) was first expanded and collimated with a telescope system (neglected in the schematic) composed of two convex lenses. The collimated beam is then directed into a pattern generator consisting of a polarized beam splitter, an achromatic half-wave plate, and a ferroelectric-liquid-crystal (FLC) SLM (QXGA-3DM-STR, 2048 × 1536 pixels, 4.5 kHz, Forth Dimension Displays Ltd., Scotland, United Kingdom). After being modulated by the FLC-SLM, the incident beam is diffracted into various orders. The ±1st-order diffractions are selected by a customized spatial filter and converted to circular polarization by an achromatic quarter-wave plate (QWP; Thorlabs, Newton, New Jersey, United States). To compensate for the polarization ellipticity caused by the dichroic mirror, a ferroelectric liquid crystal variable retarder (FLC, LVR-200-VIS, Meadowlark Optics Inc., Frederick County, Maryland, United States) is inserted before the QWP in the light path. The selected two coherent beams are then relayed onto the back focal plane of the microscope objective (Apo TIRF, NA1.49, Nikon Inc., Tokyo, Japan) by a telescope system composed of lens 2 and lens 3. Afterward, the two beams recollimated by the objective lens interfere at the focal plane, producing sinusoidal patterns on the specimen. Eventually, the emission light from the specimen is collected by the objective and imaged onto the sensor of an sCMOS camera (Orca Flash4.0 V3, 100 fps@2048 × 2048, 16 bits, Hamamatsu Inc., Hamamatsu, Japan). The specimen is mounted on a motorized XY and piezo Z-axis translation stage (PZ-2150-XYLE-FT piezo Z system, ASI Inc., Eugene, Oregon, United States). To acquire the raw images for SR-SIM, all the electrically-controlled devices of the system are synchronized by an NI-DAQ (USB-6003, National Instruments Inc., Austin, Texas, United States) via custom-developed software written in Qt C++. The time sequence for image acquisition is similar to that of Ref. 7.

Using JSFR-SIM, a real-time observation pipeline is constructed by utilizing the multithreading technique, in which four child threads are created to realize data acquisition, instant reconstruction, instant result display, and instant result storage. For a single reconstruction, these threads executed independently are triggered one after another, which therefore releases the data congestion and reduces the delay between the measurement and display. First, the acquisition thread successively acquires the nine raw images by triggering the preset timing sequence, after which the raw images are instantly transferred to the RingBuffers. After all the nine raw images of a single SR frame are gathered in the RingBuffers, the reconstruction thread immediately starts the SR reconstruction by uploading the raw images to the graphic memory of the GPU. Once the reconstruction is finished, the result is immediately downloaded to the local memory from the graphic memory, and the display thread and the saving thread instantly display the widefield image and SR image on the screen and save the raw images and the reconstructed images into the solid-state drive (SSD). The transmission of the trigger signal among these threads was achieved by the signals and slots mechanism of Qt. By repeating these procedures in every thread, this pipeline allows continuous acquisition, reconstruction, display, and storage of the results. The latency between the acquisition and display was determined to be less than 80 ms with an region of interest of 1024×1024.

By using a simplified workflow, the JSFR-SIM scheme was anticipated to significantly enhance the image reconstruction speed (Fig. S2 in the Supplemental Materials). To test this, we evaluated the execution time of JSFR-SIM as a function of image size and compared it with Wiener-SIM. As most users perform image reconstruction using the central processing unit (CPU) of the computer, comparisons were first done in this environment. Results show that for each image size, JSFR-SIM reconstructs an SR-image 2.4- to 2.9-fold faster than Wiener-SIM (Table 1).

Previous work showed that reconstruction speed can be further enhanced when code is executed in the GPU environment as GPUs execute calculations in a parallel fashion.30 Consequently, the code from both algorithms was converted to the GPU-accelerated format. This results in a 32-fold increase in processing speed for the reconstruction of a 1024×1024 image by JSFR-SIM and a 20-fold increase for Wiener-SIM. In this environment, JSFR-SIM reconstructs images, on average, fourfold faster than Wiener-SIM, corresponding to an 80-fold faster time than Wiener-SIM in the more widely-used CPU environment. Critically, the reconstruction speed using Wiener-SIM is fourfold slower than the acquisition time whereas JSFR-SIM reconstructs images more rapidly than it takes to acquire them. Thus, even in the GPU-accelerated environment, Wiener-SIM is not suitable for real-time imaging whereas JSFR-SIM is significantly closer to achieving this goal.

To test the performance of the algorithm in improving the spatial resolution and optical sectioning capability, we first simulated the image formation and reconstruction process of a simulative object, which is composed of a tilted resolution target sliced into one hundred layers. The raw SIM images are generated by the general forward imaging model for SR-SIM in Eq. (1) of the Supplementary Note 1 in the Supplemental Materials, in which the 3D PSF of the system was calculated with the expression provided by Hanser et al.31 As expected, both the wide-field and SDR-SIM images are deteriorated by the out-of-focus background due to the missing-cone problem [Figs. 2(e)–2(f)]. In contrast, by employing OTF compensation and OTF attenuation, the JSFR-SIM removes the out-of-focus information and significantly improves the lateral resolution [Figs. 2(h), 2(l), 2(p), and 2(q)], which is identical to that of the conventional Wiener-SIM [Figs. 2(g), 2(k), 2(o), and 2(q)].

Experimental results on fluorescence beads and the Argo-SIM slide using a home-built laser-interference SIM (Fig. S3 in the Supplemental Materials) further demonstrate that the spatial resolution and optical sectioning capability of JSFR-SIM are indistinguishable from that of Wiener-SIM, as shown in Fig. 3 and Fig. S4 in the Supplemental Materials. The adjacent two beads, which were unable to be resolved in the widefield image, are separated in the JSFR-SIM and Wiener-SIM images [Fig. 3(b)]. Also, quantitative results indicate that the lateral resolution of both JSFR-SIM and Wiener-SIM show an improvement of ∼115% over the wide-field microscopy. Although JSFR-SIM and Wiener-SIM cannot enhance the axial resolution, their optical sectioning capability to reject the out-of-focus background is significantly improved, as exhibited by the xz cross-sectioned images of the isolated bead in Fig. 3(c). Details of the resolution tests can be found in Supplementary Note 3 in the Supplemental Materials.

To assess the capability of JSFR-SIM in biological imaging, we experimentally observed the microtubule cytoskeleton in a prepared slide of thick COS-7 cells. The cell in Fig. 4 stretches out within a depth range of 7μm, and the whole volume of 74μm×74μm×7μm is acquired at an axial scanning interval of 0.2μm with 35 steps. The strong background fluorescence of the cytoskeleton in the widefield images [Fig. 4(b)] is effectively suppressed in the result of JSFR-SIM [Figs. 4(a) and 4(c) and Video 1). In addition, as expected, the image quality and spatial resolution of JSFR-SIM are virtually identical to that of the conventional Wiener-SIM [Figs. 4(d)–4(g)]. Similar to Wiener-SIM, JSFR-SIM can also recover the dual-color 3D dynamics of thick COS-7 cells (Fig. S5 in the Supplemental Materials and Video 2). Therefore, JSFR-SIM is capable of producing high-quality SR images with exceptional spatial resolution at a significantly more rapid reconstruction speed and, importantly, the increased speed does not come at the expense of the spatial resolution or sectioning capability.

To further evaluate JSFR-SIM, we used our laser-interference SIM system to facilitate the real-time observation of cytoskeleton dynamics in living cells. The data collection, SR reconstruction, image display, and data storage are executed with four child threads in a single personal computer to reduce the delay caused by data transmission and to minimize the data jamming in every single procedure.

We first followed the microtubule motion in live COS-7 cells (Fig. 5 and Video 3). To avoid motion blur over a prolonged time, we captured a time series by focusing on a single focal plane with 5 Hz frame rates and 10 ms raw frame exposure times. The microtubule networks near the cell boundary show a sharp and clear filamentous structure [Fig. 5(a), right]. In contrast, filaments near the cell center are immersed in the high background [Fig. 5(a), left]. This is due to increased cell thickness in this region and the centrosome and Golgi, which serve as microtubule organization centers nearby.32^–34

In the SR image reconstructed with JSFR-SIM in near real time, the fuzzy out-of-focus background observed in the widefield image is eliminated and the fine structures of the cytoskeleton are revealed, particularly in the region near the cell center [Fig. 5(b)]. To further demonstrate the utility of image reconstruction by JSFR-SIM in revealing subcellular dynamics, we focused on the boxed region, which has a reasonably high background surrounding the microtubule network [Fig. 5(c) and Video 3]. This analysis revealed that some microtubules disassociated from the microtubule intersection [Fig. 5(c), blue arrows], while another microtubule rapidly disassembled [Fig. 5(c), yellow arrows]. Also, a long-duration microtubule assembly event is recorded, enabling the assembly velocity of the microtubule tip as a function of time to be easily calculated [Fig. 5(d), arrows and 5(e), respectively]. As microtubule intersections and microtubule dynamics are crucial for motor-protein-based cargo trafficking,34^,35 our system is anticipated to provide a powerful tool for studying intracellular cargo transport and the effects of numerous drugs on these processes.

In recent years, mitochondria cristae have been gathering more and more attention from biologists due to the growing evidence for the contributions of mitochondrial fine structure deformation and dysfunction to mitochondrial disorders.36^–40 Although mitochondria have been observed using static imaging approaches including electron microscopy and more recently stimulated emission depletion microscopy (STED) and stochastic optical reconstruction microscopy (STORM),8^,41^,42 mitochondria dynamics are challenging to image as their dynamics are extremely light-sensitive. Further, unhealthy mitochondria that may have lost their membrane potential do not stain well with mitochondria-specific dyes, further complicating the imaging (Fig. S6 in the Supplemental Materials). In addition, in the investigation of mitochondria, the mitochondrial cristae are the main features of interest, which, however, cannot be resolved by widefield fluorescence microscopy. This combined with the above-mentioned challenges makes locating an ideal FOV in a time-efficient way an exceptionally challenging process. For example, in conventional SR-SIM, in the absence of real-time high-resolution and background-free feedback, locating the FOV is a laborious and time-consuming process for the microscope operators. They have to begin in the widefield mode to locate the anticipated FOV, then switch to the SR-SIM mode to acquire the raw images and wait as long as several seconds to obtain the reconstructed SR image. By this time, a poor FOV selection may be made that can only be discerned at the SR-SIM, and the biological process may have gone to completion or otherwise critical data missed entirely. In sharp contrast, in JSFR-SIM, the operators will initiate imaging in SR-mode, navigate to an FOV and obtain the SR image in less than 100 ms, with which a decision to continue imaging or move to a separate FOV can be rapidly made. As such, locating the ideal FOV aided by JSFR-SIM is a facile process enabling the routine visualization of the dynamics in the superresolved structure of mitochondria.

To demonstrate the routine SR imaging, we continuously imaged continuous mitochondrial dynamics in near real time using a 5 ms exposure time (Fig. 6). The mitochondrial cristae, which remain fuzzy in the wide-field images, can be distinguished in the OS-SR images [Figs. 6(a) and 6(d)]. With the magnified views, plenty of mitochondrial dynamics, including the mitochondrial tubulation events [Figs. 6(b) and 6(e)] and the inter-cristae merging event [Fig. 6(c) and Video 4], can be identified via the images recovered with JSFR-SIM. For example, a tubulation event between two mitochondria was readily visible between 3.65 and 5.67 s, as shown in Fig. 6(e). Here, a mitochondrion extended a tubulation tip ∼1.4μm, made contact with another mitochondrion (at t=4.95s), then immediately retreated in the opposite direction.

A recent study indicated mitochondria redistributed inner materials via dynamic tubulation including mtDNA.43 As the generation of these dynamic tubules and their subsequent fusion are crucial for forming mitochondrial networks, our system provides a convenient tool to reveal the rapid processes of these and other dynamic organelles in real time. Owing to the short exposure used to generate the images in Fig. 6, the SR images are degraded by random noise. To improve the image quality, a longer exposure time may be used with images reconstructed by JSFR-SIM, with the caveat that the extended illumination may perturb mitochondrial function.

We have demonstrated a rapid algorithm to instantly recover superresolved images from raw SIM images. This makes locating an FOV, imaged at SR, a facile and routine process, so that complex intracellular dynamics can be visualized in real time and resulting images quantitated. As verified by theoretical derivation and experimental results, the final resolution and quality of the image obtained with JSFR-SIM are identical to that of conventional Wiener-SIM. Moreover, this method enhances the reconstruction speed of SR-SIM over 80-fold with the help of GPU acceleration, allowing real-time SR reconstruction and display for all frequently-used image sizes (see Table 1). Combining with the VIGOR framework developed by Markwirth et al.,18 this approach can achieve real-time, multi-color observation with a larger FOV and shorten the latency between measurement and display. Furthermore, the speed enhancements of the JSFR-SIM are not limited to real-time observation; that is, the post-processing programs for SR-SIM such as fairSIM,17 OpenSIM,25 or SIMToolbox26 can also benefit from its fast reconstruction speed, which is particularly useful for the SR reconstruction of time-lapse movies.

JSFR-SIM is compatible with various SIM modalities as it shares the same hardware setup as the conventional 2D-SIM. Thus, it can be easily applied in many new SIM modalities including adaptive-optics-aided SIM (AO-SIM),11 polarized SIM (pSIM),7 and grazing incidence SIM (GI-SIM)8 to release the computing burden in either online or offline ways, even when imaging thicker samples. In addition to accelerating the image reconstruction speed, the components of JSFR-SIM that exclude background information could also be incorporated into other SIM modalities. For instance, the four-frame SR-SIM44^–47 methods developed to reduce the required number of raw images never took the background information in the raw images into account, which limits their applications in thick samples. As we have demonstrated that the out-of-focus information in the raw images can be removed by a high-pass filtering operation on each raw image in JSFR-SIM, we can probably improve the image quality of four-frame SR-SIM when imaging thick samples by incorporating this component of JSFR-SIM into four-frame SR-SIM, and we predict a further increase in overall imaging speed.

In this work, a rapid and simple SR reconstruction scheme for thick samples was presented. It is termed JSFR-SIM and combines spatial domain processing with the OS-SR-SIM implemented in the frequency domain. SR images are reconstructed 80-fold faster than conventional algorithms. Importantly, the enhanced reconstruction speed does not come at the expense of the image quality of the optically-sectioned, super-resolved images. We anticipate that by utilizing a GPU enhanced computer further increases in reconstruction speed will be attained, thereby bringing us ever closer to the goal of real-time SR imaging of live cells. This breakthrough will greatly improve the work efficiency of biologists and facilitate SR-SIM as a routine tool in more biomedical laboratories.

Zhaojun Wang is an assistant professor at Xi’an Jiaotong University. He received his BS degree in optical information science and technology from Huazhong University of Science and Technology in 2013, and his PhD in optics from Xi’an Institute of Optics and Precision Mechanics in 2019. His current research interests include three-dimensional imaging and super-resolution optical microscopy.

Tianyu Zhao is an assistant professor at Xi’an Jiaotong University. He received his BS degree in electronic science and technology from Tianjin University in 2014, and his PhD in optics from Xi’ an Institute of Optics and Precision Mechanics in 2021. His current research interests include super-resolution structured illumination microscopy and three-dimensional imaging.

Huiwen Hao is a postdoctoral fellowship at Peking University, school of Future Technology. She received her BS degree in engineering from Northwest University (China) of Biotechnology in 2013, and her PhD in biophysics from Peking University in 2019. Her current research interests include high-throughput single-molecule super-resolution system and precise protein map of neuronal synapses.

Yanan Cai is an assistant professor at Northwest A&F University. She received her BS degree in physics from Northeast Forestry University in 2013, and her PhD in optics from Xi’an Institute of Optics and Precision Mechanics in 2020. Her current research interests include super-resolution optical microscopy and optical tweezers.

Kun Feng is currently a postgraduate student at the School of Physics, Xi'an Jiaotong University. He received his BS degree in electronic science and technology from Xidian University in 2020. His research activity involves three-dimensional imaging and super-resolution optical microscopy.

Xue Yun is a postgraduate student at the School of Physics, Xi’an Jiaotong University. She received her BS degree in physics from Shanxi Normal University in 2019. Her current research interests are optical trapping and spatial modulation of optical fields.

Yansheng Liang is an assistant professor at Xi’an Jiaotong University. He received his BS degree in optical information science and technology from Nankai University in 2011, and his PhD in optics from University of Chinese Academy of Sciences in 2018. His current research interests include optical trapping and beam shaping.

Shaowei Wang received his BS degree in precision instrument from Tianjin University in 2011 and his PhD in optical engineering from Zhejiang University in 2016. After postdoctoral training at the National University of Singapore, he joined Xi’an Jiaotong University where he is currently a full professor in the School of Physics. His research focuses on the development of advanced multiphoton fluorescence imaging technologies for biomedical applications.

Yujie Sun obtained his BS and PhD degrees in chemistry from USTC and University of Pittsburgh, respectively. After his postdoc training at the University of Pennsylvania School of Medicine, he joined in BIOPIC at Peking University and is currently a Boya-distinguished professor. His research has been focused on developing advanced single molecule and super-resolution imaging techniques to study cellular structures and functions, and he has published more than 100 peer-reviewed works.

Piero R. Bianco conducts single molecule studies of DNA binding proteins involved in genome stability. He developed a single-molecule system that combined optical tweezers, microfluidics, and video-fluorescence microscopy that enabled him to be the first to visualize DNA unwinding by a single DNA helicase in real-time. He continues to do single molecule studies using optical and separately, magnetic tweezers and, in collaboration with Ming Lei, develop high-speed super-resolution imaging, optimized for prokaryotic cells.

Kwangsung Oh is an assistant professor in the Department of Computer Science at the University of Nebraska at Omaha. His research interests include high performance computing, distributed systems, data analytics systems, and cloud computing. He received his BS degree in Computer Science and Engineering from Sejong University, Korea, and his MS and PhD degrees in Computer Science from the University of Minnesota-Twin cities.

Ming Lei is a professor at Xi’an Jiaotong University. He received his BE degree from the School of Physics and Optoelectronic Engineering, Xidian University in 2000, and his PhD from Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, in 2007. He was trained as a postdoctoral research fellow in the Department of Chemistry, University of Konstanz, Germany, from 2008 to 2010. His research is focused on super-resolution microscopy and optical trapping technologies.