Microscopy is vital in biology [1–3], medicine [4,5], material science [6–8], and chip manufacturing [9,10]. Due to the diffraction limit, traditional optical microscopy can achieve the highest resolution of around 200–350 nm under visible light illumination (wavelength of around 400–700 nm) [11–14]. The objectives for high-resolution microscopic imaging usually have a classical numerical aperture (NA) value near 1.0. As we all know, the objective working distance decreases with increasing NA. The objectives’ working distance is usually smaller than 0.2 mm to achieve a high NA. The short working distance and large magnification usually lead to micrometer or even nanometer depth of field (DOF) in objective. Such a small DOF makes it challenging for traditional microscopy to focus fast on sections [15,16].

Computational microscopy is currently preferred to improve resolution and break the diffraction limits of traditional microscopy [17,18]. Heavy optical modulations and iterative computations are combined necessary to acquire a super-resolution reconstruction. Especially, single-pixel imaging (SPI), as a computational imaging method, usually reconstructs images with many pattern modulations. 128×128-pixel and 256×256-pixel resolution Fourier SPI imaging requires 32,768 and 131,072 patterns, respectively [19–26]. The experimental process is usually time-consuming. So the fast and accurate autofocus for computational microscopy is an especially prickly problem [27].

Currently there are two main methods to solve the autofocus problem: active focus and passive focus [28–30]. The active methods, namely measuring distance, employ infrared, ultrasonic, and laser sources for autofocus [31,32]. The sensors are necessary for active methods to measure infrared light intensity or propagation time from reflected signals. The active methods cannot be compact and are usually unavailable when the target is transparent. The passive methods include phase detection and contrast maximization [33,34]. The depth from focus and depth from defocus are two classical approaches. These passive autofocus methods, with the help of a focus search algorithm, usually are time-consuming to find a focus position. Moreover, the focus precision is limited by the acquired image resolution. A high-resolution image is necessary to judge the accurate focus position.

The traditional passive autofocus methods show low precision when used in the low-resolution SPI. A dual modulation method is proposed to solve the SPI defocus phenomenon [35]. The dual modulation means that a grating is added in SPI optics paths to modulate the target. Theoretically, the grating should be the conjugation plane of the spatial light modulator (SLM). The co-axial and off-axial configurations were designed for dual modulation. However, the dual modulation method has not been validated in single-pixel microscopic imaging (SPMI) autofocus. In the SPMI system, co-axial configurations are unavailable when switching different magnification objectives. Due to the objective’s short DOF and working distance, it is challenging for off-axial configurations to adjust the grating and SLM to be appropriate conjugation positions. Therefore, to achieve ideal and flexible autofocus in SPMI, a superior method compatible with various magnification objectives is pressing [36]. In addition, the defocus extent of traditional SPMI is only limited by the projection lens. However, if a dual modulation configuration is used for SPMI refocusing, the lens that images the target to the grating brings an additional uncertain factor that causes the reconstructed image to defocus. As a systematic error, this vital uncertain factor exists along with the dual modulation. The introduction of grating in the optics paths also interferes with SPMI and inevitably leads to poor image quality. Therefore, research on the grating-free modulation autofocus method is necessary.

This paper proposes a grating-free method to achieve SPMI’s focus by maximizing high-frequency information. Only the PD-measured time sequential signals are needed to determine whether the optical system is in focus. First, we will demonstrate that this method is derived from the physical mechanism of optical defocusing. Then, the experiments for fast focusing on the sample with different NA objective lenses will be carried out. The Fourier spectra are employed as an example to demonstrate the availability of the proposed method. Further, the Hadamard spectra are also effective for the grating-free method. Finally, our method will be validated for biological samples in addition to the binary sample. In the following, the autofocus experiments under the grating modulation and grating-free method will both be implemented. We will also discuss the autofocus robustness when the system is out of focus and objectives magnification varies.

In the digital image process, we often need convolution to smooth the target image and detect edges. In optics, convolution can also describe the out-of-focus extent (known as the photographic term “bokeh”) after lens function modulation. SPI patterns will convolute with the point spread function (PSF) in the space-invariant system when passing the objective. After the convolution process, we can obtain a blur image from an ideal on-axis point source. The PSF is the convolution kernel mathematically approximated with a 2D Gaussian profile: h(x,y)=12πσ2e−(x−μx)2+(y−μy)22σ2.(1)

(μx, μy) is the center of the PSF. The full width at half-maximum (FWHM) is given by 2.3548σ.

When the image is convoluted by the PSF, σ is a sign to quantify the image’s blur extent. The different blur extents are results of out-of-focus objective lens imaging and correspond to different PSF σ values.

Considering the space-invariant incoherent system, the image intensity distribution is I(x1,y1)=∬O(x0,y0)h(x1−x0,y1−y0)dx0dy0.(2)

O(x0,y0) is the object, and I(x1,y1) is the image. h is the Gaussian PSF. The above equation can be rewritten in the following convolutional form: I(x,y)=O(x,y)*h(x,y).(3)

* is the convolution symbol. Applying the Fourier transform to the above equation, we can obtain P(u,v)=S(u,v)H(u,v).(4)

P, S, and H are spectra of I, O, and h, respectively. The discrete spectrum expression of the Gaussian PSF is H(u,v)=∑x=1N∑y=1Nh(x,y)e−2πi(uxN+vyN).(5)

H(u,v) is also the objective lens’s optical transfer function (OTF). Figure 1(a) shows the OTF corresponding to σ=1,2,3,4. As the value of σ increases, the high-frequency gain decreases rapidly. As a result, many high-frequency components are filtered by the optics system. Therefore, due to the objective lens acting as a filter for the target’s spectrum, the acquired image often lacks sufficient high-frequency information in an out-of-focus imaging system. However, it is the high-frequency domain that corresponds to the detailed characteristics of the reconstructed image. Hence, the objective is to restore wide-field high-frequency information when conducting SPMI.

To demonstrate the proposed grating-free autofocus method, we have set up two microscopy light paths, one with dual modulation and the other without grating modulation. As shown in Figs. 1(c) and 1(d), the patterns for traditional SPI are generated by the DLP. The light screen is a ground glass located at the focal plane of the DLP. The optical modulation patterns are imaged on the sample plane by Objective 1. The USAF-1951 card is used as the target. The key difference between dual modulation and grating-free modulation lies in the utilization of Objective 2 and a grating.

In the dual modulation optical path, Objective 1 is typically initially defocused. The objective is to adjust the position of Objective 1 to ensure that the modulation patterns focus accurately on the target. The defocusing of Objective 1 can be mathematically represented as a convolution operator. Therefore, the original extent of defocus of the patterns is P1(x,y,u,v)=P0(x,y,u,v)*h1(x,y),(6)where P0 is initially generated patterns. (x,y) and (u,v) are the indices in spatial and frequency areas, respectively. h1(x,y) is the PSF of Objective 1 and represents the defocus extent in the target plane.

Subsequently, the defocused patterns P1 pass through the target and are imaged by Objective 2. The modulation at the target plane is P2(x,y,u,v)=P1(x,y,u,v)·O(x,y),(7)where P2 represents the 2D intensity distribution of patterns after passing through the target. The light beam P2 is then captured by Objective 2 and imaged on the grating.

Objective 2 still defocuses even after being focused by the naked eye. Consequently, the patterns will be blurred by Objective 2, and can be mathematically described as P3(x,y,u,v)=P2(x,y,u,v)*h2(x,y)·G(x,y),(8)where h2(x,y) represents the PSF of Objective 2 from the target plane to the grating plane. G(x,y) represents the grating in the dual modulation method. In previous studies, Objective 2 typically had a short focal length and small numerical aperture [35], resulting in a larger DOF which made it easier to focus with the naked eye. However, in microscopic imaging, an objective with an extremely short DOF is utilized to project optical patterns onto the grating. Figure 1(b) shows the objective’s DOF and WD with different NAs. The focus process of Objective 2 relies solely on the naked eye. In this process, accurately shifting the objective lens (NA=0.65) into a 1 μm effective DOF under a 500 μm WD condition is practically impossible. The blur effect caused by Objective 2’s out-of-focus condition cannot be ignored. Addressing the out-of-focus issue of Objective 2 is crucial for mitigating the blurring extent in the dual modulation microscopy system.

The PD is used to measure intensity signals transmitted from the grating. The signals have mathematical form as Y0=a+b∬ΩP3(x,y,u,v)dxdy,(9)where a represents the response of environmental illuminations, and b is the loss coefficient of the light beam. According to the Helmholtz reciprocity principles, the light source and PD are conjugate with each other. When the whiteboard is used as target, namely, O(x,y)=1, the measured signals Y1 can be rewritten as Y1∝∬ΦG(x,y)*h2′(x,y)*h1′(x,y)·P0(x,y,u,v)dxdy.(10)

h2′(x,y) represents the PSF of Objective 2 from the grating to the target plane. h1′(x,y) represents the PSF of Objective 1 from the target to the light screen plane.

From the above equation, the reconstructed image is given by G(x,y)*h2′(x,y)*h1′(x,y) using the SPI algorithm. In the presence of ideal optical systems, h2′ would be a δ impulse function. The next objective is to adjust Objective 1 in order to achieve h1′(x,y)=δ(x,y), allowing Eq. (10) to be expressed as Y1∝∬ΦG(x,y)·P0(x,y,u,v)dxdy.(11)

However, regulating the PSF of the system to an ideal δ impulse function is beyond the limits of human visual ability. Actually, in the dual modulation, the autofocus goal is to shift Objective 1 to make h1′(x,y)*h2′(x,y)=δ(x,y) rather than h1′(x,y)=δ(x,y). Since the convolution of two PSFs is equal to the δ impulse function, then these two PSFs are inverses of each other, which are mathematically denoted as h1′(x,y)=(h2′(x,y))−1. Accordingly, we will find it is the grating rather than the target that is focused in the dual modulation experiments. In this process, once Objective 2 is defocused, the grating can maintain high resolution, and the target will be blurred after the dual modulation autofocus method. Therefore, the blurring effect caused by h2′(x,y) cannot be ignored and should be solved to achieve focus on the target rather than the grating.

In Fig. 1(d), Objective 2 and the grating are removed, and the optical system is simplified. The PD measured signals are as follows: Y1∝∬ΦO(x,y)*h1′(x,y)·P0(x,y,u,v)dxdy.(12)

In this optical system, there is no concern whether Objective 2 is out of focus.

The autofocus experiments for SPMI will be done using dual and grating-free modulations to demonstrate the advantage of the grating-free method. The experimental setups are in Appendix A. The detailed procedures are shown in Fig. 2. In the following experiments, Objective 1 (4×, NA=0.1) is used for imaging SPI patterns onto the target. The autofocus goal is to adjust Objective 1 to an appropriate position. In the dual modulation, we must use Objective 2 (10×, NA=0.25) to image the target onto the grating. The USAF-1951 card is the target, and 128×128-pixel patterns and grating (14 line pairs per mm) are used for dual modulation.

In the dual modulation method, an extra calibration is done to acquire the grating frequency. The coordinate of the grating Fourier coefficient is a local maximum value that is easy to distinguish. As shown in Fig. 2(a), the highlighted coordinate g1(u,v) represents the grating frequency in the SPMI system. At present, the amplitude is 4.71 due to the out-of-focus SPMI system. The patterns that correspond to the grating frequency are derived by the four-step phase-shifting equation in Fig. 2. Then, the four-step phase-shifting patterns are employed to illuminate the target repeatedly. The PD measures the photoelectric responses as Objective 1 moves back and forth. The frequency amplitude is maximized when the SPMI patterns are focused on the grating by shifting Objective 1. The red circle highlights the position on which Objective 1 is to focus. The SPMI experiment is conducted again when Objective 1 is adjusted to be in focus. Finally, the reconstructed image demonstrated that it is the grating rather than the target being focused and deblurred. The effectivity of dual modulation autofocus is limited by the in-focus of Objective 2. In the experimental SPMI system, the pattern size focused on the target is less than 2mm×2mm. The focus state of Objective 2 is hardly distinguishable only with the help of the naked eye. The above demonstration shows that the dual modulation method is invalid in the SPMI autofocus application.

In the grating-free modulation, arbitrary high-frequency Fourier coefficients can be employed to replace previous extra-grating modulation. As we all know, noises are inevitable during SPMI experiments, and the high-frequency energy is generally weak. The high-frequency coefficients with small amplitude usually have poor PSNR. Therefore, the local maximum coefficients are preferred in the high-frequency domain to improve the focus accuracy. To verify the generality and universality of the grating-free method, the high-frequency coefficient is chosen as the eigenvalue that is to be maximized. After projecting patterns and shifting the objective, the accurate focus position is quickly captured according to the highest response value. In Fig. 2(b), the red circle highlights the focus position [amplitude is 0.29, frequency coordinate is (16,16)] under the grating-free method. The frame rate of SPMI patterns is 22.7 kHz. When the objective lens is moved, we just need to judge whether the measured eigenvalue amplitude from PD signals is equal to 0.29. The time for estimating the objective’s immediate in-focus extent is less than 180 μs under four-step phase-shifting patterns illumination. Then, the objective is moved to the position with a measured response amplitude of 0.29. In Fig. 2(b), the amplitudes of high-frequency coefficients are enhanced compared with that in initial out-of-focus spectra. Finally, the out-of-focus existing in the SPMI system is removed, and the reconstruction is deblurred without grating interference.

The wide field and high resolution are usually incompatible in SPMI. The reconstructed images under different magnification microscopic imaging are in demand. To satisfy the requirement of autofocus under these flexible scenarios, we will validate the autofocus behavior with dual modulation and grating-free methods. In the following, the autofocus experiments with switching different NA objective lenses will be demonstrated. The USAF-1951 card is used as the target. The objective lenses with 10× (NA=0.25), 20× (NA=0.4), and 40× (NA=0.65) are used for different field of view imaging. The experimental comparison results are shown in Fig. 3. The energy distributions of the measured spectra are also shown. However, instead of the target, the grating image is focused when using the dual modulation method, as shown in Fig. 3(a). The grating’s frequency coefficient coordinates and amplitude are visible in the spectrum figures. The grating frequency is near the highest limit in SPMI with 4× magnification. The grating frequency beyond this reference value cannot be measured by our SPMI system. As the objectives’ magnification increases, the grating frequency quickly decreases. Therefore, either increasing or decreasing the system magnification will eventually lead to the failure of grating modulation. Even though the grating frequency is pre-matched with the SPMI system, the experimental results in Fig. 3(a) show that the target is still blurred.

In contrast, the SPMI system focuses on the target, and there is no grating interference in Fig. 3(b), which is acquired by the grating-free autofocus method. Further, the measured spectra demonstrated that the high-frequency components are restored. In addition, dual modulation is not an interference-free method due to the conjugate constraint between the grating and the SLM. Once the Objective 1 magnification changes, all other optics devices must be adjusted correspondingly. However, the grating-free method frees this constraint.

The focus target of the above experiments is USAF-1951, which has a simple image character. The biological samples have the character of natural and common Fourier spectrum. The robust SPMI autofocus method should be highly effective with biological samples. Here, to validate the SPMI autofocus ability for the pathology section, the samples of cervical polyp and esophageal cancer are used in our experiments. The autofocus results are in Fig. 4. First, the out-of-focus reconstructed images of biological samples are easy to acquire when SPMI experimental setups are done. Unfortunately, the main high-frequency energies are lost in the measured out-of-focus Fourier spectra. The residual high-frequency coefficients with local maximum energy are chosen as eigenvalues, which are available to derive the four-step phase-shifting modulation patterns. Next, moving the Objective 1, the PD response signals are recorded simultaneously according to the modulation patterns. The coefficient amplitudes can be acquired using a four-step phase-shifting algorithm to process PD signals. In the last row of Fig. 4, the blue rectangular areas highlight the corresponding in-focus position in the process of measuring high-frequency coefficient amplitudes. Visualization 1 and Visualization 2 depict the detailed focusing process in our experiments. Visualization 1 is a demonstration of autofocus for 4× magnification SPMI, while Visualization 2 showcases that for 20× magnification SPMI.

In the above experiments, the high-frequency eigenvalue maximum has been validated for efficient and fast SPMI deblurring. The blurred image is usually along with high-frequency information damage when the SPMI optics system is out of focus. The grating-free autofocus method in this paper is to acquire authentic target Fourier spectra by enhancing high-frequency information. However, the traditional dual modulation autofocus method has to impose an obvious one-order interference to the actual Fourier spectra with grating modulation. Even if SPMI is focused with this method, the primary target Fourier spectra are damaged, and a grating interference always exists in the reconstructed image. The idea in our method is derived from the convolution property and optical transfer model. The out-of-focus phenomenon that exists in SPMI imaging can be seen as the convolution process between the PSF and the target. The PSF is transformed into an OTF. The out-of-focus imaging process that PSF convolutes the target, is equal to that OTF multiplies the target’s Fourier spectra. In the out-of-focus imaging, OTF is similar to a low-pass filter. Thereby, the PD-measured high-frequency components are decreased in SPMI. Benefiting from this phenomenon, this paper aims to restore the initial high-frequency amplitude and achieve autofocus in the SPMI process. Different from the traditional pinhole imaging model, SPI has the nature of directly measuring target Fourier spectra. Thereby, the focus process is quickly done just by PD-measured voltage values, and no postprocess is needed. This method needs just a high-frequency amplitude, so the sensitivity to the objective lens’ defocusing is high. Conversely, if multiple frequency point energies are measured in combination to determine focus, the focus sensitivity of the method is sacrificed.

Benefiting from the SPI’s capacity of directly measuring Fourier spectra, the proposed method demonstrates greater advantages compared to the classical methods. In Fig. 5, the classical contrast maximization method and the proposed method are compared with flow block diagrams. The primary obstacle for employing the classical passive method to SPMI is to acquire multiple captured high-resolution frames, as shown in Fig. 5(a). Unlike traditional cameras, SPI uses huge Fourier basis patterns to modulate objects sequentially. The SPI time consumption for capturing multiple high-resolution frames is not ignored, which severely constrains the autofocus time response. However, in Fig. 5(b), only the amplitudes of a high frequency in the Fourier spectrum are enough for SPMI’s autofocus. The amplitudes are recorded by the PD when shifting the objective lens. Only a four-step phase-shift operation is needed, and the focus plane is found.

In the proposed SPMI autofocus method, the resolution requirement for the modulation patterns is that their highest spatial frequency must satisfy the need to measure the detailed features of the target. At this time, the autofocus accuracy of our method is unaffected by the modulation patterns’ resolution. When the highest spatial frequency of the modulation patterns does not satisfy the need to measure the detailed features of the target, the autofocus accuracy of our method is limited by the patterns’ resolution. At this point, the autofocus accuracy of our method is proportional to the resolution of the modulation patterns. Though the proposed grating-free autofocus method is effective and precise in SPMI, the autofocus speed is currently limited by the mechanical response of shifting the objective lens. For future optimization, one could consider designing a liquid lens instead of shifting objective lens mechanically in SPMI’s autofocus. Our grating-free autofocus speed could potentially be further enhanced if used in conjunction with three-step or two-step phase-shift algorithms. It is also possible to achieve invisible light autofocus by maximizing the Fourier coefficient without imaging. In addition, combined with the single-pixel phase contrast imaging technology, the proposed grating-free autofocus method is expected to be applied to transparent samples’ fast autofocus by measuring only the Fourier amplitude.

Accurate focusing is essential to improve the image quality of SPMI. We have proposed a grating-free autofocus method based on the physical mechanism of the SPMI defocusing. This mechanism is derived from circular convolution and discrete Fourier transformation. The experiment results have demonstrated that only a high-frequency amplitude is measured in SPMI autofocus, which significantly reduces data acquisition and information processing. The proposed autofocus method is an efficient focusing strategy for SPMI. The grating-free autofocus method is to minimize high-frequency loss and to restore high-frequency components of initial target spectra. So the SPMI system can focus on the target rather than the grating. The autofocus experiments have been performed with the targets of the USAF-1951 card and tissue sections. Only four patterns are used for optical modulation. The time to determine if the SPMI system is in focus is less than 180 μs at a DMD frame rate of 22.7 kHz. The image qualities are evaluated by the EOG metric. The EOG values of the image are enhanced twice after autofocus. The experiments have verified that the grating-free method achieves microfocusing with an NA maximum equal to 0.65 and DOF less than 2 μm. The experiments have shown that our grating-free autofocus method does not require imaging to achieve focus. So the method can be extended to realize autofocus for non-visible light SPMI.