Multiframe full-field heterodyne digital holographic microscopy

Xiaoyu Lv; Bin Xiangli; Wenxi Zhang; Zhou Wu; Yang Li; Xinxin Kong; Zhiliang Zhou

doi:10.3788/COL201614.050901

Holography was invented by Gabor^[1]. Based on interference recording and diffraction reconstructing, one can obtain the object’s 3D information. Digital holographic microscopy (DHM) is a technique that combines digital holography with microscopy. DHM is a non-contacting, non-scanning, non-dyeing, and quantitative technique that could be widely used in biology and microelectromechanical systems (MEMS) areas^[2–11].

There are two major problems in a DHM system, the twin-image problem and the random noise problem (from mechanical vibration and air turbulence)^[2,3,12]. These problems reduce the reconstruction precision and limit applications of DHM.

Off-axis holography was invented by Leith and Upatnieks^[13]. By adding a tilt angle between the object beam and reference beam, the reconstructions of a real image and a virtual image can be spatially separated. However, this configuration limits the number of usable pixels and causes image degradation.

Phase-shifting digital holography, invented by Yamaguchi^[14,15], is an efficient technology that suppresses the twin-image term, and its on-axis configuration allows efficient use of the detector. The phase shifting is achieved by moving the mirror with a piezoelectric transducer (PZT). Four-step holograms that have $π / 2$ phase shifting were detected. Using corresponding algorithms, the twin-image term could be eliminated. However, the phase-shifting accuracy is affected by nonlinear error, which limits the precision of the reconstructed image^[16].

The heterodyne method can solve the precise phase-shifting problem. Optical scanning holography (OSH) was invented by Poon^[17–19]. This technology uses an acousto-optic modulator (AOM) to create a MHz-order frequency heterodyne to get a time-dependent Fresnel zone plate (TDFZP). Then, point detection and 2D scanning are used to create a hologram. Electron devices are used to create a second complex hologram from the first detected real hologram. This system can also eliminate the twin-image term.

Full-field heterodyne holography, proposed by Le Clerc^[20], uses two AOMs to create a low-frequency heterodyne. It is able to get holograms with a full-field camera other than by point detection. By processing the four-step phase-shifting holograms, the twin-image term can be eliminated as well. Off-axis heterodyne holography was proposed by Gross to achieve an ultimate sensitivity^[21]. Furthermore, the noise and signal scaling factors in heterodyne holography were discussed^[22].

In these DHM systems, the phase shifting is 4 steps in order to suppress the influence of the twin-image term. Sometimes multiple periods of holograms are recorded for time averaging to suppress the influence of random noise. In the case of a high precision phase-shifting interferometer, multiple steps of phase-shifting interference patterns are detected and processed with corresponding algorithms to reduce the nonlinear error of a PZT and random noise^[16,23,24]. Even a 101-frame algorithm for phase-shifting interferometry was proposed^[25].

To combine the advantages of multiperiod and multistep phase-shifting technology, a multiframe full-field heterodyne DHM is proposed. Low-frequency heterodyne AOMs, a high-speed camera, and a multiframe temporal signal processing algorithm are applied to this DHM system.

The DHM system is an on-axis holographic system based on a Mach–Zehnder configuration, as shown in Fig. 1. Two AOMs of MHz-level modulating frequency are applied to modulate the object beam and reference beam, and the modulating angular frequencies are $ω_{O}$ and $ω_{R}$ . In the object channel, the beam illuminates the object and is collected by a microscope objective (MO). In the reference channel, a second MO is used to compensate the phase curve in the object channel. The object beam and the reference beam are combined by a beam splitter (BS) and generate low-frequency heterodyne interference holograms. Then the multiframe full-field holograms could be recorded by high-speed camera.

Figure 1.Schematic diagram of a multiframe full-field heterodyne DHM. HWP: half-wave plate, PBS: polarizing beam splitter, RM: reflective mirror, and BE: beam expander.

Download full size

View all figures

In previous work, we developed a high-quality Hz-order heterodyne frequency signal generator, and the high-speed camera is commercially available^[26]. Compared with other high-speed holography, the phase shifting of the proposed heterodyne method is more linear and precise than the PZT method^[27], and can fully use the pixels of the camera without spatial segmentation^[28]. What is more, phase-locked technology was used in the AOM’s signal generator, which can ensure the phase synchronism between two channels. This is more easily achieved^[29].

The object wave $U_{O}$ of this system is $U_{O} (t) = a_{O} e^{j (ω_{O} t + φ_{O})} .$ (1)

The reference wave $U_{R}$ of this system is $U_{R} (t) = a_{R} e^{j (ω_{R} t + φ_{R})} .$ (2)

The ideal hologram $s$ without noise is $s (t) = (U_{O} + U_{R}) {(U_{O} + U_{R})}^{*} .$ (3)

The relationship between the heterodyne angular frequency $ω_{H}$ and the heterodyne frequency of the AOMs $f_{H}$ is $ω_{H} = ω_{O} - ω_{R} = 2 π f_{H} .$ (4)

The expansion of Eq. (3) is $s (t) = a_{O}^{2} + a_{R}^{2} + a_{O} a_{R} e^{j (ω_{H} t + φ_{O} - φ_{R})} + a_{O} a_{R} e^{- j (ω_{H} t + φ_{O} - φ_{R})} .$ (5)

The phase and amplitude of the object is recorded in the third item with modulating the angular frequency of $ω_{H}$ . The twin-image and direct current (DC) terms are recorded in other temporal frequencies.

In addition, the signal $s_{total}$ with random noise $φ_{Noise}$ is $s_{total} (t) = a_{o}^{2} + a_{R}^{2} + a_{o} a_{R} e^{j (ω_{H} t)} e^{j (φ_{o} - φ_{R})} e^{j φ_{Noise} (t)} + a_{o} a_{R} e^{- j (ω_{H} t)} e^{- j (φ_{o} - φ_{R})} e^{- j φ_{Noise} (t)} .$ (6)

The relationship between $f_{H}$ , recording time $t$ , and recording period $N_{period}$ is shown as $N_{period} = f_{H} t .$ (7)

The relationship between the frame rate of camera $f_{D}$ , phase-shifting steps $N_{step}$ , and $f_{H}$ is $N_{step} = \frac{f_{D}}{f_{H}} .$ (8)

The relationship between the recording hologram numbers $N_{total}$ , $N_{period}$ , and $N_{step}$ is $N_{total} = f_{D} t = N_{period} N_{step} .$ (9)

For example, with a 200 fps recording speed and a 10 Hz heterodyne frequency interference, 200 holograms of 10 periods and 20 steps can be gotten in 1 s. These holograms format a 3D matrix. A temporal modulated signal can be gotten from each pixel of the camera.

Regarding the object, the twin-image and the DC terms have different modulating frequencies in Eq. (5), and they can be easily separated in the temporal frequency spectrum. Thus, a temporal frequency spectrum analysis algorithm, such as the fast Fourier transform (FFT) algorithm, is selected to process the signal of each pixel. According to the Nyquist–Shannon sampling theorem, when $f_{D}$ is twice $f_{H}$ , the object information can be extracted from the third term of Eq. (5) at the corresponding frequency spectrum point, which is $O (x, y) = a_{O} a_{R} e^{j (φ_{O} - φ_{R})} .$ (10)

For the signal with random noise, the calculated value at frequency $ω_{H}$ is $S (ω_{H}) = \frac{1}{T} \int_{0}^{T} s_{total} (t) e^{- j (ω_{H} t)} d t .$ (11)

The detail of Eq. (11) is $S (ω_{H}) = a_{O} a_{R} e^{j (φ_{O} - φ_{R})} \frac{1}{T} \int_{0}^{T} e^{j φ_{Noise} (t)} d t .$ (12)

Part of the value is the mathematical expectation of $e^{j φ_{Noise}}$ , which is $\frac{1}{T} \int_{0}^{T} e^{j φ_{Noise} (t)} d t = E (e^{j φ_{Noise}}) .$ (13)

The random noise is very complex, which includes vibration, turbulence, etc. If the statistical distribution of this noise is a normal distribution^[3,30,31] and the expectation $μ = 0$ , the probability density function $p$ is $p (φ_{Noise}) = \frac{1}{\sqrt{2 π} σ} e^{- \frac{φ_{Noise}^{2}}{2 σ^{2}}},$ (14)where $σ$ is the standard deviation.

The characteristic functions $F$ are $F (n) = \int_{- \infty}^{+ \infty} e^{j φ_{Noise} n} p (φ_{Noise}) d φ_{Noise} = e^{- \frac{n^{2} σ^{2}}{2}} .$ (15)

The mathematical expectation of $e^{j φ_{Noise}}$ equals $F (1)$ is $E (e^{j φ_{Noise}}) = F (1) = e^{- \frac{σ^{2}}{2}} .$ (16)

Then the result of Eq. (12) is $S (ω_{H}) = e^{- \frac{σ^{2}}{2}} a_{O} a_{R} e^{j (φ_{O} - φ_{R})} .$ (17)

In the distinct form of Eq. (12), as the frame number $N_{total}$ increases, the result can more approach $S (ω_{H})$ in Eq. (17).

The procedure of extracting the amplitude of the demodulated complex value is $e^{- \frac{σ^{2}}{2}} a_{O} a_{R} = | S (ω_{H}) | = \sqrt{{Re [S (ω_{H})]}^{2} + {Im [S (ω_{H})]}^{2}} .$ (18)

In Eq. (18), Re represents the real part and Im represents the imaginary part. As $a_{R} e^{- \frac{σ^{2}}{2}}$ is a constant, the normalization of $a_{O}$ can be gotten by normalizing.

The procedure of extracting the phase of the demodulated complex value is $φ_{O} - φ_{R} = \tan^{- 1} \frac{Im [S (ω_{H})]}{Re [S (ω_{H})]} .$ (19)

The phase unwrapping process is done, with the result in Eq. (19), which is common procedure in theDHM algorithm^[2,3]. As a second MO is used to compensate the phase curve in the object channel, the unwrapped phase represents the phase of the object.

The traditional four-step phase-shifting algorithm is $\emptyset = \tan^{- 1} (\frac{I_{3} - I_{1}}{I_{0} - I_{2}}),$ (20)where $I_{0}$ , $I_{1}$ , $I_{2}$ , and $I_{3}$ are 4 holograms with $π / 2$ phase shifting, and $\emptyset$ is the phase of the object wave. This method can also be explained as using 4 discrete items to get a Fourier coefficient, which is in essence similar to the proposed multiframe method.

The main factors in the proposed method are $N_{period}$ , $N_{step}$ , and $N_{total}$ . $N_{total}$ affects the computation accuracy, which is shown in Eqs. (13) and (16). Thus, a long recording time $t$ and a high frame rate $f_{D}$ are needed. $N_{step}$ affects the bandwidth of the FFT spectrum. As the $N_{step}$ increases, the modulated signal can flexibly keep away from different noise frequencies in different environments. What is more, a larger $N_{step}$ leans to a larger $N_{total}$ and better precision under the same number of periods.

To verify the relationship between $N_{period}$ , $N_{step}$ , $N_{total}$ , and the reconstruction precision, a simulation is done and is shown in Fig. 2. $N_{period}$ varies from 1 to 20, and $N_{step}$ values of 4, 10, and 100 are selected for illustration. $N_{total}$ varies from 4 to 2000. The $σ$ of random noise is set to 0.1. Each point of the results is a statistic of 200 times repetition. The root-mean-square error (RMSE) is defined as $RMSE = \sqrt{\frac{\sum_{i = 1}^{N} {(φ_{i} - φ_{s})}^{2}}{N}},$ (21)

Figure 2.Reconstruction error under different periods and steps.

Download full size

View all figures

where $N$ is the repeat number, $φ_{i}$ is the reconstruction data, and $φ_{s}$ is the ideal data.

As the number of periods and steps increased, the reconstruction error decreased. Frame numbers should be as large as possible theoretically. However, the number is limited by the recording time, array size, signal processing speed, etc. Thus, parameters of this system should be set for different needs.

In addition, to verify the relationship between the modulation frequency and the reconstruction precision, a simulation is shown in Fig. 3. The detection speed is 100 fps, the $σ$ of the random noise is set to 0.1, and the heterodyne frequency varies from 9.9 to 10.1 Hz.

Figure 3.Reconstruction phase error under different modulation frequencies.

Download full size

View all figures

As shown in Fig. 3, the modulation frequency obviously affects the reconstruction precision, which makes a strict requirement for the driver of AOMs.

Moreover, in the practical recording of the holograms, there are other kinds of nonrandom noise that cannot be eliminated just by the multiframe method. These kinds of noise also exist in traditional DHM systems and have been well discussed^[3,16]. Here, we give a brief discussion of these additional effects. The system error should be detected though measuring a standard object, and subtracting it in each result; the discretization error can be improved by increasing the bit number of the camera, which is inversely proportional to the detection speed and should be selected carefully; the shot noise of the camera limits the SNR of the recorded hologram, which can be improved by increasing the light intensity, but should also consider the tolerance of the living cell.

A demonstrating experiment was set up. The main devices are as follows: laser (Coherent Corporation Verdi 6, 532 nm), MO (Daheng Optics Corporation, $10 \times$ , $NA = 0.25$ ), high-speed camera (Baumer Corporation, hxc13, $1280 \times 1024$ pixels, 500 fps), microlenses (Advanced Micro-Optic Systems GmbH, diameter 90 μm), biology cell (Institute of Genetics and Developmental Biology, Chinese Academy of Sciences, mouse cell).

The driver of low-differential AOMs is developed by our group, which proves a precise heterodyne signal. The design beat frequency is 10 Hz and the actual measurement is $9.963 \pm 0.003 Hz$ , the phase noise is $- 117 dBc$ , and the harmonic noise is $- 73 dB$ .

First we do the experiment with microlenses. $f_{D}$ is 100 fps and 100 frame holograms were gotten in 1 s. Ten-step holograms in 1 period are shown in Fig. 4.

Figure 4.Ten-step phase-shifting holograms of 1 period.

Download full size

View all figures

For each pixel of the hologram, a temporal signal of 100 samples was gotten. The signal of one pixel is shown in Fig. 5(a), in which the phase shifting is linear and precise. Then process signal is processed with an FFT to obtain the frequency spectrum, which is shown in Fig. 5(b).

Figure 5.(a) Sample data of one pixel of microlenses and (b) the frequency spectrum of the sample data.

Download full size

View all figures

The reconstructed amplitude and phase are shown in Figs. 6(a) and 6(b). The unwrapped phase is shown in Fig. 6(c), and a 3D view is shown in Fig. 6(d). The result indicates the feasibility of the proposed technology.

Figure 6.(a) Amplitude of microlenses, (b) the phase of microlenses, (c) the unwrapped phase of microlenses, and (d) a 3D view of the reconstructed microlenses.

Download full size

View all figures

Then we change the experiment system to the inverted configuration and deal with a non-dyeing mouse cell. One hundred holograms of the cell were recorded. One hologram is shown in Fig. 7(a), the frequency spectrum of one pixel is shown in Fig. 7(b), and a 3D view of the reconstruction cell is shown in Fig. 7(c).

Figure 7.(a) One hologram of the mouse cell, (b) the frequency spectrum of one pixel, and (c) a 3D view of the reconstruction cell.

Download full size

View all figures

As shown in Fig. 7(a), the SNR of the nondyeing cell is not very high. However, with multiframe holograms and the temporal frequency spectrum analysis algorithm the reconstructed object is clear, as shown in Fig. 7(c).

In conclusion, we propose a multiframe full-field heterodyne DHM. Using a low-frequency heterodyne signal and a high-speed camera, multiframe holograms can be gotten. By extracting the information of each pixel from a temporal frequency spectrum, the influence of a twin image can be effectively eliminated. As the frame numbers increase, the statistical value of the noise in the heterodyne frequency approaches a constant. As a result, this method can reduce the influence of twin-image and random noise, which makes it adaptable for different precision requirements.

We demonstrate the efficiency of this technology in the theory, the simulations, and the experiment. The phase shifting is linear and accurate and the reconstructed images of the microlenses and the cell are clear and sharp.

This system is able to perform a quantitative 3D imaging without dyeing or 2D scanning of the sample, which can be widely used in MEMS and biology research areas.

Category: Holography

Received: Nov. 6, 2015

Accepted: Mar. 4, 2016

Published Online: Aug. 6, 2018

The Author Email: Bin Xiangli (xiangli@aoe.ac.cn), Wenxi Zhang (zhangwenxi@aoe.ac.cn)

DOI:10.3788/COL201614.050901