Advanced Photonics, Volume. 7, Issue 2, 026001(2025)

Single-shot volumetric fluorescence imaging with neural fields

Oumeng Zhang1、†,*, Haowen Zhou1, Brandon Y. Feng2, Elin M. Larsson3, Reinaldo E. Alcalde3, Siyuan Yin4, Catherine Deng1, and Changhuei Yang1,4
Author Affiliations
  • 1California Institute of Technology, Department of Electrical Engineering, Pasadena, California, United States
  • 2Massachusetts Institute of Technology, Computer Science and Artificial Intelligence Laboratory, Cambridge, Massachusetts, United States
  • 3California Institute of Technology, Division of Biology and Biological Engineering, Pasadena, California, United States
  • 4California Institute of Technology, Department of Medical Engineering, Pasadena, California, United States
  • show less
    Figures & Tables(7)
    (a) Schematic of single-shot volumetric fluorescence imaging using the QuadraPol PSF. OL, objective lens; TL, tube lens; QWP, quarter-wave plate; DM, dichroic mirror; BF, bandpass filter. A four-polarization custom polarizer (4-Pol) is positioned at the BFP of the imaging system to modulate the emission light, with a polarization camera (PolCam) capturing the modulated fluorescence. The transmission axes of the polarizer and PolCam are 0, 45, 90, and 135 deg. (b) Assembling the custom polarizer by aligning two coverslips and four laser-cut polymer polarizers among 3D-printed holders. (c) Representative image of a point source captured by the polarization camera, visualized using (i) raw pixel readouts, (ii) a polarization image color-coded in the hue-saturation-value (HSV) scheme (AoLP as hue, DoLP as saturation, and intensity as value), and (iii) four separate images for each polarization channel. Scale bar: 50 μm.
    Amplitude and phase of the pupil and PSFs at different heights. (a) Theoretical PSFs without aberration, (b) simulated PSFs using the retrieved phase, and (c) experimental PSFs. Question marks indicate that the phase and amplitude for the experimental PSF are not accessible. Scale bar: 2 mm for the pupil images and 0.2 mm for the PSF images.
    Framework of using neural fields to extend the quality and depth range of the imaging system. (a) The RL-deconvolved image volume guides the initialization of the model with a compact learnable feature space and MLP. After model initialization, the model is further optimized for the image volume. The estimated image volume goes through the forward model of the imaging system to generate the estimated measurements. These measurements are compared with the acquired measurements and then to update the model weights and parameters. (b) Once the model is optimized, the parameters and weights are fixed. It can render an image stack with continuous sampling. The operator ⊗ denotes a convolution operation, and ⊕ indicates a summation operation along the z axis.
    Performance evaluation of the QuadraPol PSF using simulated data. (a) Lateral and (b) axial resolutions determined by the Rayleigh criterion as functions of axial position and signal level. Images show representative data with Poisson shot noise and reconstruction cross sections using RL deconvolution and neural fields. Shaded areas represent the diffraction (λ/2NA) and sampling limits. Scale bar: 20 μm in xy view and 200 μm in xz view. (c)–(e) Reconstruction results for various line structures; insets show simulated images. Parameters for simulating the QuadraPol PSF (NA=0.52, magnification M=−5.2, and camera pixel size 2.2 μm) are adjusted to match those reported for Miniscope3D.13" target="_self" style="display: inline;">13 Scale bar: 100 μm; color bar: height in μm. (f) and (g) Object and images using the polarized spiral (PS2F) and QuadraPol PSFs with (f) vertical and (g) horizontal lines at z=±0.35 mm. Scale bar: 50 μm; color bar: AoLP. Insets show the PSFs. (h) Comparison between the QuadraPol PSF and other SVF techniques.
    Imaging fluorescent beads on a 45-deg tilt coverslip using the QuadraPol PSF. (a) Raw image of the fluorescent beads. Scale bar: 1 mm. (b) 3D rendering of the reconstructed beads using MATLAB function “isosurface.” Grid size: 1 mm. (c) FWHM values for the reconstructed beads. Lines represent the average; shaded areas represent the standard deviation.
    All-in-focus imaging of E. coli tagged with mScarlet-I on sand surfaces. (a) Raw polarized fluorescence image. The inset shows a photograph of the sample captured using a smartphone camera. (b) Height map recovered using the all-in-focus algorithm. Color bar: height in mm. (c) All-in-focus fluorescence image. (d) Zoomed regions of interest in panels (a)–(c). Annotations represent the z range for each region. Scale bar: 20 mm in the inset of panel (a), 2 mm in panels (b) and (c), and 0.5 mm in panel (d).
    Volumetric imaging of wheat roots using the QuadraPol PSF. (a) Reconstruction using neural fields. The inset shows a photograph captured using a smartphone camera. (b) Raw polarized fluorescence image. (c) xy view; (d) and (e) xy and xz views of the zoomed regions in panel (a), reconstructed using (i) deconvolution with experimental PSF, (ii) deconvolution with retrieved PSF, and (iii) neural fields. Scale bar: 5 mm in panels (a) and (b), 0.5 mm in the zoomed region of panel (b), and 1 mm in panels (c)–(e). Color bar: depth in mm. Note that the color in the xy view is composed of the entire reconstructed volume, reflecting the intensity contribution from all z slices; this view predominantly displays a color corresponding to the central height of the root.
    Tools

    Get Citation

    Copy Citation Text

    Oumeng Zhang, Haowen Zhou, Brandon Y. Feng, Elin M. Larsson, Reinaldo E. Alcalde, Siyuan Yin, Catherine Deng, Changhuei Yang, "Single-shot volumetric fluorescence imaging with neural fields," Adv. Photon. 7, 026001 (2025)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Articles

    Received: Aug. 23, 2024

    Accepted: Jan. 24, 2025

    Posted: Jan. 24, 2025

    Published Online: Feb. 26, 2025

    The Author Email: Zhang Oumeng (ozhang@caltech.edu)

    DOI:10.1117/1.AP.7.2.026001

    CSTR:32187.14.1.AP.7.2.026001

    Topics