Advanced Photonics, Volume. 6, Issue 2, 026004(2024)

Tensorial tomographic Fourier ptychography with applications to muscle tissue imaging

Shiqi Xu1, Xi Yang1, Paul Ritter2, Xiang Dai1,3, Kyung Chul Lee1,4, Lucas Kreiss1,2, Kevin C. Zhou1,5, Kanghyun Kim1, Amey Chaware1, Jadee Neff6, Carolyn Glass6, Seung Ah Lee4, Oliver Friedrich2, and Roarke Horstmeyer1、*
Author Affiliations
  • 1Duke University, Durham, North Carolina, United States
  • 2Friedrich-Alexander University, Erlangen, Germany
  • 3UC San Diego, La Jolla, California, United States
  • 4Yonsei University, Seoul, Republic of Korea
  • 5UC Berkeley, Berkeley, California, United States
  • 6Duke University Medical Center, Durham, North Carolina, United States
  • show less

    We report tensorial tomographic Fourier ptychography (T2oFu), a nonscanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier ptychography, a quantitative phase imaging technique, T2oFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equipped with an LED matrix, a polarization generator, and a polarization-sensitive camera. Permittivity tensors of anisotropic samples are computationally recovered from polarized intensity measurements across three dimensions. We demonstrate T2oFu’s efficiency through volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, as well as tissue samples from muscle fibers and diseased heart tissue. Our reconstructions of healthy muscle fibers reveal their 3D fine-filament structures with consistent orientations. Additionally, we demonstrate reconstructions of a heart tissue sample that carries important polarization information for detecting cardiac amyloidosis.

    Keywords

    1 Introduction

    Quantitative phase imaging (QPI) is a well-known label-free microscopy approach that can detect phase delay introduced by semitransparent cells and tissue.1 Due in part to its ability to provide quantitative information about primarily transparent biological specimens with low phototoxicity, QPI has become an invaluable tool in scientific and clinical studies, including for monitoring neuronal firing2 and cancer cell line detection,3 to name a few. Besides scalar phase contrast, transparent specimens also exhibit alternative and important endogenous optical contrast mechanisms, including anisotropic properties such as material birefringence and orientation. Indeed, the orientation of molecular arrangements in lipid membranes can now be quantitatively monitored with polarization-sensitive microscopes for studying multi-organelle interactive activities.4,5 There are naturally a variety of polarization-sensitive microscope arrangements, including early analog designs,6,7 differential interference contrast (DIC) methods,8 and contemporary digital approaches that reconstruct quantitative specimen retardance and orientation.911 These methods have been applied to study small model organisms12,13 and to assist clinical diagnosis.1417

    Recently, there has been increased interest in measuring polarization-sensitive phase information from specimens. In general, polarization-sensitive quantitative phase imaging (PS-QPI) methods can be divided into two categories: those that rely on interferometric detection, and those that utilize computational phase-retrieval methods. Interferometric methods (e.g., off-axis holography) can create polarization-sensitive phase images with as few as a single measurement,1821 and can be extended to 3D with diffraction tomography approaches.2225 While often impressive, these methods usually require complex arrangements of coherent laser illumination and careful system alignment, which sets additional design requirements for use in clinical applications. Computational phase-retrieval methods, on the other hand, rely on image reconstruction algorithms that convert multiple intensity measurements into phase-sensitive outputs2632 and can be implemented with less expensive hardware. Due to their simple instrumentation, computational polarization microscopes have been increasingly applied to image biological samples, for instance, to study white matter tracts within whole brain slices,29 to diagnose malaria from blood smears,27 and to identify calcium phosphate crystal from urine samples.32 Furthermore, these methods have been extended to image 3D samples, such as axons and cardiac tissue with axial scanning.33,34 Thick samples such as organoids and tissue slices also naturally have intriguing 3D structures that can include anisotropic material. Creating high-resolution volumetric representations of the polarization properties of these samples is essential to studying biology and pathology. While tomographic imaging methods such as confocal-based approaches have been developed in the past to image muscle tissue and neural organoids for instance,35,36 there remain relatively few microscopic techniques to jointly capture quantitative phase and anisotropy across a large 3D volume at high resolution. One very recent study attempted to create 3D anisotropy maps using off-axis LED illumination but did not provide tomographic permittivity matrix reconstructions.37

    Here, we propose a non-scanning polarization-sensitive tomography method, termed tensorial tomographic Fourier ptychography (T2oFu), to create quantitative volumetric permittivity matrix reconstructions without any moving parts. Our method is an extension of recently developed intensity optical diffraction tomography principles3844 and has the potential to be extended to a video-rate system in the future.38 We image a variety of calibration targets, as well as 3D tissue and muscle fiber samples, through which we demonstrate the ability to resolve the fine-filament structure that resembles the results demonstrated in the literature.35 Additionally, we demonstrate reconstructions of a heart tissue sample that carries important information for detecting cardiac amyloidosis.

    2 Methods

    2.1 Experimental Setup

    The imaging setup of T2oFu is illustrated in Fig. 1(a). The illumination consists of an array of 25×25 addressable LEDs (μInventions Inc.; Portugal) and a left-circular polarizer (CP42HE; Edmund Co., USA). About 521 nm wavelength light from the LED array is circularly polarized and illuminates the 3D sample from different angles. A microcontroller (ARM Cortex-M3) and a voltage level shifter (SN74AHCT) are used to turn on each small LED pixel (WS2812b-2020) sequentially. The optical field then passes through an optical system consisting of an infinity-corrected objective (0.25 NA or 0.4 NA, Plan Achromatic; Olympus, Japan) and a tube lens (Achromatic Doublets, 180 mm focal length; Thorlabs, Inc., USA). The largest illumination NA (NAillum) is chosen to match the native NA of the objective (NAobj). The intensity image is captured with a polarization-sensitive CMOS camera (BFS-U3-51S5P; Teledyne FLIR LLC, Oregon). The polarization camera analyzes the light orientated at four different polarizations (0 deg, 45 deg, 90 deg, and 135 deg), achieved by placing 2×2 wire-grid polarizers between the pixel array and the microlens array, as illustrated in the m-dashed box in Fig. 1(a). Exemplary intensity images under illumination from different angles are shown in the box in Fig. 1(b).

    Workflow of the proposed method. (a) An illustration of T2oFu’s experimental setup. Green light from an addressable LED array is circularly polarized with a left circular polarizer to illuminate the sample. Sequential illuminations from various angles are used to scan the object in the spatial frequency domain. The sample is then imaged with an infinity-corrected optical system. The polarized light intensities at 0 deg, 45 deg, 90 deg, and 135 deg are recorded with a polarization-sensitive CMOS camera as images illustrated in (b). (c) Those intensity measurements are then fused to form volumetric sample permittivity tensor reconstructions by solving the corresponding inverse problem.

    Figure 1.Workflow of the proposed method. (a) An illustration of T2oFu’s experimental setup. Green light from an addressable LED array is circularly polarized with a left circular polarizer to illuminate the sample. Sequential illuminations from various angles are used to scan the object in the spatial frequency domain. The sample is then imaged with an infinity-corrected optical system. The polarized light intensities at 0 deg, 45 deg, 90 deg, and 135 deg are recorded with a polarization-sensitive CMOS camera as images illustrated in (b). (c) Those intensity measurements are then fused to form volumetric sample permittivity tensor reconstructions by solving the corresponding inverse problem.

    2.2 Principles of Tensorial Tomographic Fourier Ptychography

    2.2.1 Notation

    Here we introduce the notation we use in the rest of this article. First, we use · and ·¯¯ symbols to denote vectors and matrices for variables, respectively. By default, all the vectors in this article are column vectors. Second, we utilize the Mathematical Script font enclosed in curly braces to denote spatial operators. As an example, one frequently used operator is F{·}, which indicates the Fourier transform in space. Next, bold letters in lowercase represent support vectors in either frequency (u) or space (r). Finally, we utilize the ·˜ symbol to denote the frequency-domain counterparts of variables previously defined in the space domain, such as I˜(u)=F{I(r)}.

    2.2.2 Vectorial light propagation

    The optical properties of a 3D sample can be described by its 3×3 permittivity matrix,45ϵ¯¯=[ϵxx(r)ϵxy(r)ϵxz(r)ϵyx(r)ϵyy(r)ϵyz(r)ϵzx(r)ϵzy(r)ϵzz(r)],where r=(x,y,z) is the voxel position in space. Within the scope of this work, we adopt the first Born approximation,23,33 and the scattered vectorial electric field Es and illumination E0 are related as Es(r)=G¯¯(rr)V¯¯(r)E0(r)dr,where V¯¯(r)=ϵ¯¯ϵ¯¯0 is the sample scattering potential tensor with ϵ¯¯0 as the permittivity tensor of the background medium. G¯¯(r) is the dyadic Green’s tensor.46 In addition, we use illuminations with relatively small incident angles, which have weak polarization along the z axis. Therefore, we mainly consider the transverse polarization of the electrical field, E=(Ex,Ey)T, which will be discussed in more detail in the next subsection. The intensities of the vectorial electric field E at different orientations are then analyzed and recorded by a polarization-sensitive optical imaging system, which we model as a 2×2 pupil matrix at each spatial frequency u.31 Therefore, the measured intensity under plane-wave illumination with angle u analyzed by a polarizer with Jones vector a is Il(r,u)=|alTF2d1{P¯¯(u)F2d{Es(r,u)+E0(r,u)}}|2.

    For a linear polarization analyzer oriented at α, a=[cosα,sinα]T.31,47 The intuition behind this is that light can only oscillate in the same direction as the linear polarizer.31

    2.2.3 Forward model and inverse problem

    For our initial demonstration of T2oFu, we modify the model discussed in Sec. 2.2.2 by making certain approximations. The objective of this is to craft a meaningful forward model that accurately describes our experimental measurements, whose inverse problem is less ill-posed. First, following Saba et al. and other closely relevant works in the literature,23,26,27,31 we begin with a paraxial approximation, which assumes weak polarization along the optical axis of the illumination and negligible interaction between traverse and axial polarization from the sample. With this, we can simplify the 3×3 permittivity matrix to a 2×2 matrix, denoted as ϵ¯¯=[ϵxx(r)ϵxy(r)ϵyx(r)ϵyy(r)].

    While this approximation may not always be accurate for every anisotropic sample when illuminated at high angles, it is accurate under up to a 25 deg oblique illumination, based on a study using finite-element analysis.23 Additionally, we assume that the background medium is isotropic and uniform (i.e., not spatially dependent) with a diagonal permittivity tensor ϵ¯¯0=ϵ0I. This simplifies Green’s tensor to a diagonal matrix with the same component for each polarization,23G¯¯(r,r)=[G(r,r)00G(r,r)].

    G(r,r)=G(r), where r=|rr|, is the scalar Green’s function that has a Weyl expansion,45,48ejk0rr=1j2πduej(u·x+η|z|)η,with wavenumber vectors in lateral [u=(kx,ky)] and axial (η=k02|u|2) directions. k0 is the wavenumber of the isotropic background medium. This expansion is generally easier to work with when we prefer to represent the object in both lateral frequency and axial space domains.

    In addition, we make the assumption that the sample being imaged is homogeneous, which is usually assumed for many types of crystals and biological samples.10,49 The permittivity matrix is then symmetric and can be decomposed into49ϵ¯¯=[cosθsinθsinθcosθ][ϵe00ϵo][cosθsinθsinθcosθ].

    The variables ϵo and ϵe represent the permittivity values along the ordinary and extraordinary axes, respectively. The parameter θ is the angle between the principal axis and the extraordinary axis (also known as the slow axis). Following the convention introduced by an early study,50 we rename elements in Eq. (4) as {ϵ1=ϵxxϵ2=ϵyyϵ3=ϵxy=ϵyx.

    Since we use left circularly polarized illumination, the transverse scattering potential becomes [V1V3V3V2][1j]=[V1+jV3V3+jV2],where {V1=4πk02(ϵ1ϵ0)V2=4πk02(ϵ2ϵ0)V3=4πk02ϵ3.

    Finally, we refer to Ling et al.51 for two further approximations of the illumination and scattering processes. Our first assumption is that the illumination from each LED at the sample plane is a plane wave, which is commonly used in Fourier ptychographic computational microscopy.52 The second approximation involves utilizing a weak object assumption that ignores the second-order scattering term.38,53,54 Further, we disregard pupil aberration and model it as a low-pass filter P(u) with a cutoff frequency based on the numerical aperture of the objective lens.38 The Jones matrix has been demonstrated feasible in the previous literature for the correction of anisotropic aberration by jointly reconstructing the pupil,30,31 which is planned for future research. Based on these approximations, the forward model can be expressed as38I˜l,m(u,z=0)I˜0l,m(u,z=0)+[HRem(u,z)·V˜Rel(u,z)+HImm(u,z)·V˜Iml(u,z)]dz,where I˜l,m(u,z) and I˜0l,m(u,z) are 2D Fourier transform of the measurement and DC term from the mth LED illumination analyzed by the lth polarizer, respectively. HRem(u,z)=jk22S(um){P*(um)ej[ηi+η(uum)]zη(uum)P(uum)P(um)ej[ηi+η(u+um)]zη(u+um)P(uum)}and HImm(u,z)=k22S(um){P*(um)ej[ηi+η(uum)]zη(uum)P(uum)+P(um)ej[ηi+η(u+um)]zη(u+um)P(uum)},are the diffractive transfer functions in frequency and space for the real and imaginary part of the scattering potential under the mth LED illumination with shape Sm(u).53,55 Since each LED has a very small die area (<170  μm in diameter), we represent it as a delta function. l{0  deg,45  deg,90  deg,135  deg}. V˜Rel(u,z) and V˜Iml(u,z) are 2D Fourier transforms of VRel(r) and VIml(r) along lateral directions at depth z, respectively, and are related to Eq. (10) via {VRe0  deg(r)+jVIm0  deg(r)=V1(r)+jV3(r)VRe45  deg(r)+jVIm45  deg(r)=22[V1(r)+V3(r)]+j22[V2(r)+V3(r)]VRe90  deg(r)+jVIm90  deg(r)=V3(r)+jV2(r)VRe135  deg(r)+jVIm135  deg(r)=22[V1(r)+V3(r)]+j22[V2(r)V3(r)],which are the scattering potential components corresponding to each analyzer angle. We want to point out that V1,2,3(r) are complex variables that may have imaginary parts; hence, the above equation does not imply V˜Re(r)=V1(r), for example.

    For concise expression, we define a new variable vCN×M×T×3 representing all the potentials V1(r),V2(r), and V3(r), where N,M,T are width, height, and depth of the 3D sample, respectively. Further, we introduce the operator Al,m(·) as the forward model for the sample illuminated by the mth LED and analyzed by the lth linear polarizer. To reconstruct the permittivity matrix, we formulate the inverse problem as v=argminvL(v),with the loss function L(v)=lmAl,m(v)I˜l,m(u,z)22+γtv(v).tv(·) is the isotropic total variation operator. γ is a regularization parameter empirically set to be 1×106 for all experiments. The forward model is implemented in PyTorch, and the loss function is optimized using a stochastic gradient descent method with Nesterov momentum acceleration.56,57

    Subsequently, we extract polarization properties of interest, such as orientation and birefringence from T2oFu reconstructions ϵ1,2,3. First, the orientation to the slow axis can be computed as θ={12arctan2ϵ3/ϵ2ϵ1,if  ϵ1ϵ2>012arctan2ϵ3/ϵ2ϵ1+π2,otherwise.

    We can then derive the permittivity of ordinary and extraordinary axes, {ϵo=ϵ¯1/2Δϵϵe=ϵ¯+1/2Δϵ,where {ϵ¯=ϵ1+ϵ22Δϵ=|ϵ1ϵ2cos2θ|.

    Moreover, the refractive index along the ordinary and extraordinary axes no,e=ϵo,e can be further derived, along with the averaged refractive index (n=no+ne) and birefringence (Δn=neno).

    3 Experimental Results

    To validate the performance of the proposed method, we first show results from a variety of calibration targets. Due to the lack of a commercially available high-resolution 3D polarization phantom, we follow previous works to validate different aspects separately.27,31,33,58 We then show reconstructions of a single fixed muscle fiber. Finally, we show reconstructions of a tissue section sample from a human heart biopsy that is predictive for lethal cardiac amyloidosis. Additional validation for resolution and reconstruction accuracy based on numerical simulation is also provided in Sec. 1 in the Supplementary Material. All the images presented here are captured and then reconstructed using a 20×, 0.4 NA system, except for the cardiac tissue, which is recorded and processed with a 10×, 0.25 NA system.

    3.1 Tomographic Reconstruction

    In this subsection, we demonstrate tomographic reconstructions of averaged refractive index n¯ and birefringence Δn from isotropic and anisotropic calibration samples. These results are acquired with a 20× objective (0.4 NA, Olympus, Japan) imaging system. Figure 2(a) shows the image captured with center LED illumination when the imaging system is focused at the middle of a polystyrene microsphere sample. The sample is made of two layers of 3  μm-diameter microspheres immersed in n=1.575 oil. Figure 2(b) shows images taken with center LED illumination when focused at different depths. The arrows in Fig. 2(b) highlight the microspheres that are in focus, suggesting two layers of microspheres are placed at 7 and 5  μm planes. Note that since the refractive index of polystyrene (n=1.60 @ 520 nm) is very close to the background medium, the contrast of in-focus microspheres is very low. Figures 2(c) and 2(d) show the reconstructed refractive index and birefringence at two different depths. The accuracy of the tomographic depth reconstruction is validated with images displayed in Fig. 2(b). Since polystyrene is an isotropic material, the reconstruction shows no anisotropy properties except on the edge, which agrees with the well-recognized edge birefringence phenomena59 reported in prior literature.31,33Figure 2(e) shows the cross sections of the reconstructed reflective index and birefringence. Figure 2(f) plots the profile of the reconstructed birefringence and reflective index averaged across 10 microspheres.

    Reconstruction results for polystyrene microspheres. (a) and (b) 0 deg-polarization intensity images of the sample illuminated with center LED. (a) The data are captured when focused at z=0 μm. (b) Images were captured when focused at −5 and 7 μm by mechanically moving the sample. These images serve as a reference to be compared with the reconstruction. (c)–(e) The reconstructed refractive index and birefringence. All columns in (c) and (d) share the same scale bar; all the refractive index reconstructions (c), as well as all birefringence reconstructions (d) and all cross sections (e) share the same color bar. (e) Cross sections of the reconstruction in places color labeled in (a). (f) The profile of reconstruction averaged over 10 microspheres.

    Figure 2.Reconstruction results for polystyrene microspheres. (a) and (b) 0 deg-polarization intensity images of the sample illuminated with center LED. (a) The data are captured when focused at z=0  μm. (b) Images were captured when focused at 5 and 7  μm by mechanically moving the sample. These images serve as a reference to be compared with the reconstruction. (c)–(e) The reconstructed refractive index and birefringence. All columns in (c) and (d) share the same scale bar; all the refractive index reconstructions (c), as well as all birefringence reconstructions (d) and all cross sections (e) share the same color bar. (e) Cross sections of the reconstruction in places color labeled in (a). (f) The profile of reconstruction averaged over 10 microspheres.

    In addition, we show reconstructions of an anisotropic potato starch sample in Fig. 3. The sample consists of two potato starch grains immersed in n=1.515 oil at different depths. Figure 3(a) depicts captured intensity images illuminated with the center LED at four different polarizations. We can see the spiral patterns imaged with different polarization differ from each other noticeably. Similarly, Fig. 3(b) shows images when the system is focused at different depths of the sample. The black arrows point at grains that are in focus, suggesting the two potato starch grains are suspended at 9 and 6  μm. Figure 3(c) shows the reconstructed orientation and birefringence. To best visualize the results, we follow the convention21,27,31 of displaying this multidimensional data using an HSV color map, where saturation is set to one, value is associated with birefringence, and orientation is coded in hue. The reconstructed structures agree with starch grain reconstructions reported in previous holography-based literatures.23,25

    Reconstruction results of potato starch grain. (a) Polarized intensity measurements from the center LED. (b) Images taken when focused at −9 and 6 μm serve as a reference for validating the reconstruction, from which we can see adjacent 2×2 pixels that have different intensities due to sample-introduced polarization changes. (c) Tomographic reconstruction of the sample. The orientation is coded in color, while the birefringence is represented as brightness. All the images in (c) share the same color bar.

    Figure 3.Reconstruction results of potato starch grain. (a) Polarized intensity measurements from the center LED. (b) Images taken when focused at 9 and 6  μm serve as a reference for validating the reconstruction, from which we can see adjacent 2×2  pixels that have different intensities due to sample-introduced polarization changes. (c) Tomographic reconstruction of the sample. The orientation is coded in color, while the birefringence is represented as brightness. All the images in (c) share the same color bar.

    3.2 Orientation Measurement

    To verify the accuracy of the reconstructed orientation, we test our method on a sample made with monosodium urate (MSU). MSU is a needle-shaped crystal precipitated from uric acid that could trigger robust inflammation such as acute arthritis and other immune activations that cause severe pain in patients.60Figure 4(a) displays the reconstructed birefringence and orientation at two slightly different depths. These results were also obtained with a 20× objective imaging system and suggest that the reconstructed orientation values of line-shaped MSU crystals follow the structural direction (labeled next to each MSU crystal). This is in agreement with reconstruction results from LC-PolScope-based methods found in the literature.27,58 Additionally, Fig. 4(b) presents a 3D rendering of the reconstructed birefringence.

    Reconstructions of MSU crystals. (a) The reconstructed polarization orientation of the crystals at two different depths, labeled with their respective structural directions. (b) The birefringence reconstruction of the sample in 3D.

    Figure 4.Reconstructions of MSU crystals. (a) The reconstructed polarization orientation of the crystals at two different depths, labeled with their respective structural directions. (b) The birefringence reconstruction of the sample in 3D.

    3.3 Muscle Fiber Assessment

    High-contrast and high-resolution structural imaging of intrinsic signals in muscle skeletal fibers is an important task for the rapid detection of changes in myofibrillar organization that can lead to skeletal myopathies. Currently, 3D muscle tissue is typically imaged by complex and expensive systems, such as second-harmonic generation (SHG) microscopy. SHG exploits the contrast of polarization properties from the nonlinear susceptibility in molecules like myosin, by using high-power, ultrashort pulsed lasers in a point scanning configuration.35 Here we show 3D T2oFu, reconstruction results of an isolated healthy muscle fiber using our inexpensive, LED-based, and scanning-free system, which generally shows similar results as described for SHG imaging in the literature.35Figure 5(a) shows an image of a muscle fiber captured with center LED illumination. Figure 5(b) shows a volumetric reconstruction of the muscle fiber. The cross sections of regions highlighted in Fig. 5(a) are depicted. Figure 5(c) shows an image of the same muscle fiber from a different field of view, where a non-muscle fiber with a 90 deg bend is placed below the muscle fiber. Figure 5(d) depicts reconstructed orientation and birefringence at two different depths, showing the regular pattern of a healthy muscle fiber. The zoom-in region highlights the muscle grains with consistent orientations, in agreement with the results reported by Both et al.35 The reconstruction shows a change in orientation at the bend of the non-muscle fiber [Fig. 5(d1)], while the orientation of the muscle fiber remains constant. Furthermore, the spatial resolution of the reconstruction is sufficient to clearly resolve the muscle filaments [Fig. 5(d3)]. Figure 5(e) shows a histogram of orientation of the reconstruction shown in Fig. 5(d), indicating two different types of cells.

    Reconstructions of a muscle fiber. (a) The image of a muscle fiber with the center LED illumination. The imaging system is focused in the middle of the muscle fiber. (b) The reconstructed birefringence. The zoom-in region resembles the structures of healthy muscle fibers reported in literature.35" target="_self" style="display: inline;">35 (c) The image of the same muscle fiber at a different region, where a non-muscle fiber with a 90-deg bend is placed below the muscle fiber [see panel (d1)]. The imaging system is focused between this and the muscle fiber. (d) The reconstructed orientation at different depths, with a zoom-in showing the fine sarcomere structure of muscle tissue. (e) Histogram of reconstructed orientation shown in (d).

    Figure 5.Reconstructions of a muscle fiber. (a) The image of a muscle fiber with the center LED illumination. The imaging system is focused in the middle of the muscle fiber. (b) The reconstructed birefringence. The zoom-in region resembles the structures of healthy muscle fibers reported in literature.35 (c) The image of the same muscle fiber at a different region, where a non-muscle fiber with a 90-deg bend is placed below the muscle fiber [see panel (d1)]. The imaging system is focused between this and the muscle fiber. (d) The reconstructed orientation at different depths, with a zoom-in showing the fine sarcomere structure of muscle tissue. (e) Histogram of reconstructed orientation shown in (d).

    3.4 Imaging Cardiac Amyloidosis

    Finally, we apply our method to image a heart tissue sample that has cardiac amyloidosis. Cardiac amyloidosis is a lethal disease that affects <12,000 patients in the US alone, with a <5% 10-year survival rate.61 In current practice, the biopsied tissue is first frozen and thinly sliced, then stained with a congo red-colored dye, and inspected under a cross-polarized microscope. Figure 6(a) shows the bright-field image. Figure 6(b) shows the same region, imaged with a cross-polarized color microscope. The vibrant apple green color suggests mostly likely amyloid protein has built up inside the tissue sample. Figures 6(c) and 6(d) depict the reconstructed refractive index and birefringence, while zoom-ins of the boxed region are shown in Figs. 6(e)6(h). Figures 6(e) and 6(g) show lateral slices at different depths, while Figs. 6(f) and 6(h) show cross sections of the region highlighted with dashed lines in Figs. 6(c) and 6(d). Since the sample is thinly sliced, we do not observe noticeable structural changes in different layers. However, the structure of the birefringence reconstruction is correlated with the color-stained cross-polarized image, which could potentially be useful for rapid on-site inspections in the future.

    Images of a heart tissue sample with cardiac amyloidosis. (a) A brightfield image. (b) A cross-polarized image was taken with a color microscope. (c) and (d) Reconstructed refractive index and birefringence, along with zoom-ins of the boxed region depicted in (e)–(h). (e) and (g) Lateral slices at different depths; (f) and (h) cross sections of the region highlighted with dashed lines in (c) and (d). Panels (c), (e) and (f) share the same color bar, while (d), (g), and (h) share another common color bar.

    Figure 6.Images of a heart tissue sample with cardiac amyloidosis. (a) A brightfield image. (b) A cross-polarized image was taken with a color microscope. (c) and (d) Reconstructed refractive index and birefringence, along with zoom-ins of the boxed region depicted in (e)–(h). (e) and (g) Lateral slices at different depths; (f) and (h) cross sections of the region highlighted with dashed lines in (c) and (d). Panels (c), (e) and (f) share the same color bar, while (d), (g), and (h) share another common color bar.

    4 Discussion and Conclusion

    In this article, we introduce T2oFu, a new non-scanning microscopy method that reconstructs volumetric permittivity metrics of samples based on computational illumination strategies to record polarized measurements and retrieve phase information. Using relatively low-NA objectives, we demonstrate that T2oFu can provide polarization-sensitive tomographic reconstruction for various calibration samples and biological specimens that are potentially useful for future scientific and clinical studies. This particular experimental demonstration relies on the paraxial approximation, which holds up fairly well for illumination angles <25  deg.23,31 However, previous studies have already demonstrated that even higher illumination angles still enable accurate reconstruction of the lateral parts of the permittivity matrix.23,24 Another potential limitation could be the effect of scattering on the reconstruction. Although the theoretical derivation of our method employs a weak object assumption and does not account for multiple scattering, in practice, we find that our method is capable of reconstructing thick biological tissue samples very well. We tested the approach on thin, stained tissue sections, like the cardiac tissue and on thicker, unstained muscles samples of >20  μm thickness. In the future, it might be promising to test our approach on optically cleared samples, as clearing is known to reduce scattering to enable imaging at greater tissue depths and has already been reported for imaging entire muscles using SHG imaging.62 Furthermore, we aim to carry out a direct one-to-one comparison between the established SHG imaging and our new T2oFu technique for the same muscle samples in the near future.

    To ensure successful clinical translation, there are a few improvements that can be made in the next step. First, to increase the frame rate of T2oFu, high flash-rate LEDs need to be deployed.63 Novel sensitive camera sensors64,65 will also play important roles in maintaining a good signal-to-noise ratio. In terms of the reconstruction algorithm, as a first demonstration, we use a gradient-based method implemented with auto-differentiation to optimize the loss function in Eq. (16).56 As the data fidelity term has a closed-form solution, variable-splitting methods can be very effective and allow incorporating advanced regularization that does not have implicit forms.42,66 In addition, data-driven reconstruction approaches can also be deployed to further accelerate the image reconstruction.67,68 Finally, concerning modeling, to relieve the ill-posedness of the problem, we have approximated the permittivity tensor with its lateral components in this work. Albeit widely used,23,26,27,31 this simplification disregards out-of-plane anisotropy. Recent works have shown under uniaxial approximation, tomography of 3D polarization orientation can be retrieved.33 Further investigation in this direction to extend the current method to extract valuable out-of-plane information is planned.33,69

    Shiqi Xu obtained his PhD from Duke University. During this time, he developed computational algorithms for novel sensor arrays to image objects with gigapixel resolution. He also holds an MS degree from Washington University, where he worked on compressive imaging and non-invasive smart LiDAR systems for cancer diagnosis.

    Xi Yang is a PhD student in the BME Department at Duke University. She received her BS degree in physics from Nankai University. She has worked on self-accelerating beam and two-photon microscopy projects during her undergraduate research. Now, she is helping to develop the new generation of Fourier ptychography microscopy in Dr. Horstmeyer’s Group, after completing her master’s degree in the BME Department at Duke University.

    Paul Ritter is a group leader at the Institute of Medical Biotechnology, Friedrich-Alexander-Universität, passionate about medical biotechnology and bioreactor technologies. He has developed a novel bioreactor system that provides automated and standardized generation of cellular muscle scaffolds and is capable of monitoring and adjusting the progress on-the-fly.

    Xiang Dai is a PhD student at University of California San Diego (UCSD), working on computational imaging, computational photography, neuro-imaging, and ocean microscopy. Before arriving at UCSD, he was an MS student at Duke University, studying computational microscopy.

    Kyung Chul Lee is a PhD student in electrical engineering at Yonsei University advised by Prof. Seung Ah Lee. Additionally, he had the opportunity to work in the Biomedical Engineering Department at Duke University with Prof. Roarke Horstmeyer’s Lab supported by the Graduate Biomedical Research Fellowship funded by the Korean Government. His research is focused on computational imaging and microscopy, functional imaging system design, and machine-learning for bio-medical imaging. He has particular experience with Fourier ptychographic microscopy (FPM), phase mask-based lensless imaging, and speckle imaging using speckle statistics. He is also open to exploring computational deep learning techniques aiming for joint optimization of hardware and software.

    Lucas Kreiss is interested in the applications of optical technologies to biomedical sciences. He works on diffuse correlation spectroscopy for label-free imaging of brain activity. Additionally, he learns more about how machine learning can provide computational specificity, for instance for digital staining of label-free imaging. Before coming to Duke University, he developed, engineered, and applied optical systems, such as multiphoton imaging, Raman spectroscopy, and diffuse reflectance spectroscopy. His multiphoton endomicroscope was particularly fascinating since it allowed label-free, in vivo imaging of the bowel at cellular resolution, without requiring surgery or biopsy samples.

    Kevin C. Zhou is a 2022 Schmidt Science Fellow and postdoc at UC Berkeley, working with Profs. Laura Waller and Hillel Adesnik. Before that, he was a postdoc (with Prof. Roarke Horstmeyer) and PhD student (with Profs. Joseph Izatt, Warren Warren, and Sina Farsiu) at Duke University. His research interests are broadly in computational imaging, coherent (and incoherent) optical imaging, tomographic reconstruction algorithms, inverse problems, and machine learning. He has particular experience with optical coherence tomography (OCT), camera array-based microscopy, Fourier ptychography, diffraction tomography, and nonlinear microscopy, but he’s always open to exploring and applying computational optimization techniques to other forms of imaging!

    Kanghyun Kim is currently pursuing a PhD in biomedical engineering. He recently completed an MS degree in electrical and computer engineering at Duke University. He received his BS degree in statistics and a minor in computer science from Chung-Ang University. His research focuses on designing task-specific microscopes using a deep neural network to improve image classification accuracy.

    Amey Chaware is a student focused on making a new breed of task-based optical systems. He is increasingly using machine learning to process the images taken. Instead of optimizing the two components—optics and algorithms—separately and sequentially, he treats the entire chain as a single system and develop an end-to-end optimization framework. Such a learning-based framework can potentially create better sensor systems for a variety of tasks.

    Jadee Neff is a hematopathologist and molecular genetic pathologist at Duke University. Her clinical interests are focused on the histologic examination of tissue and bone marrow biopsies to diagnose hematologic malignancies (leukemia, lymphoma, myeloma, etc.) as well as testing DNA from tumors or from blood to detect inherited or acquired mutations that can guide therapeutic management and predict clinical outcomes. Her research interests involve understanding the biology of chronic leukemias/lymphomas of cytotoxic lymphocytes; defining the immunomodulatory response to neoplastic disease; developing methods to monitor immune response and thereby refine tumor immunotherapy; and exploring novel applications of tumor genetics in the diagnosis, prognosis, and management of cancer.

    Carolyn Glass is an associate professor of pathology and a cardiothoracic pathologist at Duke University. She has research interests in development, new molecular biomarkers, and epigenetics. She completed medical residency in anatomic pathology at Brigham and Women’s Hospital, Harvard Medical School, followed by fellowships in cardiothoracic pathology also at Brigham and Women’s Hospital, Harvard Medical School, and pulmonary/cardiac transplant pathology at the University of Texas Southwestern Medical Center. She initially trained as a vascular surgeon with a focus on endovascular/interventional procedures through the 0+5 Integrated Vascular Surgery Program at the University of Rochester Medical Center from 2007 to 2011. As a recipient of the NIH National Lung Blood Institute T32 Ruth Kirschstein National Service Research Award, she completed a PhD with a concentration in genomics and epigenetics in 2014.

    Seung Ah Lee is an associate professor at the School of Electrical and Electronic Engineering at Yonsei University. Prior to Yonsei University, she was a scientist at Verily Life Sciences, and a former Google [x] team member. She received her PhD in electrical engineering at Caltech in 2014 and underwent postdoctoral training at Stanford Bioengineering. She completed her BS and MS degrees in electrical engineering at Seoul National University in 2007 and 2009.

    Oliver Friedrich is the head of Institute of Medical Biotechnology at Friedrich-Alexander-Universität (FAU). He had a quite diverse education, studying medicine and physics at University of Heidelberg, then specializing as physiologist and biophysicist before finally ending up at FAU as biotechnologist with a strong focus on label-free optical technologies development and applications. In between, he spent a year in Brisbane at the University of Queensland and has been going back and forth between Germany and Australia ever since. In Erlangen, he directs the Institute of Medical Biotechnology.

    Roarke Horstmeyer is an assistant professor in the Biomedical Engineering Department at Duke University. He develops microscopes, cameras, and computer algorithms for a wide range of applications, from forming 3D reconstructions of organisms to detecting neural activity deep within tissue. His areas of interest include optics, signal processing, optimization, and neuroscience. Most recently, he was a guest professor at the University of Erlangen in Germany and an Einstein postdoctoral fellow at Charitè Medical School in Berlin. Prior to his time in Germany, he earned his PhD from Caltech’s Electrical Engineering Department in 2016, his Master of Science degree from MIT Media Lab in 2011, and his bachelor’s degree in physics and Japanese from Duke University in 2006.

    [6] W. J. Schmidt. Die Bausteine des Tierkörpers in polarisiertem Lichte(1924).

    [8] G. Nomarski. Differential microinterferometer with polarized waves. J. Phys. Radium Paris, 16, 9S(1955).

    [32] L. Yang et al. Lensless polarimetric coded ptychography (POL-CP) for high-resolution, high-throughput birefringence imaging on a chip(2023).

    [33] L.-H. Yeh et al. uPTI: uniaxial permittivity tensor imaging of intrinsic density and anisotropy(2020).

    [36] K. Seo et al. Symmetry breaking of HPSCS in micropattern generates a polarized spinal cord-like organoid (PSCO) with dorsoventral organization(2021).

    [37] S. Song et al. Polarization sensitive intensity diffraction tomography. Light Sci. Appl., 12, 124(2023).

    [45] M. Born, E. Wolf. Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light(2013).

    [46] A. D. Yaghjian. A Direct Approach to the Derivation of Electric Dyadic Green’s Functions(1978).

    [49] R. A. Chipman, W. S. T. Lam, G. Young. Polarized Light and Optical Systems(2018).

    [56] A. Paszke et al. PyTorch: an imperative style, high-performance deep learning library(2019).

    [57] Y. E. Nesterov. A method of solving a convex programming problem with convergence rate O(k1/2). Doklady Akademii Nauk, 269, 543-547(1983).

    Tools

    Get Citation

    Copy Citation Text

    Shiqi Xu, Xi Yang, Paul Ritter, Xiang Dai, Kyung Chul Lee, Lucas Kreiss, Kevin C. Zhou, Kanghyun Kim, Amey Chaware, Jadee Neff, Carolyn Glass, Seung Ah Lee, Oliver Friedrich, Roarke Horstmeyer. Tensorial tomographic Fourier ptychography with applications to muscle tissue imaging[J]. Advanced Photonics, 2024, 6(2): 026004

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Articles

    Received: May. 2, 2023

    Accepted: Feb. 1, 2024

    Posted: Feb. 2, 2024

    Published Online: Mar. 5, 2024

    The Author Email: Horstmeyer Roarke (roarke.w.horstmeyer@duke.edu)

    DOI:10.1117/1.AP.6.2.026004

    Topics