The development of high-resolution in vivo endoscopy at the cellular/subcellular level is essential for the early diagnosis of cancer.1^–6 Current microendoscopic imaging is currently based primarily on two major methods: the first is an electronic endoscope that captures wide-field images using miniaturized lenses and detectors; the second is fiber-optic endoscopy based on confocal laser scanning.2^,7^–14 Despite these technological advances, one practical need in the development of microendoscopy remains unmet: it is related to light scattering, which limits the useful optical penetration depth achievable in biological tissue. It is imperative that microendoscopes overcome the problem of optical scattering within tissues to facilitate imaging at deeper tissue layers. This capability is essential for the detailed visualization and analysis of subsurface structures, which is critical in biomedical applications, such as cancer diagnosis.

Recently, ultrathin microendoscopes have been constructed using multimode fibers (MMFs) and successfully demonstrated for in vivo deep brain and alimentary tract imaging.15^–22 MMF endoscopy has the potential to realize high-fidelity subcellular imaging and to assist in the diagnosis of early-stage cancer progression, providing optical access to otherwise inaccessible deep brain and lumenal locations. However, when light passes through an MMF and interacts with deep biological tissues, the information conveyed by the endoscope is perturbed and degraded. In MMFs, severe wavefront distortions caused by modal dispersion and mode coupling result in highly complex and unpredictable output wavefronts. Traditional adaptive microscopy techniques, such as wavefront direct sensing and image-based optimization, are ineffective in this context because they rely on the assumption of relatively stable and correctable wavefront distortions.23^–36 The intricate modal interactions in MMFs lead to significant challenges in accurately sensing and correcting the wavefront, rendering these methods inadequate for adaptive imaging through MMFs.

In this work, we have developed a dynamically encoding, cascaded optical and ultrathin polychromatic light-field endoscopy (DECOUPLE). The method corrects the effects of sample-induced aberration and MMF dispersion, simultaneously. DECOUPLE is also well suited for integration with beacon methods, making it highly compatible with applications involving flexible and moving fibers. We demonstrate that DECOUPLE can be utilized for three-dimensional (3D) cell-level imaging beneath the tissue surface. The application of DECOUPLE significantly enhances both the image resolution by 1.4 times and the peak signal by 2 times. Fine structures that were previously unrecognizable in the presence of various aberrating samples (glue, 120-μm-thick of onion epidermis, 80-μm-thick layers of fat emulsions, etc.), become clearly resolved. These methods may enable the future of in vivo imaging of cancer tissue with cellular resolution, enabling research into next-generation cancer treatment. The integration of deep learning with the DECOUPLE technique holds promise for enhancing system performance, optimizing image quality, and improving efficiency in future applications, as deep learning has the potential to effectively process and extract features from complex optical data, offering significant advancements in adaptive and MMF imaging.37^,38

For MMF microendoscopy imaging in thick luminal tissues, the degradation of image quality is caused by sample-induced aberration and MMF dispersion [Fig. 1(a)]. Due to their interaction, it is imperative to simultaneously decouple these varying quantities. Moreover, these factors are also wavelength-dependent, forming a complex dynamic system, as described in the following equation: Iout=∑λ∑x,y[C(λ,x,y)×R(λ,x,y)×A(λ,t,x,y)×T(λ,x,y)×ψIn(λ,x,y)]2.(1)

Here, ψIn(λ,x,y) is the input light field at wavelength λ. T(λ,x,y) represents the MMF’s transmission matrix (TM), considering wavelength λ. A(λ,t,x,y) describes the aberrations caused by biological tissues at a wavelength and change over time t. R(λ,x,y) denotes the sample’s reflection/fluorescence characteristics at a given location. C(λ,x,y) indicates the efficiency of the MMF in capturing the returned light intensity. Iout represents the total light intensity detected by a bucket detector.

The DECOUPLE method distinguishes itself by efficiently separating and analyzing disturbances from both the fiber and tissue scattering through a two-step cascaded process. This involves shifting the problem to the spatial-frequency domain for better handling of coupling phenomena [Fig. 1(a)].

First, the MMF’s TM is measured using an off-axis holography method combined with a Hadamard basis under orthogonal polarization [as shown in Fig. 1(b)]. To minimize noise from air-flow disturbances and camera signal fluctuations, the process is repeated multiple times, and the resulting matrices are averaged. This high-fidelity transmission matrix measurement is essential for precise control of the fiber’s output light field through spatial-frequency regulation, effectively preventing modal dispersion from interfering with subsequent adaptive imaging. Following this, to achieve optimal focus within an optical system, it is crucial that all incoming light rays are precisely refracted upon entering the system’s pupil, ensuring their convergence at a single point with a uniform phase. This point of maximum constructive interference leads to optimal focus and resolution. However, in deep-tissue imaging, a further challenge arises from the mismatch between the refractive indices of the fiber material and the sample tissue. These discrepancies lead to the deflection of light rays from their intended paths and induce phase shifts, hindering their ability to converge at a unified focal point. To address this, DECOUPLE partitions the fiber’s output spatial-frequency domain into N subregions. In the first step, the system adjusts the tilt of different pupils to ensure that their focus positions align at a common point [as shown in Fig. 1(c)]. In the second step, two pupils are opened and focused on the same point [as shown in Fig. 1(d)]. By scanning this position and varying the phase of one pupil, the system collects the fluorescence/reflectance signal intensity to determine whether the focal point has achieved maximum constructive interference. This process is repeated for different pupils, ensuring that the phase is optimal across the system. This advanced manipulation corrects aberrations introduced by refractive-index inhomogeneities, ensuring that light rays converge accurately at the intended focal point [Fig. 1(e)]. Thus, through a cascaded approach, DECOUPLE effectively handles both the initial challenges of modal dispersion and the subsequent aberrations, as discussed in Note 1 and Figs. S1 and S2 in the Supplementary Material.

In stark contrast to conventional techniques,18^–21^,39^,40 DECOUPLE supports multicolor imaging and simultaneously compensates for both fiber dispersion and sample aberrations. Its unique aberration measurement approach eliminates the need for wavefront sensors, enabling noninvasive, high-resolution imaging ideal for in vivo applications. This innovation represents a significant advancement in adaptive imaging for complex optical systems, addressing the limitations of conventional approaches (as discussed in Note 3 in the Supplementary Material).

To verify the capability of DECOUPLE to eliminate dispersion in the MMF as well as accurately compensate for sample-induced aberration, we conducted simulations using fluorescent microspheres, a resolution test target, and colored letters, as illustrated in Fig. 2(a). By introducing simulated aberrations and dispersion to the ground truth, we aimed to test the ability of our system to correct these distortions and quantitatively assess improvements in peak signal-to-noise ratio (PSNR). Mode dispersion and simulated aberrations (a random combination of Zernike modes up to the 13th, excluding piston, tip, tilt, and defocus) were introduced to the ground truth, resulting in a blurred image. The application of DECOUPLE enabled the gradual restoration of the image to a clear state. The second column shows the results after the TM has been correctly adjusted to account for MMF dispersion. In the final step (third column) the sample-induced aberration is also corrected. Figure 2(b) displays both the simulated and measured aberration phase profiles. To quantify these results, we calculated the PSNR for the image of the resolution test target in the region indicated by the white dotted box in Fig. 2(a). The PSNR increased from 12 to 15 [Fig. 2(c)], providing evidence of DECOUPLE’s effectiveness in enhancing image clarity and detail. Furthermore, in the context of imaging in multiple colors, we realized an improvement in the PSNR from 18 to 22 for the colored letter sample [Fig. 2(d)]. This signifies DECOUPLE’s capacity to correct for multicolor aberrations, underpinning its versatility for a variety of imaging applications.

The experimental setup depicting a DECOUPLE system is illustrated in Fig. 3. The corresponding model is shown in Table S1 in the Supplementary Material. The entire system is divided into four parts: the modulation module, the transmission measurement module, the reflection measurement module, and the collection module. The modulation module is used to control the polarization, phase, and amplitude of the incident light field. The output from one of the light sources (a semiconductor laser at 488 nm) is divided into a reference and a signal beam. The reference light is guided into a single-mode fiber through an optical coupler. The signal beam is expanded and collimated by lenses L1 and L2 to form a plane wave. Subsequently, the light is directed toward a digital micromirror device (DMD) and reflected out. The DMD modulates the phase of the first-order diffracted light, and an iris blocks other diffraction orders. To image the Fourier plane of the DMD onto the input facet of the MMF, a microscope objective (OBJ1) with a 4f system comprising lenses L4 and L3 is employed. A quarter-wave plate (QWP1) and a half-wave plate (HWP1) adjust the light to circular polarization, ensuring the two orthogonally polarized beams from PBS3 have equal energy for exciting the corresponding fiber modes. Custom polarization modes were generated by employing two-angle holograms on the DMD, which directed the light into separate regions within the Fourier plane. Following this, the beams were divided by a polarizing beam splitter (PBS3), and their trajectories were altered using a pair of mirrors (M3, M4). The polarization rotation was achieved with two quarter-wave plates (QWP3, QWP4). After the light passed through the QWPs for the second time, the polarization of the exiting beam was orthogonal to its original state. The beams were then recombined using the PBS3. The method enables the superposition of two light orders in the Fourier plane, allowing for the manipulation of various polarization states through precise pixel-block-level adjustment of the relative phase between two polarized beams.

The transmission measurement module is used to measure the outgoing light field of the MMF. For the output measurement, images of the output facet of the fiber are acquired by a movable calibration module. The output plane is re-imaged to the camera CMOS1, utilizing a microscope objective (OBJ2) and a lens L5. These reference beams are coupled by a fiber-optic broadband coupler (FOBC), indicated by the dashed box titled “Reference Beam Coupler,” chosen for its compactness and resistance to environmental interference, enhancing the system’s stability and reliability. After passing through the FOBC, the reference light travels through single-mode fibers (S1, S2) and is then collimated and expanded by fiber collimators (F3). The light from the MMF and the reference light from the single-mode fiber are both filtered to the same linear polarization state by a linear polarizer (LP), ensuring interference contrast before they are combined with the reference signal using a beam splitter (BS3). Another color excitation path for a 561-nm wavelength laser source is arranged in similar fashion to the 488 nm laser.

The reflection measurement module is used to measure the reflection matrix and track the deformation state of the MMF during imaging. For the reflection matrix measurement, the 4f system consisting of lenses L10 and L11 images the back focal plane of the microscope objective (OBJ1) onto the camera CMOS2. Compared with single-color interferometric measurement, a reference-free measurement approach is employed to achieve stable and straightforward reflection matrix (RM) measurement in the multicolor system (Note 4 in the Supplementary Material). After calibration of the TM and reflection matrix, various designed patterns are projected onto the fiber’s proximal end using the DMD. This achieves raster scanning at the distal end of the fiber.

The collection module is used to collect reflected light/fluorescence and optimize imaging through pupil segmentation. Fluorescent samples were excited, and the emitted fluorescence was collected with the same MMF. Along the emission path, two short-pass dichroic mirrors are installed in the collection light path to guide the fluorescence to PMT1 (488-nm laser excitation) and PMT2 (561-nm laser excitation). After obtaining the image signal, DECOUPLE is performed to eliminate aberrations, ensuring the accurate reconstruction of the imaged samples.

From the simulations, it is seen that it is necessary to compensate for dispersion and aberrations simultaneously to achieve a high-resolution and nondistorted image. The complex interplay of these processes underscores the need for a holistic approach to image enhancement in fiber-optic imaging. In this section, a series of experimental designs that we employed to validate the efficacy of DECOUPLE is presented. Through a systematic staging of different compensation procedures, we can achieve optimal imaging performance and show that the scheme is fit for application under realistic imaging scenarios and computationally and practically feasible.

In a typical imaging environment involving biological tissue, aberrations can be classified into two categories, refractive index mismatch and multilayer cell scattering. We present two examples to assess the effectiveness of DECOUPLE in correcting for these two types of aberrations, respectively. In the first example, we used an MMF to image fluorescent microspheres with a diameter of 1μm. Because the initial TM is calibrated in air, the introduction of a liquid during biological imaging results in spherical aberrations that decrease image contrast during imaging [Fig. 4(a)]. The number of pupil subdivisions plays a crucial role in correcting these aberrations. Increasing the number of subdivisions enhances the precision of aberration measurements but also presents challenges. A higher number of subdivisions reduces the available excitation light intensity per pupil, which results in increased detection noise. It also decreases the number of modulation modes per pupil, thereby reducing the power ratio of the focal point output through the optical fiber (as shown in Fig. S3 and Note 2 in the Supplementary Material). Considering the practical trade-offs, we have chosen a subdivision precision of 4×4. Using DECOUPLE with 16 independent subregions that do not overlap with each other, the PSNR increases twofold, and the full width at half-maximum of the bead images approximates the diffraction limit [Fig. 4(b)]. To validate the effectiveness of our method in correcting aberrations induced by multilayer cell scattering, we conducted an experiment using 120-μm-depth onion epidermal slices placed above fluorescent beads. Before correction, scattering in the slice leads to severe corruption of the wavefront of the excitation light, resulting in a degraded image. After correction in 16 independent subregions that do not overlap each other, the image contrast and the peak signal increase by about 2 times [Fig. 4(c)]. Through these two examples, we demonstrate the capabilities of our MMF microendoscope in correcting aberrations caused by refractive index mismatch and multilayer cell scattering.

Having established the DECOUPLE capabilities for nondeformable fiber scenarios, we extended our investigation to bending settings. When an MMF’s curvature changes, its refractive index varies, causing dynamic dispersion. DECOUPLE corrects sample aberrations after obtaining the TM, whereas spatial-frequency tracking adaptive beacon light-field-encoded method efficiently tracks the transmission matrix of the curved MMF.18 If combined, they offer a robust solution for aberration correction in flexible MMFs. We utilized a flexible fiber subjected to bending to image a resolution test target covered by a 10% fat emulsion. Using the DECOUPLE scheme, the image transitions from a state of complete chaos to a blurred image and ultimately a clear image emerges [Fig. 5(a)]. This progression not only illustrates the method’s adaptability in dynamic environments but also underscores its potential to significantly enhance image clarity and resolution in complex imaging scenarios. Following this demonstration, we performed a quantitative characterization of the resolution enhancement achievable with DECOUPLE imaging in dynamically disturbed environments. This step is crucial to appraise the practical ability of DECOUPLE to perform adaptive correction in real time during imaging in challenging environments. For these experiments, we placed slices of female Ascaris roundworms and lung tissue underneath the 120-μm onion epidermis and, in a separate experiment, used a 10% fat emulsion to cover a resolution target, both for comparative imaging under dynamic MMF bending conditions [as shown in Figs. 5(b) and 5(c)]. In Fig. 5(d), we observe substantial improvements in image quality and spatial resolution after DECOUPLE. The peak signal intensity increased approximately by a factor of 2 after applying the correction with significant improvements in lateral resolution also observed [Fig. 5(e)]. We also assess image sharpness through a calculation of the power spectrum of spatial frequencies, k, contained in images before and after correction. Here, a higher spatial-frequency k is substantially recovered by DECOUPLE correction, which can be observed by comparing the power of spatial-frequency distribution [Fig. 5(f)]. We can see a significant improvement in the distribution and intensity of the frequency components, especially in the high-frequency bands, showing that DECOUPLE correction restores information and improves image quality.

Transitioning from 2D to 3D multicolor imaging enhances diagnostic capability for the study of complex 3D tissue. Adjusting the defocus contained in the TM can be used for refocusing the imaging plane without mechanically moving any parts. We demonstrate dual-color imaging with DECOUPLE by imaging a collection of fluorescent beads randomly distributed underneath an onion epidermis [Fig. 6(a)]. As shown in Fig. 6(b), the images were captured at depths ranging from 110 to 180μm. The bottom row shows a clear improvement in imaging fidelity after DECOUPLE is deployed, with clear improvements both in the lateral resolution and in image contrast.

DECOUPLE comprises a comprehensive framework of algorithms and hardware designed to eliminate dynamically changing aberrations and dispersion in endoscopic imaging. The method shows particular promise for MMF microendoscopes, deals with fiber mode dispersion, and enables the raster-scanning imaging method without the requirement for any mechanically moving parts. The current decoupling process takes ∼30s, primarily involving the calculation of holograms for spatial-frequency control and their upload to the DMD. Although this duration is relatively long, it remains feasible for applications such as imaging in fixed brain regions and endoscopic imaging, where the degradation of tissue aberrations occurs over a longer timescale than this processing time. Its effectiveness has been validated across a range of scenarios, including index-mismatched and weakly scattering samples. We show that depth-resolved imaging is possible in complex samples in two colors. DECOUPLE leverages the advantages of minimal invasiveness of MMF, empowering this imaging method for 3D imaging within biological tissues and in luminal spaces within the body. Our method shows that adaptive microendoscopy is possible for dynamic real-time imaging in realistic biomedical scenarios. There is scope for many developments. For example, two-photon excitation could be integrated into the method and deep-learning frameworks deployed to empower adaptive control over the imaging process. Both promise to bring further enhancements for tissue penetration and to speed up aberration and dispersion compensation. Our ultimate goal is to develop these tools so that in situ biological studies can be performed routinely within whole organisms, enabling the real-time monitoring of physiological processes deep within freely moving animals.

Zhong Wen received his PhD from the College of Optical Science and Engineering, Zhejiang University, in 2023. He is now an assistant researcher at Zhejiang University. His research focuses on multimode fiber endoscopes and computational imaging.

Xu Liu received his DSc from L’Ecole Nationale Supérieure de Physiquede Marseille in France. He has been a professor at the College of Optical Science and Engineering, Zhejiang University since 1995. His research interests include optoelectronic display, optics and optoelectronic thin films, optical imaging, and bio-optical technologies.

Qing Yang received her PhD from the College of Materials Science and Engineering, Zhejiang University, in 2006. She was a visiting scientist in the Department of Materials Science and Engineering, Georgia Institute of Technology, from 2009 to 2012. She was a visiting scientist at the University of Cambridge in 2018. Currently, she is a professor at the College of Optical Science and Engineering, Zhejiang University. Her research focuses on micro/nanophotonics, nanomaterials, and endoscopy imaging.

Biographies of the other authors are not available.