Advanced Imaging, Volume. 1, Issue 2, 021001(2024)

High-throughput, nondestructive, and low-cost histological imaging with deep-learning-assisted UV microscopy

Jiajie Wu... Weixing Dai, Claudia T. K. Lo, Lauren W. K. Tsui and Terence T. W. Wong* |Show fewer author(s)
Author Affiliations
  • Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
  • show less

    Pathological examination is essential for cancer diagnosis. Frozen sectioning has been the gold standard for intraoperative tissue assessment, which, however, is hampered by its laborious processing steps and often provides inadequate tissue slide quality. To address these limitations, we developed a deep-learning-assisted, ultraviolet light-emitting diode (UV-LED) microscope for label-free and slide-free tissue imaging. Using UV-based light-sheet (UV-LS) imaging mode as the learning target, UV-LED images with high contrast are generated by employing a weakly supervised network for contrast enhancement. With our approach, the image acquisition speed for providing contrast-enhanced UV-LED (CE-LED) images is 47s/cm2, 25 times faster than that of the UV-LS system. The results show that this approach significantly enhances the image quality of UV-LED, revealing essential tissue structures in cancerous samples. The resulting CE-LED offers a low-cost, nondestructive, and high-throughput alternative histological imaging technique for intraoperative cancer detection.

    Keywords

    1. Introduction

    Formalin-fixed and paraffin-embedded (FFPE) tissue processing is the gold standard for making histology slides for cancer diagnosis. However, the tissue preparation protocol normally requires up to a week. Currently, the intraoperative approach relies on the frozen section procedure, which encompasses tissue freezing, sectioning, fixation, and staining. Despite being the only standard method for intraoperative histological slide preparation, this process is complex and requires a long turnaround time, typically taking around 30 min for tissue transportation and to generate tissue slides for diagnosis under traditional bright-field microscopes. This complicated sample preparation procedure not only compromises slide quality, which affects the diagnosis outcomes (e.g., when handling lipid-rich tissue) but also limits the throughput of procedures, making it difficult for pathologists to rapidly and reliably extract histological information during tumor excision surgeries to help tissue harvesting decisions.

    Modern slide-free histological imaging techniques offer rapid cellular visualization, both with and without chemical stains, to address the aforementioned pain points by eliminating tissue preparation steps (including freezing, sectioning, fixation, or even staining). Techniques like fluorescence confocal microscopy[1,2], fluorescence light-sheet microscopy[3,4], and microscopy with ultraviolet (UV) surface excitation[5] utilize exogenous stains to mimic contrast similar to hematoxylin and eosin (H&E) staining. However, the use of chemical stains is often prohibited in operating theaters due to volatility and toxicity concerns. Additionally, these stains may affect the quality of downstream molecular analyses. Label-free imaging technologies, such as reflective confocal microscopy (RCM)[6,7], optical coherence tomography (OCT)[8,9], ultraviolet photoacoustic microscopy (UV-PAM)[1015], stimulated Raman scattering (SRS) microscopy[1618], and deep-UV wide-field microscopy[19,20], eliminate the need for chemical stains by probing the intrinsic properties of biological tissues. The imaging contrast of RCM and OCT is based on different tissue refractive indices, which may not correlate well with nuclear and cytoplasmic contrast generated by a histological H&E stain. Comparatively, SRS microscopy based on molecular-specific signals and UV-PAM based on the UV-absorption properties of DNA and RNA may provide more accurate subcellular and connective tissue structures that are shown in H&E-stained tissue. However, these point-scanning imaging techniques are inherently limited in imaging speed, posing challenges for intraoperative applications. Deep-UV wide-field microscopy enables slide-free histological imaging by UV-autofluorescence excitation on the tissue surface. DNA and RNA in nuclei have strong absorption to deep-UV[21] while appearing dark in the image, while endogenous molecules, including NADH, tryptophan, flavin, collagen, and elastin in the cytoplasm and connective tissue emit autofluorescence[22], providing high molecular contrast. Our previous work, the computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP)[20], utilizes a UV excitation to provide high molecular contrast and a laser for pattern illumination to improve lateral resolution, subsequently revealing similar structures as H&E-stained histological images. Also, another recent work, microscopy with ultraviolet single-plane illumination (MUSI)[23], shows the possibility of utilizing single-plane illumination microscopy based on autofluorescence contrast for three-dimensional (3D) tissue imaging with high axial resolution, demonstrating the capability of revealing lung cancer subtypes with multiple layers. Despite their notable advantages, both CHAMP and MUSI require illumination with a UV laser. Nonetheless, UV lasers pose challenges due to their costliness, fragility, and difficulty in maintaining high and stable energy, which could impede their clinical applications.

    To achieve histological contrast without the need for chemical stains and high-quality imaging without laser illumination, we acquire UV light-emitting diode (LED)-based wide-field microscopy (termed UV-LED) images with a deep-learning-based contrast enhancement network to learn the target UV light-sheet (termed UV-LS) images. A dual-modality imaging system available to acquire the source domain data from the UV-LED and the target domain data from the UV-LS has been constructed to minimize tissue and image distortion for training the deep-learning network [Fig. 1(a)].

    Dual-modality tissue imaging systems for UV-LED and UV-LS image acquisition with the deep-learning-based contrast enhancement frameworks. (a) Overview of the dual-modality imaging system, facilitating both UV-LED and UV-LS imaging. Both modes share an optical path for data acquisition. (b) Detailed setup of the tissue scanning region, incorporating a quartz solid immersion lens to correct refraction in the illumination and detection paths. (c) Selection of a 2D image layer close to the sample surface from the 3D UV-LS image stack. (d) Application of the U-Frame network to enhance the contrast of UV-LED images by learning from UV-LS images.

    Figure 1.Dual-modality tissue imaging systems for UV-LED and UV-LS image acquisition with the deep-learning-based contrast enhancement frameworks. (a) Overview of the dual-modality imaging system, facilitating both UV-LED and UV-LS imaging. Both modes share an optical path for data acquisition. (b) Detailed setup of the tissue scanning region, incorporating a quartz solid immersion lens to correct refraction in the illumination and detection paths. (c) Selection of a 2D image layer close to the sample surface from the 3D UV-LS image stack. (d) Application of the U-Frame network to enhance the contrast of UV-LED images by learning from UV-LS images.

    In terms of microscopic imaging style transformation, deep-learning techniques are primarily divided into supervised and unsupervised learning networks. Supervised learning is renowned for its effectiveness in tasks such as virtual staining[24] and superresolution[25], offering outstanding performance when training data are well paired, whereas unsupervised learning, such as cycle-consistent adversarial network (CycleGAN)[26] and the transformation for optical microscopy (UTOM)[27], accommodates training with unpaired images.

    In this work, we employ a contrast enhancement network to transform UV-LED images into target UV-LS images to generate contrast-enhanced (CE)-LED images [Fig. 1(d)]. The source domain, represented by UV-LED images, captures the projection of the autofluorescence signal within the UV penetration depth. Conversely, the target domain, represented by UV-LS images, is one of the 2D image layers (optical sectioning thickness = 1.8 µm) selected from a 3D stack within the UV penetration depth. Consequently, the two image domains are not perfectly paired. However, they share a high degree of morphological similarity. In the context of imaging systems where UV-LED and UV-LS images only partially align, traditional supervised learning approaches may accumulate errors due to the requirement of perfectly aligned image pairs. Conversely, unsupervised learning methods might fail to fully utilize the high morphological similarities between the images from these two domains. To resolve these issues, we applied our recently developed unified deep-learning framework, termed U-Frame[28], which can adjust the learning level from supervised learning to unsupervised learning with an automatically adjusted tolerance size according to the image misalignment, thus handling partially paired data effectively without necessitating exact registration between image pairs.

    In our study, we demonstrate that by utilizing the U-Frame network to transform UV-LED images into target UV-LS images, it is possible to significantly enhance the contrast and highly resemble the tissue structures shown in corresponding H&E-stained images rapidly through direct inference. UV-LS microscopy typically requires around 22  min/cm2 for image acquisition. In contrast, our CE-LED method, which combines UV-LED image capture (30  s/cm2) with deep-learning inference using the U-Frame network (17  s/cm2 with an NVIDIA GeForce RTX 3090 GPU), provides high-contrast images similar to high-axial resolution UV-LS images in 47  s/cm2. Also, after training, this method eliminates the need for an expensive and fragile UV laser and utilizes a stable and cost-effective UV-LED wide-field system instead. Additionally, we demonstrate the potential of virtual H&E staining on CE-LED images using the U-Frame network to further enhance interpretability for clinical use. Overall, our approach offers a high-throughput, high-resolution, economical, and nondestructive alternative for rapid intraoperative tissue imaging.

    2. Methods

    2.1. Preparation of biological tissues

    Mouse brains were collected from C57BL/6 mice. The mice were euthanized in a CO2-infused chamber, and their organs were subsequently harvested and fixed in 4% neutral-buffered formalin for 24 h at room temperature. These procedures were strictly adhered to under a protocol approved by the Health, Safety, and Environment Office of The Hong Kong University of Science and Technology (license number: AH18038). Human lung cancer specimens were obtained by surgeons from Queen Mary Hospital during lobectomy or sublobectomy procedures and immediately placed in formalin. These tissues were then sectioned into smaller pieces, rebuffered in formalin, and transported to the lab for imaging, per an ethical protocol approved by the University of Hong Kong/Hospital Authority Hong Kong West Cluster (reference number: UW 20-335).

    Both the animal and human specimens underwent further processing, being sectioned into 0.5 mm slices using a vibratome (VT1200S, Leica Microsystems Inc.) to generate more samples for image acquisition. The slices were preserved in phosphate-buffered saline at 4°C. After imaging by UV-LED and UV-LS, tissue was processed by the FFPE procedure to be made into slides for image quality validation. This included a 2-h dehydration step using a tissue processor (Revos, Thermo Fisher Scientific Inc.), followed by embedding in paraffin with an embedding machine (HistoStar, Thermo Fisher Scientific Inc.). The embedded tissue in the paraffin blocks was then sectioned into 5 µm slices using a microtome (RM2235, Leica Microsystems Inc.). These sections were deparaffinized, stained with H&E, and digitally imaged at 20× magnification using a whole slide scanner (NanoZoomer-SQ, Hamamatsu Photonics K.K.).

    2.2. System setup and image acquisition

    2.2.1. System setup and image acquisition for UV-LED mode

    Figure 1(a) shows the UV-LED mode, an inverted microscope setup with an optical detection path perpendicular to the imaging plane. Light from a UV-LED (M265L4, Thorlabs Inc.) is expanded with a pair of lenses (ASL2520-UV and LA4725-UV, Thorlabs Inc.) for wide-field illumination. The specimen is positioned on a lab-made aluminum sample holder attached with a 1-mm thick, round-shaped JGS1 quartz as the bottom imaging window. This sample holder is attached to a three-axis motorized translation stage (L-509.20SD00, PI miCos GmbH). A 5 × UV-objective lens (LMU-5×-NUV, numerical aperture (NA)=0.12, Thorlabs Inc.) is used to capture the emitted autofluorescence signal in a reflection mode, which is then filtered by a 450 nm long-pass filter (FELH0450, Thorlabs Inc.). The autofluorescence signal is then focused on a monochromatic camera (PCO edge 4.2 bi, PCO Inc.) by a two-inch tube lens (TTL200-A, Thorlabs Inc.) with a focal length of 200 mm. The monochromatic camera has 2048 pixel × 2048 pixel with a pixel size of 6.5 µm. The lateral resolution of the UV-LED system is 2.6 µm. The exposure time for each field of view is 500 ms, and when the sample holder is raster scanned with a 20% overlap, a sequence of images is recorded in 16-bit, achieving an image acquisition speed of 30  s/cm2.

    2.2.2. System setup and image acquisition for UV-LS mode

    The UV-LS mode, shown in Fig. 1(a), utilizes an open-top light-sheet microscope setup, enabling surface scanning of large and thick samples. In the UV-LS setup, illumination and detection paths are perpendicular to each other, each with a 45° angle to the imaging plane [Fig. 1(b)]. The illumination is provided by a 266-nm UV laser, with its beam expanded by lenses (LA4600-UV and LA4184-UV, Thorlabs Inc.) and in between spatially filtered by a pinhole (P30MB, Thorlabs Inc.). The beam diameter is regulated by an iris (ID20, Thorlabs Inc.) and shaped into a 3-mm long Gaussian sheet of light, with a beam waist of 1.8 µm focused by a cylindrical lens (LJ4395RM, Thorlabs Inc.). The detection path is adjusted from 90° for UV-LED imaging to 45° to be perpendicular to the path of the light sheet via an angle-adjustable mounting plate for UV-LS imaging. A lab-made JGS1 quartz solid immersion lens, matched with a nonfluorescent UV oil layer, cancels the refraction when illumination light and detection signal transition between air and the specimen [Fig. 1(b)]. The system leverages motorized sample movement across the static light sheet to capture a 3D volume image with both axial and lateral resolutions of 1.8 µm. The tissue surface roughness is mostly within 40 µm with weight compression during imaging, while the Rayleigh range of the illumination is 65 µm, ensuring clear imaging of the tissue surface. Tissue sections are raster-scanned in overlapping stripes covering 30% of the adjacent area to image the entire sample. The image acquisition speed is 2.77 min per 2048 pixel × 11111 pixel × 60 pixel, representing a volume of 1.84  mm×1  cm×54  μm.

    2.3. Image processing

    To stitch each field of view acquired from the UV-LED system, we fuse them with the Fiji plugin, grid/collection stitching, using a linear blending method in the sequence of image acquisition with 20% overlap to compose a complete image covering the scanning area. To reconstruct the 3D UV-LS images, 3D-volume shear transformation to 45° is applied in Python code. After reconstruction, as demonstrated in Fig. 1(c), within the penetration depth of the tissue, a 2D layer that is close to the tissue surface and resembles the morphology corresponding to the UV-LED image would be digitally extracted. Different 2D layers at the same depth in each 3D stripe of UV-LS image are extracted. These layers were then stitched into a large image covering the image region of the tissue, using the maximum intensity or linear blending method in the grid/collection stitching plugin of Fuji.

    Before the images are trained by the deep-learning network, the UV-LED image is registered to the UV-LS image using the Fiji plugin, TrakEM2, with manually selected points to improve image alignment. Given the disparity in optical sectioning thickness between the UV-LS (1.8 µm) and UV-LED (30 µm) systems, the two images are intrinsically different. Therefore, a 2D binary mask is generated from the UV-LS image to apply on the UV-LED image to obscure noncorresponding structures, enhancing the effectiveness of the training process.

    The U-Frame framework[28], depicted in Fig. 1(d), is utilized to train the contrast enhancement algorithm. This training uses UV-LED images as the source domain and UV-LS images as the target domain. Images were segmented into 128  pixel×128  pixel regions, expanded to 512  pixel×512  pixel with adjacent regions included as input patches to the generator. Each generated image undergoes evaluation by a discriminator, assessing the image as either real or fake. Once an image is deemed real by the classifier, it would further classify the image into one of the classes. Each class corresponds to the divided region in the input images with the tolerance size. A global sampling rule is applied to balance rare and common features in the data set. To evaluate the reliability of the generated CE-LED, spatial characters of cellular nuclei, including cross-sectional area and intercellular distance, are analyzed. CE-LED, UV-LS, and the corresponding H&E-stained images are transformed into 8-bit binary images and then segmented through thresholding to extract the nuclear regions using the Fiji plugin, threshold. Another Fiji plugin, termed analyze particles, was then applied to calculate the cross-sectional area and the centroid of the segmented nuclei. The minimal distance between each nucleus and its closest neighboring nucleus is calculated with a MATLAB script to get their intercellular distance.

    While training the virtual staining algorithm using the U-Frame network, the input source domain is the CE-LED image, and the target domain is the corresponding H&E-stained image acquired after tissue processing. The H&E-stained image is not perfectly paired with the autofluorescence image due to tissue distortion in the FFPE process and tissue consumption in tissue sectioning. However, they usually share morphological similarities. Since the source images are gray-scale (single-channel) and the target images are color (RGB), the Sobel–Feldman operator[29] was employed to compute two derivatives, which were normalized and converted into an 8-bit format, effectively creating two additional channels that were then merged with the original single-channel CE-LED image to form a three-channel input.

    3. Results

    We initially evaluated the performance of the CE-LED [Fig. 2(b)] imaging on mouse brain tissue. The mouse brain tissue was fixed, sectioned into 0.5 mm slices using a vibratome, and then scanned by the dual-modality system to acquire the UV-LED image [Fig. 2(a)] and UV-LS image [Fig. 2(c)]. Subsequently, the tissue underwent the FFPE process followed by scanning through a whole slide scanner to obtain the H&E-stained image [Fig. 2(d)] as the clinical ground truth for comparison. As shown in Fig. 2, both the zoomed-in UV-LED images [Figs. 2(g) and 2(k)] and UV-LS images [Figs. 2(i) and 2(m)] display negative nuclear contrast due to the strong absorption of UV light by DNA and RNA without emission. In contrast, endogenous molecules in the cytoplasm and connective tissue emit autofluorescence and appear bright. Consequently, UV autofluorescence images achieve a similar contrast compared to the corresponding images of H&E ground truth [Figs. 2(j) and 2(n)]. However, UV-LED images appear blurrier and have lower contrast relative to UV-LS images. This difference in clarity and contrast is attributed to UV-LED capturing the projection of signals along the UV penetration depth, whereas the UV-LS image is a one-pixel thick layer from the 3D stack, with a voxel size of 0.9μm×0.9μm×0.9μm. To test the performance of U-Frame for contrast enhancement of UV-LED while eliminating the challenge of training data insufficiency, we separated the mouse brain into two sets, each containing half of the brain image for training and inference, respectively. The CE-LED zoomed-in images [Figs. 2(h) and 2(l)] show that U-Frame generated outcomes closely matched the image quality of UV-LS images, significantly enhancing the contrast compared to the original UV-LED images. When comparing CE-LED [Fig. 2(h)] to UV-LS [Fig. 2(i)] and the corresponding H&E-stained images [Fig. 2(j)], both the cross-sectional area and the intercellular distance have no significant difference among the three image domains, with all P values >0.05. Ordinary one-way ANOVA testing is conducted across all the extracted nuclei, with the number of nuclei more than 200 in each image, for comparison. The results show the reliability of our CE-LED image in revealing the real tissue features similar to the clinical ground truth.

    Comparison among UV-LED, CE-LED, UV-LS, and the corresponding H&E-stained images on thick healthy mouse brain tissue. (a) UV-LED image of half of the mouse brain; (b) CE-LED image generated from the UV-LED image in (a); (c) UV-LS image of the same mouse brain specimen as (a); (d) corresponding H&E-stained image obtained after imaging with UV-LED and UV-LS modes; (e) cross-sectional area measurement of (h)–(j); (f) intercellular distance measurement of (h)–(j); (g)–(j) zoomed-in views of green dashed rectangular regions marked in (a)–(d), respectively; (k)–(n) zoomed-in views of blue rectangular regions marked in (a)–(d), respectively.

    Figure 2.Comparison among UV-LED, CE-LED, UV-LS, and the corresponding H&E-stained images on thick healthy mouse brain tissue. (a) UV-LED image of half of the mouse brain; (b) CE-LED image generated from the UV-LED image in (a); (c) UV-LS image of the same mouse brain specimen as (a); (d) corresponding H&E-stained image obtained after imaging with UV-LED and UV-LS modes; (e) cross-sectional area measurement of (h)–(j); (f) intercellular distance measurement of (h)–(j); (g)–(j) zoomed-in views of green dashed rectangular regions marked in (a)–(d), respectively; (k)–(n) zoomed-in views of blue rectangular regions marked in (a)–(d), respectively.

    To evaluate the performance of the U-Frame algorithm in processing complex human cancerous tissues compared to other well-established algorithms, cycleGAN (the unsupervised method) and pix2pix (the supervised learning method), we use the same input data for training to generate the results of these three deep learning networks for comparison. Our training and testing data are acquired from two adjacent vibratome sections to minimize any influence from insufficient tissue features in the training data set that appears in the testing one. The original UV-LED image and the corresponding UV-LS image are displayed in Figs. 3(a) and 3(e), respectively. Figures 3(b)3(d) demonstrate the inference results from cycleGAN, pix2pix, and U-Frame networks, respectively. U-Frame demonstrates better enhancement of tissue contrast while preserving the correct tissue structures compared to both cycleGAN and pix2pix. The yellow arrow in the zoomed-in cycleGAN result [Fig. 3(g)] reveals a mosaic artifact, while the zoomed-in pix2pix result [Fig. 3(h)] fails to reveal the bright fibrous structures indicated by the green arrow. In contrast, the U-Frame output [Fig. 3(i)] provides a stabler result that closely resembles the actual UV-LS image [Fig. 3(j)], offering better tissue representation. Additionally, in other zoomed-in views [Figs. 3(k)3(o)], as highlighted by the yellow arrow, the cycleGAN output [Fig. 3(l)] tends to exaggerate fibrous structures, while the pix2pix result [Fig. 3(m)] exhibits poorer cellular contrast in the region indicated by the green arrow. In comparison, the tissue structure in the U-Frame result [Fig. 3(n)] is more similar to that in the UV-LS image [Fig. 3(o)]. Furthermore, in the final set of zoomed-in views [Figs. 3(p)3(t)], the yellow arrow reveals denser and smaller nuclear structures in the cycleGAN output [Fig. 3(q)], and the pix2pix result fails to generate a space in the gland-like structure, which should be located in the area highlighted by the green arrow. Referring to the target UV-LS image [Fig. 3(t)], the U-Frame output [Fig. 3(s)] surpasses the outputs of the other two models in terms of image quality and tissue structure.

    Comparison of contrast enhancement results using cycleGAN, pix2pix, and U-Frame compared to the original UV-LED and UV-LS on human lung cancer tissue. (a) UV-LED image. (b)–(d) CycleGAN, pix2pix, and U-Frame results of CE-LED images generated from (a), respectively; (e) UV-LS image of the same specimen as in (a); (f)–(j) zoomed-in views of yellow dashed rectangular regions marked in (a)–(e), respectively; (k)–(o) zoomed-in views of light green rectangular regions marked in (a)–(e), respectively; (p)–(t) zoomed-in views of red rectangular regions marked in (a)–(e), respectively.

    Figure 3.Comparison of contrast enhancement results using cycleGAN, pix2pix, and U-Frame compared to the original UV-LED and UV-LS on human lung cancer tissue. (a) UV-LED image. (b)–(d) CycleGAN, pix2pix, and U-Frame results of CE-LED images generated from (a), respectively; (e) UV-LS image of the same specimen as in (a); (f)–(j) zoomed-in views of yellow dashed rectangular regions marked in (a)–(e), respectively; (k)–(o) zoomed-in views of light green rectangular regions marked in (a)–(e), respectively; (p)–(t) zoomed-in views of red rectangular regions marked in (a)–(e), respectively.

    Moreover, peak signal-to-noise ratio (PSNR) and Fréchet inception distance (FID) were calculated for the quantitative analysis of CE-LED images generated by cycleGAN [Fig. 3(b)], pix2pix [Fig. 3(c)], and U-Frame [Fig. 3(d)]. Table 1 shows that the PSNR values, calculated from the CE-LED image generated by pix2pix and U-Frame, are close to that from the UV-LS image. However, in terms of the FID value, the one from U-Frame has a much lower value compared to the ones generated by cycleGAN and pix2pix, indicating that U-Frame attains the best performance in generating images with features similar to the target UV-LS.

    • Table 1. PSNR and FID Comparison among the CE-LED Images Generated by CycleGAN, Pix2pix, and U-Frame

      Table 1. PSNR and FID Comparison among the CE-LED Images Generated by CycleGAN, Pix2pix, and U-Frame

      Comparison among the CE-LED Results Generated by Different ModelsPSNRFID
      CE-LED by cycleGAN [Fig. 3(b)] to UV-LS [Fig. 3(e)]28.2588.38
      CE-LED by pix2pix [Fig. 3(c)] to UV-LS [Fig. 3(e)]28.5567.03
      CE-LED by U-Frame [Fig. 3(d)] to UV-LS [Fig. 3(e)]28.5029.90

    We then extended our evaluation of the performance of CE-LED images to human lung cancer tissues. Two adjacent sections with 0.5 mm in thickness were acquired separately for training and testing. Figure 4 illustrates the comparative imaging results of UV-LED [Fig. 4(a)], CE-LED [Fig. 4(b)], UV-LS [Fig. 4(c)], and corresponding H&E-stained images [Fig. 4(d)] of the same tissue. The regions marked in red reveal significant contrast improvements in CE-LED [Fig. 4(f)] compared to the original UV-LED image [Fig. 4(e)], mirroring the quality of the UV-LS images [Fig. 4(g)]. In both CE-LED and UV-LS, the negative nuclear contrast allows the observation of acinar structures that appear glandular-like features with enlarged malignant cells inside the oval-shaped glands, identical to the structure in the corresponding H&E-stained image. This feature could potentially aid in diagnosing chronic lung conditions such as idiopathic pulmonary fibrosis and other interstitial lung diseases[30]. Figures 4(i)4(l) outline a large cancerous area, with notable contrast enhancement of the CE-LED image compared to the original LED image to match the quality of the UV-LS image. Detailed zoomed-in views of the yellow dashed regions [Figs. 4(m)4(p)] show that CE-LED [Fig. 4(n)] significantly boosts the contrast of the UV-LED image [Fig. 4(m)], achieving image quality similar to the UV-LS image [Fig. 4(o)]. The CE-LED image reveals a detailed fibrous structure encircling a cavity, identical to the morphology of the corresponding H&E-stained image [Fig. 4(p)]. Additionally, the fibrous structures exhibiting strong autofluorescence are more distinctly visible in the autofluorescence images, which typically require Masson’s trichrome staining to visualize them during standard histopathological assessments, which can potentially further expand the applications of virtual special staining. Another set of zoomed-in views from the light green regions [Figs. 4(q)4(t)] display that both CE-LED [Fig. 4(r)] and UV-LS [Fig. 4(s)] images notably reveal enlarged cancer nuclei, comparable to the H&E-stained sections [Fig. 4(t)]. In contrast, the UV-LED image [Fig. 4(q)] lacks sufficient contrast for clear nuclear visualization. We further computed the PSNR and structural similarity (SSIM) of UV-LED [Fig. 4(i)] and CE-LED [Fig. 4(j)] to UV-LS [Fig. 4(k)] for comparison. The results are shown in Table 2. A PSNR of 27.49 is calculated from UV-LED to UV-LS, while there is an increase of PSNR (29.04) when comparing CE-LED to UV-LS. The SSIM is calculated accordingly, resulting in 0.72 versus 0.78 compared between UV-LED and CE-LED, both with UV-LS as the baseline (Table 2). Both PSNR and SSIM values support that the CE-LED image successfully mimics the UV-LS image quality, with a more identical appearance as UV-LS than that of the UV-LED image.

    Comparison among UV-LED, CE-LED, UV-LS, and the corresponding H&E-stained images on thick human lung cancer tissue. (a) UV-LED image of lung cancer tissue; (b) CE-LED image generated from the UV-LED image in (a); (c) UV-LS image of the same specimen in (a); (d) corresponding H&E-stained image after acquiring (a) and (c); (e)–(h) zoomed-in views of red rectangular regions marked in (a)–(d), respectively; (i)–(l) zoomed-in views of blue dashed rectangular regions marked in (a)–(d), respectively; (m)–(p) and (q)–(t) are the further zoom-in images highlighted by the yellow dashed rectangular regions and the light-green rectangular regions marked in (i)–(l), respectively.

    Figure 4.Comparison among UV-LED, CE-LED, UV-LS, and the corresponding H&E-stained images on thick human lung cancer tissue. (a) UV-LED image of lung cancer tissue; (b) CE-LED image generated from the UV-LED image in (a); (c) UV-LS image of the same specimen in (a); (d) corresponding H&E-stained image after acquiring (a) and (c); (e)–(h) zoomed-in views of red rectangular regions marked in (a)–(d), respectively; (i)–(l) zoomed-in views of blue dashed rectangular regions marked in (a)–(d), respectively; (m)–(p) and (q)–(t) are the further zoom-in images highlighted by the yellow dashed rectangular regions and the light-green rectangular regions marked in (i)–(l), respectively.

    • Table 2. PSNR and SSIM Comparison between UV-LED to UV-LS, and CE-LED to UV-LS in Human Lung Cancer Tissue

      Table 2. PSNR and SSIM Comparison between UV-LED to UV-LS, and CE-LED to UV-LS in Human Lung Cancer Tissue

      Human Lung Cancer Tissue ImagePSNRSSIM
      UV-LED [Fig. 4(i)] to UV-LS [Fig. 4(k)]27.490.72
      CE-LED [Fig. 4(j)] to UV-LS [Fig. 4(k)]29.040.78

    To evaluate the application of virtual staining on the gray-scale CE-LED images to eliminate the habitual barrier for pathological examination, we use the U-Frame network to learn the style transformation from the CE-LED image to the corresponding H&E-stained image, and then inference on the CE-LED [Fig. 5(a)] image acquired from its adjacent tissue section to get the virtual H&E-stained image [Fig. 5(b)]. The zoomed-in virtual staining result [Fig. 5(e)] shows it can correctly stain the fibrous structure (indicated by orange arrows) that appears bright in the CE-LED image [Fig. 5(d)] of the same region. More importantly, as individual cells can be observed in the CE-LED, the virtually stained image [Fig. 5(e)] clearly shows the structure of the acinar (indicated by green arrows) that is identical to the corresponding adjacent H&E-stained image [Fig. 5(c)] which shows cuboidal or columnar epithelial cells in gland-like acinar patterns. We also trained the virtual staining model on the original UV-LED image to get the inference result for comparison. Figure 5(f) is the original UV-LED image of Fig. 5(d), which has a lower contrast comparatively. Although it is difficult to identify individual cells in the UV-LED image, virtual staining algorithms based on the U-Frame network show an impressive ability to reveal tissue structure. The virtual stained image from UV-LED [Fig. 5(g)] reveals the fiber structure successfully (indicated by orange arrows) and shows some regions with acinar structures similar to that in the corresponding H&E-stained image [Fig. 5(c)]. However, the blurry UV-LED image also affects the virtual staining results by creating histological structures with cancer cells aggregated together (indicated by blue arrows) without forming a gland-like structure, affecting the number of cell counts and diagnostic accuracy.

    Comparison of the virtual staining results of CE-LED and UV-LED. (a) CE-LED image of lung cancer tissue; (b) corresponding virtual staining result of (a) generated by the U-Frame virtual staining model; (c) corresponding H&E-stained image of the yellow rectangular region marked in (a); (d)–(e) zoomed-in views of the yellow rectangular regions marked in (a) and (b), respectively; (f) original UV-LED image of (d); (g) virtual staining result of (f).

    Figure 5.Comparison of the virtual staining results of CE-LED and UV-LED. (a) CE-LED image of lung cancer tissue; (b) corresponding virtual staining result of (a) generated by the U-Frame virtual staining model; (c) corresponding H&E-stained image of the yellow rectangular region marked in (a); (d)–(e) zoomed-in views of the yellow rectangular regions marked in (a) and (b), respectively; (f) original UV-LED image of (d); (g) virtual staining result of (f).

    4. Discussion

    In our study, we introduced a contrast-enhanced UV-LED system with a deep-learning algorithm to target the image quality of the UV-LS system. This method can be used to increase image quality from one modality to another as a learning target. Here, UV-LS is used instead of other UV laser-based autofluorescence imaging techniques, such as CHAMP[20], because it provides comparatively higher axial and lateral resolutions with direct image acquisition and robust reconstruction, resulting in stable image quality for potential large-scale imaging acquisition to set up high-quality training database for contrast enhancement. The image acquisition time for UV-LS is longer than that of the ordinary fluorescence microscope, since the autofluorescence signal requires a longer exposure time to have enough signal-to-noise ratio, thus limiting the direct intraoperative application of the UV-LS system.

    Although the results show that the image quality is capable of revealing tissue structures, histopathology usually has high requirements in magnification. High numerical aperture (NA) objectives can be used for both the UV-LED and UV-LS systems, along with additional image processing or system adjustment to achieve submicrometer resolutions. In the UV-LED system, high NA in the optical collection path may lead to out-of-focus images, since the depth of field may exceed the range of tissue roughness. To generate high-resolution and in-focus images, 3D capture followed by an extended depth-of-field algorithm[31] is a possible solution that merges all particularly focused images taken at different focus planes into an in-focus 2D image.

    Furthermore, besides image quality improvement, our demonstrated dual-modality system with a U-Frame network can potentially be applied to enable additional information extraction, for example, transforming autofluorescence images into fluorescence images. Autofluorescence imaging contains rich endogenous fluorophores, such as NADH and flavin, that link to the state of tissue and cell pathological changes, and may serve as natural biomarkers[22]. Leveraging fluorescence light-sheet microscopy with a corresponding biomarker, and subsequently applying deep-learning algorithms to transform the label-free autofluorescence image to the corresponding biomarker-specific image may help to reveal information correlation with high specificity.

    In terms of the deep-learning network for style transformation, the U-Frame network has no strict requirement in pixel-level alignment like supervised learning while keeping the learning ability at the region level to take advantage of the morphology similarity of the two image domains (i.e., UV-LED and UV-LS). Comparative analyses among U-Frame, pix2pix, and cycleGAN networks show that U-Frame generally outperforms both pix2pix and cycleGAN in achieving image quality closest to the target domain while preserving the tissue and cell morphology. In some regions, deep-learning algorithms tend to be oversensitive to the intensity variation of UV-LED images, transforming cellular features into fibrous structures by mistakes. However, given the promising results with only limited data, we believe that by increasing the data size, U-Frame would learn different autofluorescence features with different intensities for more reliable contrast enhancement outcomes. The virtual staining results from CE-LED are highly encouraging and show great promise for clinical applications. The comparison of the virtual staining results from the original UV-LED and CE-LED shows the advantage of using a contrast-enhancement algorithm as a bridge for more accurate virtual staining outcomes.

    5. Conclusion

    We successfully demonstrated a deep-learning-assisted UV-LED system that approximates the image quality of UV-LS for cost-effective and rapid histological imaging without the need for chemical staining and the use of expensive lasers. The imaging speed of our generated CE-LED is 47  s/cm2, showing a significant improvement over the 20  min/cm2 required by traditional UV-LS, enhancing its suitability for intraoperative cancer detection. Our dual-modality system, integrating UV-LED and UV-LS, minimizes tissue distortion and enhances tissue structure similarity to train the contrast enhancement network. With the deep-learning outcome, our results indicate that CE-LED significantly enhances the contrast of the original UV-LED image, achieving improvements in both PSNR and SSIM in histological imaging (e.g., for human lung tissues). The U-Frame network, a weakly supervised method, outperforms both pix2pix and cycleGAN in all our tests due to the absence of perfect data alignment while presenting high morphology similarity between the source and target domains. Additionally, we demonstrated the application of virtual staining with the U-Frame network to facilitate the clinical application of CE-LED images. It is promising that our proposed deep-learning-assisted UV-LED system offers a high-throughput, high-resolution, nondestructive, and economical alternative for intraoperative tissue imaging, facilitating rapid histological imaging that could greatly assist in the current clinical pathology examination workflow.

    [26] J. Zhu et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE International Conference on Computer Vision (ICCV)(2017).

    [31] B. Forster-Heinlein et al. Extended depth-of-focus for multi-channel microscopy images: a complex wavelet approach(2004).

    Tools

    Get Citation

    Copy Citation Text

    Jiajie Wu, Weixing Dai, Claudia T. K. Lo, Lauren W. K. Tsui, Terence T. W. Wong, "High-throughput, nondestructive, and low-cost histological imaging with deep-learning-assisted UV microscopy," Adv. Imaging 1, 021001 (2024)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Article

    Received: May. 22, 2024

    Accepted: Jul. 1, 2024

    Published Online: Aug. 8, 2024

    The Author Email: Wong Terence T. W. (ttwwong@ust.hk)

    DOI:10.3788/AI.2024.10007

    Topics