Researching | Advances in Deep Learning-Based Virtual Staining of Pathological Tissues

Laser & Optoelectronics Progress, Volume. 62, Issue 14, 1400001(2025)

Advances in Deep Learning-Based Virtual Staining of Pathological Tissues

Junhong Huang, Tingdong Kou, Tianyue He, Cui Huang, Chaoqiang Wu, and Junfei Shen^*

Author Affiliations

D Sensing and Machine Vision Laboratory, College of Electronics and Information Engineering, Sichuan University, Chengdu 610065, Sichuan , China

show less

Abstract Get PDF(in Chinese)

Figures&Tables (16)

References (111)

Paper Information

References(111)

[1] Alturkistani H A, Tashkandi F M, Mohammedsaleh Z M. Histological stains: a literature review and case study[J]. Global Journal of Health Science, 8, 72(2015).

[2] Titford M. The long history of hematoxylin[J]. Biotechnic & Histochemistry, 80, 73-78(2005).

[3] Foot N C. The Masson trichrome staining methods in routine laboratory use[J]. Stain Technology, 8, 101-110(1933).

[4] Pollitt C C. Basement membrane pathology: a feature of acute equine laminitis[J]. Equine Veterinary Journal, 28, 38-46(1996).

[5] Ramos-Vara J A, Miller M A. When tissue antigens and antibodies get along: revisiting the technical aspects of immunohistochemistry: the red, brown, and blue technique[J]. Veterinary Pathology, 51, 42-87(2014).

[6] Liu C H, Fu L W, Chen H H et al. Toward cell nuclei precision between OCT and H&E images translation using signal-to-noise ratio cycle-consistency[J]. Computer Methods and Programs in Biomedicine, 242, 107824(2023).

[7] Rivenson Y, Wang H D, Wei Z S et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning[J]. Nature Biomedical Engineering, 3, 466-477(2019).

[8] Alajaji S A, Khoury Z H, Elgharib M et al. Generative adversarial networks in digital histopathology: current applications, limitations, ethical considerations, and future directions[J]. Modern Pathology, 37, 100369(2024).

[9] Goodfellow I, Pouget-Abadie J, Mirza M et al. Generative adversarial networks[J]. Communications of the ACM, 63, 139-144(2020).

[10] de Haan K, Zhang Y J, Zuckerman J E et al. Deep learning-based transformation of H&E stained tissues into special stains[J]. Nature Communications, 12, 4884(2021).

[11] Naglah A, Khalifa F, El-Baz A et al. Conditional GANs based system for fibrosis detection and quantification in Hematoxylin and Eosin whole slide images[J]. Medical Image Analysis, 81, 102537(2022).

[12] Yan R N, He Q M, Liu Y Q et al. Unpaired virtual histological staining using prior-guided generative adversarial networks[J]. Computerized Medical Imaging and Graphics, 105, 102185(2023).

[13] Biswas T, Suzuki H, Ishikawa M et al. Generative adversarial network based digital stain conversion for generating RGB EVG stained image from hyperspectral H&E stained image[J]. Journal of Biomedical Optics, 28, 056501(2023).

[14] Levy J J, Jackson C R, Sriharan A et al. Preliminary evaluation of the utility of deep generative histopathology image translation at a mid-sized NCI cancer center[EB/OL]. https://www.biorxiv.org/content/10.1101/2020.01.07.897801v2

[15] Teramoto A, Yamada A, Tsukamoto T et al. Mutual stain conversion between Giemsa and Papanicolaou in cytological images using cycle generative adversarial network[J]. Heliyon, 7, e06331(2021).

[16] Guan X C, Wang Y F, Lin Y Y et al. Unsupervised multi-domain progressive stain transfer guided by style encoding dictionary[J]. IEEE Transactions on Image Processing, 33, 767-779(2024).

[17] Yang X L, Bai B J, Zhang Y J et al. Virtual stain transfer in histology via cascaded deep neural networks[J]. ACS Photonics, 9, 3134-3143(2022).

[18] Zhang R R, Cao Y K, Li Y J et al. MVFStain: Multiple virtual functional stain histopathology images generation based on specific domain mapping[J]. Medical Image Analysis, 80, 102520(2022).

[19] Liu S J, Zhu C, Xu F et al. BCI: breast cancer immunohistochemical image generation through pyramid Pix2Pix[C], 1814-1823(2022).

[20] Xie W S, Reder N P, Koyuncu C et al. Prostate cancer risk stratification via nondestructive 3D pathology with deep learning-assisted gland analysis[J]. Cancer Research, 82, 334-345(2022).

[21] Hong Y Y, Heo Y J, Kim B et al. Deep learning-based virtual cytokeratin staining of gastric carcinomas to measure tumor-stroma ratio[J]. Scientific Reports, 11, 19255(2021).

[22] Cetin O, Chen M Z, Ziegler P et al. Deep learning-based restaining of histopathological images[C], 1467-1474(2022).

[23] Dubey S, Kataria T, Knudsen B et al. Structural CycleGAN for virtual immunohistochemistry staining of gland markers in the colon[M]. Machine learning in medical imaging, 14349, 447-456(2023).

[24] Lin Y Y, Wang Y F, Chen Y et al. Unpaired H&E to PR stain transfer with self-supervised auxiliary segmentation[C], 1-5(2023).

[25] Mercan C, Mooij G C A M, Tellez D et al. Virtual staining for mitosis detection in breast histopathology[C], 1770-1774(2020).

[26] Jackson C R, Sriharan A, Vaickus L J. A machine learning algorithm for simulating immunohistochemistry: development of SOX10 virtual IHC and evaluation on primarily melanocytic neoplasms[J]. Modern Pathology, 33, 1638-1648(2020).

[27] Trullo R, Bui Q A, Tang Q et al. Image translation based nuclei segmentation for immunohistochemistry images[M]. Deep generative models, 13609, 87-96(2022).

[28] Ghahremani P, Li Y Y, Kaufman A et al. Deep learning-inferred multiplex immunofluorescence for immunohistochemical image quantification[J]. Nature Machine Intelligence, 4, 401-412(2022).

[29] Wölflein G, Um I H, Harrison D J et al. HoechstGAN: virtual lymphocyte staining using generative adversarial networks[C], 4986-4996(2023).

[30] Burlingame E A, McDonnell M, Schau G F et al. SHIFT: speedy histological-to-immunofluorescent translation of a tumor signature enabled by deep learning[J]. Scientific Reports, 10, 17507(2020).

[31] Lin Y Y, Zeng B W, Wang Y F et al. Unpaired multi-domain stain transfer for kidney histopathological images[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 1630-1637(2022).

[32] Stenman S, Bychkov D, Kucukel H et al. Antibody supervised training of a deep learning based algorithm for leukocyte segmentation in papillary thyroid carcinoma[J]. IEEE Journal of Biomedical and Health Informatics, 25, 422-428(2021).

[33] Huang K, Cheng Y F, Gao Q et al. Weakly unpaired image translation from hematoxylin and eosin staining image to immunohistochemistry staining image[C], 1013-1019(2022).

[34] Ma Y H, Zhang Y Q, Wang Z R et al. DSFF-GAN: a novel stain transfer network for generating immunohistochemical image of endometrial cancer[J]. Computers in Biology and Medicine, 170, 108046(2024).

[35] Zhao W, Qi B Z, Li Y C et al. Clinical trial histology image based end-to-end biomarker expression levels prediction and visualization using constrained GANs[M]. Applications of medical artificial intelligence, 14313, 1-10(2023).

[36] Bouteldja N, Klinkhammer B M, Schlaich T et al. Improving unsupervised stain-to-stain translation using self-supervision and meta-learning[J]. Journal of Pathology Informatics, 13, 100107(2022).

[37] Zhou Z X, Jiang Y M, Li R J et al. Ultrafast labeling for multiplexed immunobiomarkers from label-free fluorescent images[M]. Applications of medical artificial intelligence, 14313, 125-134(2023).

[38] Chen Z H, Yu W T, Wong I H M et al. Deep-learning-assisted microscopy with ultraviolet surface excitation for rapid slide-free histological imaging[J]. Biomedical Optics Express, 12, 5920-5938(2021).

[39] Jin L B, Tang Y B, Coole J B et al. DeepDOF-SE: affordable deep-learning microscopy platform for slide-free histology[J]. Nature Communications, 15, 2935(2024).

[40] Shi L L, Wong I H M, Lo C T K et al. One-side virtual histological staining model for complex human samples[C], 1-4(2022).

[41] Zhang Y J, de Haan K, Rivenson Y et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue[EB/OL]. https://arxiv.org/abs/2001.07267v1

[42] Bai B J, Wang H D, Li Y Z et al. Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning[J]. BME Frontiers, 2022, 9786242(2022).

[43] Li Y, Pillar N, Li J et al. Virtual histological staining of unlabeled autopsy tissue[J]. Nature Communications, 15, 1684(2024).

[44] Shen B L, Li Z L, Pan Y et al. Noninvasive nonlinear optical computational histology[J]. Advanced Science, 11, 2308630(2024).

[45] Borhani N, Bower A J, Boppart S A et al. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy[J]. Biomedical Optics Express, 10, 1339-1350(2019).

[46] Cao R, Nelson S D, Davis S et al. Label-free intraoperative histology of bone tissue via deep-learning-assisted ultraviolet photoacoustic microscopy[J]. Nature Biomedical Engineering, 7, 124-134(2023).

[47] Kang L, Li X F, Zhang Y et al. Deep learning enables ultraviolet photoacoustic microscopy based histological imaging with near real-time virtual staining[J]. Photoacoustics, 25, 100308(2022).

[48] Martell M T, Haven N J M, Cikaluk B D et al. Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy[J]. Nature Communications, 14, 5967(2023).

[49] Restall B S, Haven N J M, Kedarisetti P et al. Virtual hematoxylin and eosin histopathology using simultaneous photoacoustic remote sensing and scattering microscopy[J]. Optics Express, 29, 13864-13875(2021).

[50] Boktor M, Ecclestone B R, Pekar V et al. Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS)[J]. Scientific Reports, 12, 10296(2022).

[51] Fereidouni F, Harmany Z T, Tian M et al. Microscopy with ultraviolet surface excitation for rapid slide-free histology[J]. Nature Biomedical Engineering, 1, 957-966(2017).

[52] Kreiss L, Jiang S W, Li X et al. Digital staining in optical microscopy using deep learning- a review[J]. PhotoniX, 4, 34(2023).

[53] Meng X Y, Li X, Wang X. A computationally virtual histological staining method to ovarian cancer tissue by deep generative adversarial networks[J]. Computational and Mathematical Methods in Medicine, 2021, 4244157(2021).

[54] Wong I H M, Chen Z H, Shi L L et al. Deep learning-assisted low-cost autofluorescence microscopy for rapid slide-free imaging with virtual histological staining[J]. Biomedical Optics Express, 15, 2187-2201(2024).

[55] Chen Z H, Wong I H M, Dai W X et al. Lung cancer diagnosis on virtual histologically stained tissue using weakly supervised learning[J]. Modern Pathology, 37, 100487(2024).

[56] Shi L L, Zhang Y, Wong I H M et al. MulHiST: multiple histological staining for thick biological samples via unsupervised image-to-image translation[M]. Medical image computing and computer assisted intervention-MICCAI 2023, 14225, 735-744(2023).

[57] Kedarisetti P, Restall B S, Haven N J M et al. F-mode ultraviolet photoacoustic remote sensing for label-free virtual H&E histopathology using a single excitation wavelength[J]. Optics Letters, 46, 3500-3503(2021).

[58] Boktor M, Tweel J E D, Ecclestone B R et al. Multi-channel feature extraction for virtual histological staining of photon absorption remote sensing images[J]. Scientific Reports, 14, 2009(2024).

[59] Li D, Hui H, Zhang Y Q et al. Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue[J]. Molecular Imaging and Biology, 22, 1301-1309(2020).

[60] Asaf M Z, Rao B, Akram M U et al. Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images[J]. Scientific Reports, 14, 2335(2024).

[61] Li J, Garfinkel J, Zhang X et al. Biopsy-free in vivo virtual histology of skin using deep learning[J]. Light: Science & Applications, 10, 233(2021).

[62] Mayerich D, Walsh M J, Kadjacsy-Balla A et al. Stain-less staining for computed histopathology[J]. Technology, 3, 27-31(2015).

[63] Soltani S, Ojaghi A, Qiao H et al. Prostate cancer histopathology using label-free multispectral deep-UV microscopy quantifies phenotypes of tumor aggressiveness and enables multiple diagnostic virtual stains[J]. Scientific Reports, 12, 9329(2022).

[64] Abraham T M, Costa P C, Serafini C E et al. Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual hematoxylin and eosin staining[J]. Optica, 10, 1605-1618(2023).

[65] Rivenson Y, Liu T R, Wei Z S et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning[J]. Light: Science & Applications, 8, 23(2019).

[66] Min E, Aimakov N, Lee S et al. Multi-contrast digital histopathology of mouse organs using quantitative phase imaging and virtual staining[J]. Biomed Opt Express, 14, 2068-2079(2023).

[67] Zhang G H, Ning B, Hui H et al. Image-to-images translation for multiple virtual histological staining of unlabeled human carotid atherosclerotic tissue[J]. Molecular Imaging and Biology, 24, 31-41(2022).

[68] Khan U, Koivukoski S, Valkonen M et al. The effect of neural network architecture on virtual H&E staining: systematic assessment of histological feasibility[J]. Patterns, 4, 100725(2023).

[69] Ye S W, Zou J J, Huang C M et al. Rapid and label-free histological imaging of unprocessed surgical tissues via dark-field reflectance ultraviolet microscopy[J]. iScience, 26, 105849(2023).

[70] Li X S, Liu H S, Song X Y et al. Structural constrained virtual histology staining for human coronary imaging using deep learning[C], 1-5(2023).

[71] McNeil C, Wong P F, Sridhar N et al. An end-to-end platform for digital pathology using hyperspectral autofluorescence microscopy and deep learning-based virtual histology[J]. Modern Pathology, 37, 100377(2024).

[72] Wei S L, Si L, Huang T Y et al. Deep-learning-based cross-modality translation from Stokes image to bright-field contrast[J]. Journal of Biomedical Optics, 28, 102911(2023).

[73] Chae H, Kim J, Jeon J et al. Restoring H&E stain in faded slides via phase-to-color virtual staining in near-infrared[J]. APL Photonics, 9, 106112(2024).

[74] Winetraub Y, Van Vleck A, Yuan E et al. Noninvasive virtual biopsy using micro-registered optical coherence tomography (OCT) in human subjects[J]. Science Advances, 10, eadi5794.

[75] Dorota O M, Miroslaw P, Przemyslaw M. Virtual UV fluorescence microscopy from hematoxylin and eosin staining of liver images using deep learning convolutional neural network[J]. Applied Sciences, 10, 7815(2020).

[76] Vahadane A, Peng T Y, Sethi A et al. Structure-preserving color normalization and sparse stain separation for histological images[J]. IEEE Transactions on Medical Imaging, 35, 1962-1971(2016).

[77] Shen Y Q, Sowmya A, Luo Y L et al. A federated learning system for histopathology image analysis with an orchestral stain-normalization GAN[J]. IEEE Transactions on Medical Imaging, 42, 1969-1981(2023).

[78] Barua B, Bora K, Kr Das A et al. Stain color translation of multi-domain OSCC histopathology images using attention gated cGAN[J]. Computerized Medical Imaging and Graphics, 106, 102202(2023).

[79] Moghadam A Z, Azarnoush H, Seyyedsalehi S A et al. Stain transfer using Generative Adversarial Networks and disentangled features[J]. Computers in Biology and Medicine, 142, 105219(2022).

[80] Cong C, Liu S D, Ieva A D et al. Colour adaptive generative networks for stain normalisation of histopathology images[J]. Medical Image Analysis, 82, 102580(2022).

[81] Wang H D, Rivenson Y, Jin Y Y et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy[J]. Nature Methods, 16, 103-110(2019).

[82] Bay H, Ess A, Tuytelaars T et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 110, 346-359(2008).

[83] Lowe D G. Object recognition from local scale-invariant features[C], 1150-1157(1999).

[84] Fischler M A, Bolles R C. Random sample consensus[J]. Communications of the ACM, 24, 381-395(1981).

[85] Culley S, Albrecht D, Jacobs C et al. Quantitative mapping and minimization of super-resolution optical imaging artifacts[J]. Nature Methods, 15, 263-266(2018).

[86] Isola P, Zhu J Y, Zhou T H et al. Image-to-image translation with conditional adversarial networks[C], 5967-5976(2017).

[87] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[M]. Medical image computing and computer-assisted intervention-MICCAI 2015, 9351, 234-241(2015).

[88] Schlemper J, Oktay O, Schaap M et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 53, 197-207(2019).

[89] Klages P, Benslimane I, Riyahi S et al. Patch-based generative adversarial neural network models for head and neck MR-only planning[J]. Medical Physics, 47, 626-642(2020).

[90] Chen M K, Liu Y T, Khan F S et al. Single color digital H&E staining with In-and-Out Net[J]. Computerized Medical Imaging and Graphics, 118, 102468(2024).

[91] Rana A, Yauney G, Lowe A et al. Computational histological staining and destaining of prostate core biopsy RGB images with generative adversarial neural networks[C], 828-834(2018).

[92] Bayramoglu N, Kaakinen M, Eklund L et al. Towards virtual H&E staining of hyperspectral lung histology images using conditional generative adversarial networks[C], 64-71(2017).

[93] Salvi M, Michielli N, Salamone L et al. Computational synthesis of histological stains: a step toward virtual enhanced digital pathology[J]. International Journal of Imaging Systems and Technology, 34, e23165(2024).

[94] Zhu J Y, Park T, Isola P et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C], 2242-2251(2017).

[95] Shaban M T, Baur C, Navab N et al. Staingan: stain style transfer for digital histological images[C], 953-956(2019).

[96] Liu S T, Zhang B C, Liu Y Q et al. Unpaired stain transfer using pathology-consistent constrained generative adversarial networks[J]. IEEE Transactions on Medical Imaging, 40, 1977-1989(2021).

[97] Li X Y, Zhang G X, Qiao H et al. Unsupervised content-preserving transformation for optical microscopy[J]. Light: Science & Applications, 10, 44(2021).

[98] Liu Z J, Chen L C, Cheng H X et al. Virtual formalin-fixed and paraffin-embedded staining of fresh brain tissue via stimulated Raman CycleGAN model[J]. Science Advances, 10, eadn3426(2024).

[99] Ling Y, Tan W M, Yan B. Self-supervised digital histopathology image disentanglement for arbitrary domain stain transfer[J]. IEEE Transactions on Medical Imaging, 42, 3625-3638(2023).

[100] Rana A M, Lowe A, Lithgow M et al. Use of deep learning to develop and analyze computational hematoxylin and eosin staining of prostate core biopsy images for tumor diagnosis[J]. JAMA Network Open, 3, e205111(2020).

[101] Salehi P, Chalechale A. Pix2Pix-based stain-to-stain translation: a solution for robust stain normalization in histopathology images analysis[C], 1-7(2020).

[102] Shi L L, Wong I H M, Lo C T K et al. Unsupervised multiple virtual histological staining from label-free autofluorescence images[C], 1-5(2023).

[103] Tweel J E D, Ecclestone B R, Gaouda H et al. Photon absorption remote sensing imaging of breast needle core biopsies is diagnostically equivalent to gold standard H&E histologic assessment[J]. Current Oncology, 30, 9760-9771(2023).

[104] Zhang G H, Hui H, Ning B et al. Self-attention based virtual staining for bright-field images of label-free human carotid atherosclerotic plaque tissue section[C], 3492-3495(2021).

[105] Koivukoski S, Khan U, Ruusuvuori P et al. Unstained tissue imaging and virtual hematoxylin and eosin staining of histologic whole slide images[J]. Laboratory Investigation, 103, 100070(2023).

[106] Li X S, Liu H S, Song X Y et al. Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images[J]. Journal of Biomedical Optics, 29, 036004(2024).

[107] Nielsen P S, Georgsen J B, Vinding M S et al. Computer-assisted annotation of digital H&E/SOX10 dual stains generates high-performing convolutional neural network for calculating tumor burden in H&E-stained cutaneous melanoma[J]. International Journal of Environmental Research and Public Health, 19, 14327(2022).

[108] Tellez D, Balkenhol M, Otte-Höller I et al. Whole-slide mitosis detection in H&E breast histology using PHH3 as a reference to train distilled stain-invariant convolutional networks[J]. IEEE Transactions on Medical Imaging, 37, 2126-2136(2018).

[109] Stirling D R, Swain-Bowden M J, Lucas A M et al. CellProfiler 4: improvements in speed, utility and usability[J]. BMC Bioinformatics, 22, 433(2021).

[110] Sloboda T, Hudec L, Halinkovič M et al. Attention-enhanced unpaired xAI-GANs for transformation of histological stain images[J]. Journal of Imaging, 10, 32(2024).

[111] Landis J R, Koch G G. The measurement of observer agreement for categorical data[J]. Biometrics, 33, 159(1977).

Tools

Get Citation

Copy Citation Text

Junhong Huang, Tingdong Kou, Tianyue He, Cui Huang, Chaoqiang Wu, Junfei Shen. Advances in Deep Learning-Based Virtual Staining of Pathological Tissues[J]. Laser & Optoelectronics Progress, 2025, 62(14): 1400001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Reviews

Received: Nov. 19, 2024

Accepted: Mar. 13, 2025

Published Online: Jul. 16, 2025

The Author Email: Junfei Shen (shenjunfei@scu.edu.cn)

DOI:10.3788/LOP242293

CSTR:32186.14.LOP242293

Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology