Advanced Photonics, Volume. 6, Issue 6, 064001(2024)
Cross-modality transformations in biological microscopy enabled by deep learning
[1] D. Murphy, M. Davidson. Fundamentals of Light Microscopy and Electronic Imaging(2012).
[2] K. Suvarna, C. Layton, J. Bancroft. Bancroft’s Theory and Practice of Histological Techniques E-Book(2012).
[4] I. Johnson. Molecular probes handbook: a guide to fluorescent probes and labeling technologies(2010).
[6] S. Shashkova, M. Leake. Single-molecule fluorescence microscopy review: shedding new light on old problems. Biosci. Rep., 37, BSR20170031(2017).
[11] J. Ferreira, L. Groc. Surface Glutamate Receptor Nanoscale Organization with Super-Resolution Microscopy (dSTORM), 35-52(2024).
[17] X. Cao et al. Deep learning based inter-modality image registration supervised by intra-modality similarity. Lecture Notes in Computer Science, 55-63(2018).
[19] J. Johnson, A. Alahi, L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. Lecture Notes in Computer Science, 694-711(2016).
[20] C. Ledig et al. Photo-realistic single image super-resolution using a generative adversarial network, 15640-15649(2017).
[21] K. Zhang et al. Negative-aware attention framework for image-text matching(2022).
[22] T. S. Gurina, L. Simms. Histology, staining(2023).
[25] B. Bai et al. Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning. BME Frontiers, 2022, 9786242(2022).
[29] X. Wang et al. Single-shot isotropic differential interference contrast microscopy. Nat. Commun., 14, 2063(2023).
[34] E. Breznik et al. Cross-modality sub-image retrieval using contrastive multimodal image representations. Sci. Rep., 14, 18798(2024).
[35] A. Lahiani et al. Virtualization of tissue staining in digital pathology using an unsupervised deep learning approach. Lecture Notes in Comput. Sci., 47-55(2019).
[48] Y. Hong et al. Deep learning-based virtual cytokeratin staining of gastric carcinomas to measure tumor–stroma ratio. Sci. Rep., 11, 19255(2021).
[50] J. Y. Zhu et al. Unpaired image-to-image translation using cycle-consistent adversarial networks, 2242-2251(2017).
[55] X. Li et al. Unsupervised content-preserving transformation for optical microscopy. Light Sci. Appl., 10, 44(2021).
[61] R. Sanyal, D. Kar, R. Sarkar. Carcinoma type classification from high-resolution breast microscopy images using a hybrid ensemble of deep convolutional features and gradient boosting trees classifiers. IEEE/ACM Trans. Comput. Biol. Bioinf., 19, 2124-2136(2021).
[65] Q. Dou et al. Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss, 691-697(2018).
[72] M. Dohmen et al. Similarity metrics for MR image-to-image translation(2024).
[76] M. Luella, B. Paul, A. Javad. Generative AI in medical imaging and its application in low dose computed tomography (CT) image denoising. Applications of Generative AI, 387-401(2024).
[77] Y. Choi et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation, 8789-8797(2018).
[78] J. Vasiljević et al. HistostarGAN: a unified approach to stain normalisation, stain transfer and stain invariant segmentation in renal histopathology. Knowl.-Based Syst., 277, 110780(2023).
[82] S. Banerji, S. Mitra. Deep learning in histopathology: a review. WIREs Data Min. Knowl. Discovery, 12, e1439(2022).
[83] J. Xu et al. Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers, 73-95(2017).
[89] P. A. Moghadam et al. A morphology focused diffusion probabilistic model for synthesis of histopathology images, 1999-2008(2023).
[90] A. Greenspan, Y. Shen, J. Keet?al.. StainDiff: transfer stain styles of histology images with denoising diffusion probabilistic models and self-ensemble. Med. Image Computing and Computer Assisted Intervention, 549-559(2023).
[91] T. Kataria, B. Knudsen, S. Y. Elhabian. StainDiffuser: multitask dual diffusion model for virtual staining(2024).
[92] S. Dubey, X. Xu et al. VIMS: virtual immunohistochemistry multiplex staining via text-to-stain diffusion trained on uniplex stains. Mach. Learn. Med. Imaging, 143-155(2024).
[93] T. M. Abraham, R. Levenson. A comparison of diffusion models and CycleGANs for virtual staining of slide-free microscopy images, 1-6(2023).
[108] C. L. Cooke et al. Physics-enhanced machine learning for virtual fluorescence microscopy, 3803-3813(2021).
[119] E. Wolf. Progress in Optics(2008).
[121] C. Ledig et al. Photo-realistic single image super-resolution using a generative adversarial network, 105-114(2016).
[123] C. Chen et al. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation, 865-872(2019).
[124] D. Maleki, H. Tizhoosh. LILE: look in-depth before looking elsewhere: a dual attention network using transformers for cross-modal information retrieval in histopathology archives(2022).
[126] R. Naseem et al. Cross modality guided liver image enhancement of CT using MRI, 46-51(2019).
[127] C. Dong, D. Fleet et al. Learning a deep convolutional network for image super-resolution, 184-199(2014).
[133] E. Kang, J. Min, J. C. Ye. A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Med. Phys., 44, e360-e375(2017).
[134] C. Dong, J. M. Leibe, C. C. Loy, X. Tang et al. Accelerating the super-resolution convolutional neural network. Computer Vision–ECCV 2016, 391-407(2016).
[139] M. Haris, G. Shakhnarovich, N. Ukita. Deep back-projection networks for super-resolution, 1664-1673(2018).
[144] J.-Y. Lin, Y.-C. Chang, W. H. Hsu. Efficient and phase-aware video super-resolution for cardiac MRI, 66-76(2020).
[145] Y. Huang, L. Shao, A. F. Frangi. Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding, 5787-5796(2017).
[146] W.-S. Lai et al. Deep Laplacian pyramid networks for fast and accurate super-resolution, 624-632(2017).
[151] N. Boyd et al. DeepLOCO: fast 3D localization microscopy using neural networks. bioRxiv(2018).
[163] J. Soh, S. Cho, N. Cho. Meta-transfer learning for zero-shot super-resolution, 3513-3522(2020).
[165] H. Sahak et al. Denoising diffusion probabilistic models for robust image super-resolution in the wild(2023).
[166] S. Gao et al. Implicit diffusion models for continuous super-resolution, 10021-10030(2023).
[167] J. Ho et al. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23, 1-33(2021).
[168] R. Rombach et al. High-resolution image synthesis with latent diffusion models, 10674-10685(2021).
[169] A. Saguy et al. This microtubule does not exist: super-resolution microscopy image generation by a diffusion model, 2400672(2024).
[170] H. Greenspan, M. Pan et al. DiffuseIR: diffusion models for isotropic reconstruction of 3D microscopic images, 323-332(2023).
[171] S. Gao et al. GDPR Requirements for Biobanking Activities Across Europe, 10021-10030(2023).
[172] V. Colcelli et al. GDPR requirements for biobanking activities across europe(2023).
[173] Health insurance portability and accountability act of 1996 (HIPAA), 104-191(1996).
[174] (2018).
[176] D. Dhingra, A. Dabas. Global strategy on digital health. Indian Pediatrics, 57, 356-358(2020).
[177] The protection of personal data in health information systems–principles and processes for public health(2020).
[180] A. S. Pillai. Utilizing deep learning in medical image analysis for enhanced diagnostic accuracy and patient care: challenges, opportunities, and ethical implications. J. Deep Learn. Genomic Data Anal., 1, 1-17(2021).
[182] N. Forgó et al. Big data, AI and health data: between national, european, and international legal frameworks. Legal Challenges in the New Digital Age, 358-394(2023).
[186] K. Grünberg et al. Ethical and privacy aspects of using medical image data. Cloud-Based Benchmarking of Medical Image Analysis, 33-43(2017).
Get Citation
Copy Citation Text
Dana Hassan, Jesús Domínguez, Benjamin Midtvedt, Henrik Klein Moberg, Jesús Pineda, Christoph Langhammer, Giovanni Volpe, Antoni Homs Corbera, Caroline B. Adiels, "Cross-modality transformations in biological microscopy enabled by deep learning," Adv. Photon. 6, 064001 (2024)
Category: Reviews
Received: Jun. 18, 2024
Accepted: Oct. 28, 2024
Posted: Oct. 28, 2024
Published Online: Nov. 29, 2024
The Author Email: Caroline B. Adiels (caroline.adiels@physics.gu.se)