Advanced Photonics, Volume. 6, Issue 6, (2024)
Cross-modality transformations in biological microscopy enabled by deep learning [Early Posting]
Recent advancements in deep learning have propelled the virtual transformation of microscopy images across optical modalities, enabling unprecedented multi-modal imaging analysis hitherto impossible. Despite these strides, the integration of such algorithms into scientists’ daily routines and clinical trials remains limited, largely due to a lack of recognition within the irrespective fields and the plethora of available transformation methods. To address this, we present a structured overview of cross-modality transformations, encompassing applications, datasets and implementations, aimed at unifying this evolving field. Our review focuses on deep learning solutions for two key applications: contrast enhancement of targeted features within images and resolution enhancements. We identify cross-modality transformations as a valuable asset for biologists. Notably, they enable high-contrast, high-specificity imaging akin to fluorescence microscopy without the need for laborious, costly, and disruptive physical staining pro- cedures. Additionally, they facilitate the realisation of imaging with properties that would typically require costly or complex physical modifications, such as achieving super-resolution capabilities. By consolidating the current state of research in this review, we aim to catalyse further investigation and development, ultimately bringing the potential of cross-modality transformations into the hands of researchers and clinicians alike.