Journal of Innovative Optical Health Sciences, Volume. 18, Issue 1, 2450019(2025)

Streamlined photoacoustic image processing with foundation models: A training-free solution

Handi Deng1,2,3、§, Yucheng Zhou4、§, Jiaxuan Xiang5, Liujie Gu1,2,3, Yan Luo1, Hai Feng6, Mingyuan Liu6、*, and Cheng Ma1,2,3、**
Author Affiliations
  • 1Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 2Institute for Precision Healthcare, Tsinghua University, 77 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 3Institute for Intelligent Healthcare, Tsinghua University, 77 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 4School of Biological Science and Medical Engineering, Beihang University, 37 XueYuan Road, Haidian, Beijing 100191, P. R. China
  • 5TsingPAI Technology Co., Ltd., 27 Jiancaicheng Middle Road, Haidian, Beijing 100096, P. R. China
  • 6Department of Vascular Surgery, Beijing Friendship Hospital, Capital Medical University, 95 Yongan Road, Haidian, Beijing 100050, P. R. China
  • show less
    References(29)

    [1] R. Bommasani et al. On the opportunities and risks of foundation models(2021).

    [2] C. Zhou et al. A comprehensive survey on pretrained foundation models: A history from BERT to ChatGPT(2023).

    [3] T. Wu et al. A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA J. Autom. Sin., 10, 1122-1136(2023).

    [4] S. S. Biswas. Role of Chat GPT in public health. Ann. Biomed. Eng., 51, 868-869(2023).

    [5] M. Awais et al. Foundational models defining a new era in vision: A survey and outlook(2023).

    [6] A. Dosovitskiy et al. An image is worth 16×16 words: Transformers for image recognition at scale(2020).

    [7] A. Radford et al. Learning transferable visual models from natural language supervision. Proc. 38th Int. Conf. Machine Learning, 139, 1-16(2021).

    [8] A. Kirillov et al. Segment anything. Proc. 2023 IEEE/CVF Int. Conf. Computer Vision(2023).

    [9] J. Wang et al. Review of large vision models and visual prompt engineering. Meta-Radiol., 1, 100047(2023).

    [10] T. D. Le, S.-Y. Kwon, C. Lee. Segmentation and quantitative analysis of photoacoustic imaging: A review. Photonics, 9, 176(2022).

    [11] K. Tang et al. Advanced image post-processing methods for photoacoustic tomography: A review. Photonics, 10, 707(2023).

    [12] A. Y. Yuan et al. Hybrid deep learning network for vascular segmentation in photoacoustic imaging. Biomed. Opt. Express, 11, 6445-6457(2020).

    [13] T. Vaiyapuri et al. Design of metaheuristic optimization-based vascular segmentation techniques for photoacoustic images. Contrast Media Mol. Imaging, 2022, 4736113(2022).

    [14] C. D. Ly et al. Full-view in vivo skin and blood vessels profile segmentation in photoacoustic imaging based on deep learning. Photoacoustics, 25, 100310(2022).

    [15] Y. Gao et al. Deep learning-based photoacoustic imaging of vascular network through thick porous media. IEEE Trans. Med. Imaging, 41, 2191-2204(2022).

    [16] M. Sun et al. Full three-dimensional segmentation and quantification of tumor vessels for photoacoustic images. Photoacoustics, 20, 100212(2020).

    [17] J. Zhang et al. Photoacoustic image classification and segmentation of breast cancer: A feasibility study. IEEE Access, 7, 5457-5466(2018).

    [18] K. Jnawali et al. Automatic cancer tissue detection using multispectral photoacoustic imaging. Int. J. Comput. Assist. Radiol. Surg., 15, 309-320(2020).

    [19] S. Mandal, X. L. Deán-Ben, D. Razansky. Visual quality enhancement in optoacoustic tomography using active contour segmentation priors. IEEE Trans. Med. Imaging, 35, 2209-2217(2016).

    [20] L. Li et al. Single-impulse panoramic photoacoustic computed tomography of small-animal whole-body dynamics at high spatiotemporal resolution. Nat. Biomed. Eng., 1, 0071(2017).

    [21] C. Huang et al. Dual-scan photoacoustic tomography for the imaging of vascular structure on foot. IEEE Trans. Ultrason. Ferroelectr. Freq. Control, 70, 1703-1713(2023).

    [22] X. Lin et al. Handheld interventional ultrasound/photoacoustic puncture needle navigation based on deep learning segmentation. Biomed. Opt. Express, 14, 5979-5993(2023).

    [23] N. Davoudi, X. L. Deán-Ben, D. Razansky. Deep learning optoacoustic tomography with sparse data. Nat. Mach. Intell., 1, 453-460(2019).

    [24] C. Dehner et al. A deep neural network for real-time optoacoustic image reconstruction with adjustable speed of sound. Nat. Mach. Intell., 5, 1130-1141(2023).

    [25] S. Na et al. Massively parallel functional photoacoustic computed tomography of the human brain. Nat. Biomed. Eng., 6, 584-592(2022).

    [26] P. van Es et al. Coregistered photoacoustic and ultrasound tomography of healthy and inflamed human interphalangeal joints. Proc. SPIE, 9539, 95390C(2015).

    [27] P. van Es et al. Initial results of finger imaging using photoacoustic computed tomography. J. Biomed. Opt., 19, 060501(2014).

    [28] H. Wang et al. SAM-Med3D(2023).

    [29] W. Zheng et al. Deep learning enhanced volumetric photoacoustic imaging of vasculature in human. Adv. Sci., 10, 2301277(2023).

    Tools

    Get Citation

    Copy Citation Text

    Handi Deng, Yucheng Zhou, Jiaxuan Xiang, Liujie Gu, Yan Luo, Hai Feng, Mingyuan Liu, Cheng Ma. Streamlined photoacoustic image processing with foundation models: A training-free solution[J]. Journal of Innovative Optical Health Sciences, 2025, 18(1): 2450019

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Articles

    Received: Apr. 15, 2024

    Accepted: Jul. 9, 2024

    Published Online: Feb. 21, 2025

    The Author Email: Liu Mingyuan (dr.mingyuanliu@pku.edu.cn), Ma Cheng (cheng_ma@tsinghua.edu.cn)

    DOI:10.1142/S1793545824500196

    Topics