Journal of Innovative Optical Health Sciences, Volume. 18, Issue 1, 2450019(2025)

Streamlined photoacoustic image processing with foundation models: A training-free solution

Handi Deng1...2,3,§, Yucheng Zhou4,§, Jiaxuan Xiang5, Liujie Gu1,2,3, Yan Luo1, Hai Feng6, Mingyuan Liu6,*, and Cheng Ma1,23,** |Show fewer author(s)
Author Affiliations
  • 1Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 2Institute for Precision Healthcare, Tsinghua University, 77 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 3Institute for Intelligent Healthcare, Tsinghua University, 77 Shuangqing Road, Haidian, Beijing 100084, P. R. China
  • 4School of Biological Science and Medical Engineering, Beihang University, 37 XueYuan Road, Haidian, Beijing 100191, P. R. China
  • 5TsingPAI Technology Co., Ltd., 27 Jiancaicheng Middle Road, Haidian, Beijing 100096, P. R. China
  • 6Department of Vascular Surgery, Beijing Friendship Hospital, Capital Medical University, 95 Yongan Road, Haidian, Beijing 100050, P. R. China
  • show less

    Foundation models (FMs) have rapidly evolved and have achieved significant accomplishments in computer vision tasks. Specifically, the prompt mechanism conveniently allows users to integrate image prior information into the model, making it possible to apply models without any training. Therefore, we proposed a workflow based on foundation models and zero training to solve the tasks of photoacoustic (PA) image processing. We employed the Segment Anything Model (SAM) by setting simple prompts and integrating the model’s outputs with prior knowledge of the imaged objects to accomplish various tasks, including: (1) removing the skin signal in three-dimensional PA image rendering; (2) dual speed-of-sound reconstruction, and (3) segmentation of finger blood vessels. Through these demonstrations, we have concluded that FMs can be directly applied in PA imaging without the requirement for network design and training. This potentially allows for a hands-on, convenient approach to achieving efficient and accurate segmentation of PA images. This paper serves as a comprehensive tutorial, facilitating the mastery of the technique through the provision of code and sample datasets.

    Tools

    Get Citation

    Copy Citation Text

    Handi Deng, Yucheng Zhou, Jiaxuan Xiang, Liujie Gu, Yan Luo, Hai Feng, Mingyuan Liu, Cheng Ma. Streamlined photoacoustic image processing with foundation models: A training-free solution[J]. Journal of Innovative Optical Health Sciences, 2025, 18(1): 2450019

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Articles

    Received: Apr. 15, 2024

    Accepted: Jul. 9, 2024

    Published Online: Feb. 21, 2025

    The Author Email: Liu Mingyuan (dr.mingyuanliu@pku.edu.cn), Ma Cheng (cheng_ma@tsinghua.edu.cn)

    DOI:10.1142/S1793545824500196

    Topics