Acta Optica Sinica, Volume. 44, Issue 7, 0733001(2024)

Dynamic Three-Dimensional Reconstruction of Soft Tissue in Neural Radiation Field for Robotic Surgery Simulators

Qi Chen1,2, Zhibao Qin1,2, Xiaoyu Cai1,2, Shijie Li1,2, Zijun Wang1,2, Junsheng Shi1,2、*, and Yonghang Tai1,2、*
Author Affiliations
  • 1School of Physics and Electronic Information, Yunnan Normal University, Kunming 650500, Yunnan, China
  • 2Yunnan Key Laboratory of Optoelectronic Information Technology, Kunming 650500, Yunnan, China
  • show less
    Figures & Tables(14)
    Schematic of optically based reproduction of color information of soft tissues
    Key implementation steps
    Reconstruct the endoscopic surgery scene into a composite radiation field, and use the SASM to divide the endoscopic surgery scene into three regions: dynamic region, static region, and deformable region
    Overview of the general framework
    By promoting a more concentrated density distribution along each camera ray in the static component, the recovered background contains more highlights to enhance the realism of the image
    Experimental setup and robotic surgical simulation system
    Visualization results of segmentation in real surgical scenes
    Results of each soft tissue reconstruction in the endoscopic scenario of robotic surgery simulation (qualitative)
    Line graphs of color difference of each organ. (a) Lung color difference line graph; (b) kidney color difference line graph; (c) liver color difference line graph
    Different color difference representations indicate that the color of the reconstructed organ is close to the true color of biological soft tissue. (a) Photographed original kidney map; (b) 3D scatter plot of positional color difference; (c) contour map of magnification chromatic aberration; (d) 3D reconstructed kidney map; (e) vector chromatic aberration map in color coded map of R channel; (f) vector chromatic aberration map in color coded map of G channel
    Performance of reconstructed biological soft tissue improves over time (represented by increasing number of iterations)
    Influence of hyperparameters (the number of input images) on reconstruction performance. (a) PSNR; (b) SSIM; (c) MSSIM
    Influence of hyperparameters (number of hidden layers, regularization weights) in hash grids on reconstruction quality. (a) PSNR; (b) MAE; (c) NRMSE
    • Table 1. Dataset reconstructed by the introduction of supervised loss after each soft tissue structure is trained for 50000 times in each robotic surgery simulation scenario, and the dataset is compared with the ground truth

      View table

      Table 1. Dataset reconstructed by the introduction of supervised loss after each soft tissue structure is trained for 50000 times in each robotic surgery simulation scenario, and the dataset is compared with the ground truth

      MethodPSNR of lungSSIM of lungLPIPS of lungPSNR of kidneySSIM of kidneyLPIPS of kidneyPSNR of liverSSIM of liverLPIPS of liver
      NeRF19.90.5530.90520.10.5840.89723.50.5960.879
      InstantNGP23.50.7920.81628.50.6960.87126.00.6010.792
      Ours28.60.8600.79229.40.8880.79428.40.8990.738
    Tools

    Get Citation

    Copy Citation Text

    Qi Chen, Zhibao Qin, Xiaoyu Cai, Shijie Li, Zijun Wang, Junsheng Shi, Yonghang Tai. Dynamic Three-Dimensional Reconstruction of Soft Tissue in Neural Radiation Field for Robotic Surgery Simulators[J]. Acta Optica Sinica, 2024, 44(7): 0733001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Vision, Color, and Visual Optics

    Received: Sep. 8, 2023

    Accepted: Jan. 10, 2024

    Published Online: Apr. 11, 2024

    The Author Email: Shi Junsheng (taiyonghang@126.com), Tai Yonghang (taiyonghang@126.com)

    DOI:10.3788/AOS231537

    CSTR:32393.14.AOS231537

    Topics