Single-View Endoscopic Surgical Light Field Reconstruction Combining Vision Transformer and Diffusion Model(Invited)

Fig. 2. The operational details of the decoder module. (a) Reassemble tokens into feature maps with two spatial dimensions; (b) continuously upsampling feature maps and fusing features at different scales; (c) reconstructing the feature map into final MPI representation

Download full size

Fig. 3. Visual comparison of light field reconstruction results with Zhou et al. and Tucker et al.

Download full size

Fig. 4. Ablation study of diffusion model-based background prediction: Da Vinci robot surgical data

Download full size

Fig. 5. Results of diffusion model-based background prediction: surgical microscope data

Download full size

Table 1. Comparison of different methods

View table

Table 1. Comparison of different methods

Method	PSNR↑	SSIM↑	LPIPS↓	FID↓
Zhou et al.^［14］	26.71±0.9625	0.8646±0.0245	0.1383±0.0484	78.2366
Tucker et al.^［27］	26.92±0.9838	0.8732±0.0596	0.1293±0.0515	76.3018
ViT-MPI^［33］	27.27±0.4715	0.8599±0.0132	0.1181±0.0293	70.1089
Proposed	27.55±1.9038	0.8902±0.0403	0.1157±0.0589	58.1178

Table 2. Ablation experiment

View table

Table 2. Ablation experiment

	PSNR↑	SSIM↑	LPIPS↓	FID↓
L1 Loss	25.81±1.2192	0.8515±0.0122	0.1947±0.0537	124.9708
Proposed w/o global token	24.43±2.7232	0.8537±0.0168	0.1238±0.0174	75.8218
Proposed w/o occlusion prediction	27.33±1.1817	0.8559±0.0406	0.1213±0.0615	70.1089
Proposed（D=8）	25.01±0.8435	0.7683±0.0296	0.1692±0.0465	169.6536
Proposed（D=16）	25.41±0.2761	0.8074±0.0113	0.1432±0.0346	130.1493
Proposed（D=32）	27.55±1.9038	0.8902±0.0403	0.1157±0.0589	58.1178

Table 3. Computational complexity of models in proposed method
View table
Table 3. Computational complexity of models in proposed method
Model GFLOPS Parameter /M
ViT 13.58 28.54
Diffusion 116.14 387.25

Tools

Get Citation

Copy Citation Text

Chenming Han, Gaochang Wu. Single-View Endoscopic Surgical Light Field Reconstruction Combining Vision Transformer and Diffusion Model(Invited)[J]. Laser & Optoelectronics Progress, 2024, 61(16): 1611013

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites