Real-Time Video Stitching Method Based on Dense Viewpoint Interpolation

Wei Heng; Jian Yu; Feipeng Da

doi:10.3788/AOS230509

Acta Optica Sinica, Volume. 43, Issue 14, 1415003(2023)

Real-Time Video Stitching Method Based on Dense Viewpoint Interpolation

Wei Heng^1,2, Jian Yu^1,2、*, and Feipeng Da^1,2,3、**

Author Affiliations

¹School of Automation, Southeast University, Nanjing 210096, Jiangsu, China

²Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, Jiangsu, China

³Shenzhen Research Institute, Southeast University, Shenzhen 518063, Guangdong, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(16)

Fig. 1. Effect of parallax on imaging results. (a) Views cannot be aligned when cameras' optical centers do not coincide; (b) views can be aligned when cameras' optical centers are completely coincident

Download full size

Fig. 2. Camera arrangement diagram

Download full size

Fig. 3. Add $k$ intermediate viewpoints on the baseline with width of $b$

Download full size

Fig. 4. Relation between coordinate points

Download full size

Fig. 5. Overall architecture of the network

Download full size

Fig. 6. Binary mask generated from depth map. (a) Depth map; (b) generated binary mask ( $k = 100$ )

Download full size

Fig. 7. Results of image fusion. (a) Multiband blending; (b) proposed, $k = 10$ ; (c) proposed, $k = 100$

Download full size

Fig. 8. Specific process of stitching

Download full size

Fig. 9. Comparison of stitching results under different scenarios

Download full size

Fig. 10. Comparison of stitching of video frames

Download full size

Fig. 11. Proportion of running time of different modules

Download full size

Fig. 12. Fusion results under different baseline widths. (a)(b) 0.5 m; (c)(d) 1 m; (e)(f) 1.5 m; (g)(h) 2 m

Download full size

Fig. 13. PSNR metrics under different angles of view

Download full size

Table 1. Comparison of SSIM and PSNR metrics

View table

Table 1. Comparison of SSIM and PSNR metrics

Method	FlyThings3D		KITTI 2015
Method	SSIM	PSNR	SSIM	PSNR
Multiband blending	0.555	15.78	0.416	14.51
APAP	0.672	16.42	0.523	16.03
LPC	0.648	16.33	0.568	15.89
LB-UDHN	0.718	20.41	0.654	18.59
MGDH	0.764	20.42	0.616	18.65
Proposed_v1	0.807	21.94	0.693	20.15
Proposed	0.826	23.31	0.708	20.54

Table 2. Comparison of model size and the speed
View table
Table 2. Comparison of model size and the speed
Method Model size /MB Runtime /ms
LB-UDHN 643 245
MGDH 187 97
Proposed 2.4 24

Table 3. SSIM and PSNR metrics under different baseline widths
View table
Table 3. SSIM and PSNR metrics under different baseline widths
Baseline width /m SSIM PSNR
Multiband blending Proposed Multiband blending Proposed
0.5 0.591 0.818 19.88 24.48
1.0 0.581 0.808 19.67 24.23
1.5 0.569 0.805 19.34 24.16
2.0 0.552 0.796 18.87 23.83