Real-Time Stitching Method of Panoramic Videos Based on Three-Dimensional Spherical Model

Congcong Song; Ce Gao; Yanchao Zhang; Jiaxing Xu; Haibo Liu

doi:10.3788/AOS221659

Acta Optica Sinica, Volume. 43, Issue 10, 1010002(2023)

Real-Time Stitching Method of Panoramic Videos Based on Three-Dimensional Spherical Model

Congcong Song¹, Ce Gao^1、*, Yanchao Zhang¹, Jiaxing Xu¹, and Haibo Liu²

¹Precision Instrument and Equipment R&D Center, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, Jilin , China

²Engineering Research Management Department, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, Jilin , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(15)

Fig. 1. Schematic diagram of the specific structure of the photoelectric early warning system

Download full size

Fig. 2. Panoramic reconstruction of three-dimensional space. (a) Stereogram; (b) top view

Download full size

Fig. 3. Schematic diagram of camera coordinate system rotation

Download full size

Fig. 4. Optimal registration line based on 3D spherical model

Download full size

Fig. 5. Diagram of position relationship between adjacent frames

Download full size

Fig. 6. Panoramic image straightening. (a) Before straightening; (b) after straightening

Download full size

Fig. 7. Stitching effect of complex background images when the pitch angle is 0°. (a) Template and matching area of NCC; (b) stitching result of NCC; (c) template and matching area of DDIS; (d) stitching result of DDIS; (e) PSO-SIFT matching points; (f) stitching result of PSO-SIFT; (g) CAO-C2F matching points; (h) MS-HLMO matching points; (i) stitching result of the proposed method

Download full size

Fig. 8. Stitching effect of wall scene images when the pitch angle is 5°. (a) Template and matching area of NCC; (b) stitching result of NCC; (c) template and matching area of DDIS; (d) stitching result of DDIS; (e) PSO-SIFT matching points; (f) stitching result of PSO-SIFT; (g) CAO-C2F matching points; (h) MS-HLMO matching points; (i) stitching result of the proposed method

Download full size

Fig. 9. Stitching effect of tower scene images when the pitch angle is 5°. (a) Template and matching area of NCC; (b) stitching result of NCC; (c) template and matching area of DDIS; (d) stitching result of DDIS; (e) PSO-SIFT matching points; (f) stitching result of PSO-SIFT; (g) CAO-C2F matching points; (h) MS-HLMO matching points; (i) stitching result of the proposed method

Download full size

Fig. 10. Stitching effect of building scene images when the pitch angle is $20 °$ . (a) Template and matching area of NCC; (b) stitching result of NCC; (c) template and matching area of DDIS; (d) stitching result of DDIS; (e) PSO-SIFT matching points; (f) stitching result of PSO-SIFT; (g) CAO-C2F matching points; (h) MS-HLMO matching points; (i) stitching result of the proposed method

Download full size

Fig. 11. Partial panorama stitched by the proposed method. (a) Partial panorama of complex background when the pitch angle is 0°; (b) partial panorama of wall scene when the pitch angle is 5°; (c) partial panorama of tower scene when the pitch angle is 5°; (d) partial panorama of building scene when the pitch angle is 20°

Download full size

Fig. 12. Registration effect of different horizontal field angles when the pitch angle is 0°. (a) Horizontal field angle is 4.50°; (b) horizontal field angle is 4.48°; (c) horizontal field angle is 4.46°; (d) horizontal field angle is 4.44°; (e) horizontal field angle is 4.40°; (f) horizontal field angle is 4.38°; (g) horizontal field angle is 4.42°

Download full size

Fig. 13. Registration effect of different horizontal field angles when the pitch angle is 5°. (a) Horizontal field angle is 4.50°; (b) horizontal field angle is 4.48°; (c) horizontal field angle is 4.46°; (d) horizontal field angle is 4.44°; (e) horizontal field angle is 4.40°; (f) horizontal field angle is 4.38°; (g) horizontal field angle is 4.42°

Download full size

Table 1. Average time consumption of different methods when stitching adjacent images
View table
Table 1. Average time consumption of different methods when stitching adjacent images
Method CAO-C2F MS-HLMO PSO-SIFT NCC DDIS Ours
Time 4.205 16.089 12.883 0.104 1.921 0.015

Table 2. $f_{B_{2} M_{2}}$ , $f_{A_{2} N_{2}}$ at different horizontal field angles

View table

Table 2. $f_{B_{2} M_{2}}$ , $f_{A_{2} N_{2}}$ at different horizontal field angles

Horizontal viewing angle /（°）	Pixel focus	$(f_{B_{2} M_{2}}, f_{A_{2} N_{2}})$ when the pitch angle is 0°	$(f_{B_{2} M_{2}}, f_{A_{2} N_{2}})$ when the pitch angle is 5°
4.50	8145	（12，12）	（14，12）
4.48	8181	（10，10）	（12，11）
4.46	8218	（9，9）	（11，9）
4.44	8255	（7，7）	（9，8）
4.40	8330	（5，5）	（7，5）
4.38	8368	（3，3）	（5，3）
4.42	8292	（6，6）	（8，6）

Tools

Get Citation

Copy Citation Text

Congcong Song, Ce Gao, Yanchao Zhang, Jiaxing Xu, Haibo Liu. Real-Time Stitching Method of Panoramic Videos Based on Three-Dimensional Spherical Model[J]. Acta Optica Sinica, 2023, 43(10): 1010002

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Image Processing

Received: Aug. 29, 2022

Accepted: Jan. 9, 2023

Published Online: May. 9, 2023

The Author Email: Gao Ce (50616636@qq.com)

DOI:10.3788/AOS221659

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Average time consumption of different methods when stitching adjacent images

Table 1. Average time consumption of different methods when stitching adjacent images

Table 2. fB2M2, fA2N2 at different horizontal field angles

Table 2. fB2M2, fA2N2 at different horizontal field angles

Table 2. $f_{B_{2} M_{2}}$ , $f_{A_{2} N_{2}}$ at different horizontal field angles

Table 2. $f_{B_{2} M_{2}}$ , $f_{A_{2} N_{2}}$ at different horizontal field angles