Acta Optica Sinica, Volume. 43, Issue 14, 1415003(2023)

Real-Time Video Stitching Method Based on Dense Viewpoint Interpolation

Wei Heng1,2, Jian Yu1,2、*, and Feipeng Da1,2,3、**
Author Affiliations
  • 1School of Automation, Southeast University, Nanjing 210096, Jiangsu, China
  • 2Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, Jiangsu, China
  • 3Shenzhen Research Institute, Southeast University, Shenzhen 518063, Guangdong, China
  • show less
    References(30)

    [1] Song C C, Gao C, Zhang Y C et al. Real-time stitching method of panoramic video based on three-dimensional spherical model[J]. Acta Optica Sinica, 43, 1010002(2023).

    [2] Chen H, Yang K L, Hu W J et al. Semantic visual odometry based on panoramic annular imaging[J]. Acta Optica Sinica, 41, 2215002(2021).

    [3] Kim H G, Lim H T, Ro Y M. Deep virtual reality image quality assessment with human perception guider for omnidirectional image[J]. IEEE Transactions on Circuits and Systems for Video Technology, 30, 917-928(2020).

    [4] Brown M, Lowe D G. Recognising panoramas[C], 1218-1225(2008).

    [5] Gao J H, Kim S J, Brown M S. Constructing image panoramas using dual-homography warping[C], 49-56(2011).

    [6] Zaragoza J, Chin T J, Brown M S et al. As-projective-as-possible image stitching with moving DLT[C], 2339-2346(2013).

    [7] Lee K Y, Sim J Y. Warping residual based image stitching for large parallax[C], 8195-8203(2020).

    [8] Gao J, Li Y, Chin T J et al. Seam-driven image stitching[C], 45-48(2013).

    [9] Lin K M, Jiang N J, Cheong L F et al. SEAGULL: seam-guided local alignment for parallax-tolerant image stitching[M]. Leibe B, Matas J, Sebe N, et al. Computer vision-ECCV 2016. Lecture notes in computer science, 9907, 370-385(2016).

    [10] He B T, Yu S H. Parallax-robust surveillance video stitching[J]. Sensors, 16, 7(2015).

    [11] Lo I C, Shih K T, Chen H H. Efficient and accurate stitching for 360° dual-fisheye images and videos[J]. IEEE Transactions on Image Processing, 31, 251-262(2022).

    [12] Nie Y W, Su T, Zhang Z S et al. Dynamic video stitching via shakiness removing[J]. IEEE Transactions on Image Processing, 27, 164-178(2018).

    [13] Li J M, Fan Y X, Wang N et al. Relative orientation method for airborne pushbroom combined imaging system[J]. Acta Optica Sinica, 41, 1811001(2021).

    [14] Anderson R, Gallup D, Barron J T et al. Jump: virtual reality video[J]. ACM Transactions on Graphics, 35, 1-13(2016).

    [15] Lee J, Kim B, Kim K et al. Rich360: optimized spherical representation from structured panoramic camera arrays[J]. ACM Transactions on Graphics, 35, 1-11(2016).

    [16] Shi Z F, Li H, Cao Q J et al. An image mosaic method based on convolutional neural network semantic features extraction[J]. Journal of Signal Processing Systems, 92, 435-444(2020).

    [17] Hoang V D, Tran D P, Nhu N G et al. Deep feature extraction for panoramic image stitching[M]. Nguyen N T, Jearanaitanakij K, Selamat A, et al. Intelligent Information and Database Systems. Lecture notes in computer science, 12034, 141-151(2020).

    [18] Zhao Q, Ma Y K, Zhu C et al. Image stitching via deep homography estimation[J]. Neurocomputing, 450, 219-229(2021).

    [19] Nie L, Lin C Y, Liao K et al. A view-free image stitching network based on global homography[J]. Journal of Visual Communication and Image Representation, 73, 102950(2020).

    [20] Lai W S, Gallo O, Gu J et al. Video stitching for linear camera arrays[C], 1-12(2019).

    [21] Li A C, Guo J, Guo Y W. Image stitching based on semantic planar region consensus[J]. IEEE Transactions on Image Processing, 30, 5545-5558(2021).

    [22] Dai Q Y, Fang F M, Li J C et al. Edge-guided composition network for image stitching[J]. Pattern Recognition, 118, 108019(2021).

    [23] Song D Y, Um G M, Lee H K et al. End-to-end image stitching network via multi-homography estimation[J]. IEEE Signal Processing Letters, 28, 763-767(2021).

    [24] Nie L, Lin C Y, Liao K et al. Depth-aware multi-grid deep homography estimation with contextual correlation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 32, 4460-4472(2022).

    [25] Nie L, Lin C Y, Liao K et al. Unsupervised deep image stitching: reconstructing stitched features to images[J]. IEEE Transactions on Image Processing, 30, 6184-6197(2021).

    [27] Khamis S, Fanello S, Rhemann C et al. StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction[M]. Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11219, 596-613(2018).

    [28] Sun D Q, Yang X D, Liu M Y et al. PWC-net: CNNs for optical flow using pyramid, warping, and cost volume[C], 8934-8943(2018).

    [30] Jia Q, Li Z J, Fan X et al. Leveraging line-point consistence to preserve structures for wide parallax image stitching[C], 12181-12190(2021).

    Tools

    Get Citation

    Copy Citation Text

    Wei Heng, Jian Yu, Feipeng Da. Real-Time Video Stitching Method Based on Dense Viewpoint Interpolation[J]. Acta Optica Sinica, 2023, 43(14): 1415003

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Feb. 6, 2023

    Accepted: Apr. 6, 2023

    Published Online: Jul. 13, 2023

    The Author Email: Jian Yu (yujian@seu.edu.cn), Feipeng Da (dafp@seu.edu.cn)

    DOI:10.3788/AOS230509

    Topics