Laser & Optoelectronics Progress, Volume. 61, Issue 16, 1611016(2024)

Dynamic Panoramic Image Synthesis Technology Using Compound Eye Camera Array (Invited)

Mingzhi Cao1,2, Bowen Wang1,2, Jingya Qi3, Fujie Wu1,2, Yingjun Sang1,2, Sheng Li1,2, Lin Li3、*, Yuzhen Zhang1,2、**, Qian Chen1,2, and Chao Zuo1,2、***
Author Affiliations
  • 1Smart Computational Imaging Laboratory, School of Electronic and Optical Engineering, Nanjing University of Science & Technology, Nanjing 210094, Jiangsu, China
  • 2Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing 210094, Jiangsu, China
  • 3Lab of Space Optoelectronic Measurement & Intelligent Sense, Beijing Institute of Control Engineering, Beijing 100190, China
  • show less

    In optoelectronic imaging systems, the use of multi-camera/multi-aperture computational imaging technology is increasingly recognized as an essential method for achieving wide-field and high-resolution image reconstruction. This technology seeks to overcome the spatial bandwidth product limitations inherent in single imaging systems. However, multi-aperture image synthesis typically involves complex processes such as feature point extraction, descriptor matching, and alignment between sub-eye images, which substantially increases the computational complexity and poses challenges for real-time performance. This is particularly problematic in scenes with moving objects, where the reconstructed images often suffer from ghosting phenomena such as artifacts and misalignment, thereby degrading the imaging quality. To address these issues, this paper introduces a dynamic panoramic image synthesis technology using a multi-aperture compound eye camera array. The technology leverages compute unified device architecture (CUDA) to enhance and accelerate the registration algorithm based on speeded up robust features (SURF). Furthermore, it incorporates a frame difference method to minimize redundant information in the multi-frame registration process, enhancing the registration accuracy of the synthesized image by 15.91% and reducing the registration time by 91.23%. Additionally, the visual background extraction (VIBE) algorithm is integrated into the energy function of the seam line, with established update criteria for the seam line, facilitating the synthesis of panoramic images from multi-frame motion images without misalignment and artifacts. A 5×5 multi-aperture compound eye camera array imaging system was developed to achieve a horizontal 90° field of view synthesis. Compared with traditional stitching algorithms, this approach demonstrates improvements of 1.96 and 1.85 on the reference-free image space quality evaluation index (BRISQUE) and the perception-based image quality evaluation index (PIQE), respectively. Moreover, the system can complete the stitching, reconstruction, and synthesis of 25 sub images at interactive frame rate of 13 frame/s, reducing the reconstruction time by 99.7% compared to non-CUDA accelerated stitching algorithms.

    Keywords
    Tools

    Get Citation

    Copy Citation Text

    Mingzhi Cao, Bowen Wang, Jingya Qi, Fujie Wu, Yingjun Sang, Sheng Li, Lin Li, Yuzhen Zhang, Qian Chen, Chao Zuo. Dynamic Panoramic Image Synthesis Technology Using Compound Eye Camera Array (Invited)[J]. Laser & Optoelectronics Progress, 2024, 61(16): 1611016

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Imaging Systems

    Received: Apr. 9, 2024

    Accepted: May. 20, 2024

    Published Online: Aug. 12, 2024

    The Author Email: Lin Li (cast_lilin@163.com), Yuzhen Zhang (479608575@qq.com), Chao Zuo (zuochao@njust.edu.cn)

    DOI:10.3788/LOP241062

    CSTR:32186.14.LOP241062

    Topics