Acta Photonica Sinica, Volume. 52, Issue 4, 0410004(2023)

Light Field All-in-focus Image Fusion Based on MDLatLRR and KPCA

Zefeng HUANG, Shen YANG*, Huiping DENG, and Qingson LI
Author Affiliations
  • School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China
  • show less

    The imaging of a light field camera can retain the spatial and angular information of light, therefore, different from traditional two-dimensional imaging, the light field camera can capture the light field directly in one shot, but it will sacrifice the spatial resolution and angular resolution of the image, so the quality of the image obtained is lower than that of the image generated by the native image sensor. This problem has prevented the application of light field imaging from gaining popularity. The development of multi-focus image fusion technology and digital refocusing technology provides ideas for improving the resolution of light field imaging. To improve the spatial resolution of light field imaging, we propose a full-focus fusion algorithm of light field image based on multi-scale latent low-rank decomposition and kernel principal component analysis by combining digital refocusing of light field image with multi-focus image fusion. First, by reprojecting the light field, the light is projected from the original focusing plane to the refocusing plane to generate a refocusing image with the focusing region, defocus region and blurred boundary region. After digitally refocusing, the spatial resolution of the light field image located in the focusing area is greatly improved. To extract the focus area accurately, the multi-level image decomposition method based on latent low-rank representation is used to decompose each refocusing image into a base layer and several saliency layers. Then, a two-region image sharpness extraction algorithm is used to calculate the image sharpness of the base layer, and a multi-scale saliency extraction algorithm is used to extract the visual saliency of the saliency layer's gradient domain. Secondly, the feature coefficient matrices of the base layer and each saliency layer are reshaped and concatenated. The kernel principal component analysis is used for dimension reduction fusion to obtain the fusion feature coefficient matrix which retains the feature information of both the base layer and the salient layer. Finally, the initial focusing decision map was generated by comparing the fusion feature coefficient matrix corresponding to each refocusing image. The small structure removal is applied to the initial decision map to eliminate the influence caused by image noise, and guided filtering is used to process the initial focusing decision map to generate the final focusing decision map, which solves the fusion problem at the fuzzy boundary, so that the fused image is smoother at the boundary of the focusing decision map and has a better visual effect. In order to verify the effectiveness of the proposed method, two sets of experiments are carried out: the full-focus fusion experiment and the multi-focus image fusion experiment. In the full-focus fusion experiment, several kinds of light field data, including objects located at different depths and flat backgrounds were selected from the HCI dataset, which can test the full-focus fusion ability of the proposed algorithm for complex textures and multi-objects. In the multi-focus image fusion experiment, six traditional multi-focus image fusion methods are selected for comparison with the proposed method, the experimental data are from the LFSD dataset, and ten objective evaluation indicators are selected for comparison. The experimental results show that the algorithm has better performance in visual effect and edge information richness compared with traditional methods. Subjectively, the full-focus images fused by the proposed method have higher spatial resolution and detailed texture compared with the original light field imaging. The resulting full-focus image is more in line with the visual perception of human eyes, objectively. Compared with the other six multi-focus image fusion methods, the multi-focus image generated by the proposed algorithm has the characteristics of high definition, excellent visual effect and high image contrast.

    Tools

    Get Citation

    Copy Citation Text

    Zefeng HUANG, Shen YANG, Huiping DENG, Qingson LI. Light Field All-in-focus Image Fusion Based on MDLatLRR and KPCA[J]. Acta Photonica Sinica, 2023, 52(4): 0410004

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jul. 21, 2022

    Accepted: Aug. 30, 2022

    Published Online: Jun. 21, 2023

    The Author Email: YANG Shen (yangshen@wust.edu.cn)

    DOI:10.3788/gzxb20235204.0410004

    Topics