Acta Optica Sinica, Volume. 43, Issue 14, 1411001(2023)

Neural Radiance Field-Based Light Field Super-Resolution in Angular Domain

Yuan Miao, Chang Liu, and Jun Qiu*
Author Affiliations
  • Institute of Applied Mathematics, Beijing Information Science and Technology University, Beijing 100101, China
  • show less
    Figures & Tables(11)
    Overview of training NeRF using light field data. By sampling light field 5D coordinates (position and viewing directions) along camera rays, feeding these positions into MLP to produce color and volume densities, and using volume rendering techniques to synthesize these values into corresponding image. Rendering losses are fitted using gradient descent to optimize scene representation
    Process of NeRF based light field super-resolution in angular domain. (a) Input 5*5 sub-aperture image; (b) expression of optical field using NeRF; (c) output 9*9 sub-aperture image
    Diagram of NeRF training network structure. MLP takes x (3D coordinate points and view directions) as input through first eight fully connected layers (FC), after which feature vectors will be connected to position encoding γ(d) of input observation directions, with output σ and RGB values
    New perspective for angular super-resolution generation of HCI antinous scenes using different methods. (a) Ground truth; (b) LFEPICNN method; (c) LLFF method; (d) proposed method; (e), (f), (g), (h) local zooms of antinous scene after using above methods, respectively
    New perspective for angular super-resolution generation of HCI boardgames scenes using different methods. (a) Ground truth; (b) LFEPICNN method; (c) LLFF method; (d) proposed method; (e), (f), (g), (h) local zooms of boardgames scene after using above methods, respectively
    Comparison of partial EPI before and after angular super-resolution of HCI boardgames scene. (a) Fix u=3, y=200 to draw EPI; (b) partial EPI of original data; (c) partial EPI of position corresponding to original data after angular super-resolution; (d) partial EPI after angular super-resolution
    New perspective for angular super-resolution generation of Stanford knights scenes. (a) Ground truth; (b) LFEPICNN method; (c) LLFF method; (d) proposed method; (e), (f), (g), (h) local zooms of knights scene after using above methods, respectively
    New perspective for angular super-resolution generation of Stanford bunny scenes. (a) Ground truth; (b) LFEPICNN method; (c) LLFF method; (d) proposed method; (e), (f), (g), (h) local zooms of bunny scene after using above methods, respectively
    Comparison of partial EPI before and after angular super-resolution of Stanford knights scene. (a) Fix u=3, y=210 to draw EPI; (b) partial EPI of original data; (c) partial EPI of position corresponding to original data after angular super-resolution; (d) partial EPI after angular super-resolution
    • Table 1. Metrics evaluation on HCI simulated light field dataset

      View table

      Table 1. Metrics evaluation on HCI simulated light field dataset

      AntinousBoardgames
      PSNRSSIMLPIPSPSNRSSIMLPIPS
      LFEPICNN36.250.85420.09438.750.90230.068
      LLFF33.860.78300.02835.330.83720.046
      Proposed43.230.99360.01142.150.98290.013
    • Table 2. Metrics evaluation on Stanford actual light field dataset

      View table

      Table 2. Metrics evaluation on Stanford actual light field dataset

      BunnyKnights
      PSNRSSIMLPIPSPSNRSSIMLPIPS
      LFEPICNN36.530.89710.09635.790.85420.128
      LLFF35.340.83320.05337.440.90230.045
      Proposed40.720.94360.01441.350.95210.012
    Tools

    Get Citation

    Copy Citation Text

    Yuan Miao, Chang Liu, Jun Qiu. Neural Radiance Field-Based Light Field Super-Resolution in Angular Domain[J]. Acta Optica Sinica, 2023, 43(14): 1411001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Imaging Systems

    Received: Feb. 14, 2023

    Accepted: Mar. 24, 2023

    Published Online: Jul. 13, 2023

    The Author Email: Jun Qiu (qiujun@bistu.edu.cn)

    DOI:10.3788/AOS230549

    Topics