Acta Optica Sinica, Volume. 44, Issue 19, 1915001(2024)

Reconstruction of Dynamic Human Neural Radiance Fields Based on Monocular Vision

Chao Sun, Jun Qiu, Lina Wu, and Chang Liu*
Author Affiliations
  • Institute of Applied Mathematics, Beijing Information Science and Technology University, Beijing 100101, China
  • show less
    Figures & Tables(11)
    Overview of methods for joint reconstruction of dynamic and static neural radiation fields
    Human body decomposition diagram. (a) Original image; (b) decomposition results of SAM; (c) decomposition results of Mask-RCNN
    Alignment optimization example. (a) Original image; (b) alignment image by ROMP algorithm; (c) optimized alignment image
    Video data collection scenarios human 1‒6
    Background reconstruction. (a) Video capture scene; (b) static background reconstruction results
    Human body reconstruction results. (a) Da-pose new perspective map; (b) free viewing angle view
    Background and human body joint reconstruction of new pose rendering results. (a) Rollover; (b) walk; (c) roll; (d) balance
    Comparison of qualitative analysis results. (a) NSFF method; (b) Neuman method; (c) proposed method; (d) ground truth
    Ablation experiment. (a) No GC module; (b) no LSMPL module; (c) full module
    • Table 1. Quantitative analysis

      View table

      Table 1. Quantitative analysis

      MethodHuman 1Human 2Human 3
      PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
      NSFF20.110.670.4820.580.620.4421.270.690.43
      Neuman24.450.880.2524.150.750.3225.230.830.23
      Proposed25.220.910.1325.740.910.1126.560.900.12
      MethodHuman 4Human 5Human 6
      PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
      NSFF21.070.680.4021.790.650.4221.310.690.47
      Neuman25.490.840.2723.880.770.3225.430.840.28
      Proposed27.660.870.2525.540.880.2526.070.890.11
    • Table 2. Quantitative analysis of ablation

      View table

      Table 2. Quantitative analysis of ablation

      ModuleHuman 1Human 2Human 3
      PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓PSNR↑SSIM↑LPIPS↓
      No LSMPL22.040.710.3820.420.690.4822.870.730.36
      No GC24.320.840.2724.670.810.2724.820.850.17
      Full25.220.910.1325.740.910.1126.560.900.12
    Tools

    Get Citation

    Copy Citation Text

    Chao Sun, Jun Qiu, Lina Wu, Chang Liu. Reconstruction of Dynamic Human Neural Radiance Fields Based on Monocular Vision[J]. Acta Optica Sinica, 2024, 44(19): 1915001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Apr. 7, 2024

    Accepted: May. 13, 2024

    Published Online: Oct. 12, 2024

    The Author Email: Liu Chang (liu.chang.cn@ieee.org)

    DOI:10.3788/AOS240809

    CSTR:32393.14.AOS240809

    Topics