Progress in Novel View Synthesis Using Neural Radiance Fields

Gaoxiang He; Bin Zhu; Bo Xie; Yi Chen

doi:10.3788/LOP231578

Laser & Optoelectronics Progress, Volume. 61, Issue 12, 1200005(2024)

Progress in Novel View Synthesis Using Neural Radiance Fields

Gaoxiang He^1,2, Bin Zhu^1,2、*, Bo Xie^1,2, and Yi Chen^1,2

¹State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering, National University of Defense Technology, Hefei 230037, Anhui , China

²Infrared and Low Temperature Plasma Key Laboratory of Anhui Province, Hefei 230037, Anhui , China

show less

Abstract Get PDF(in Chinese)

References(74)

[1] Tong W W, Xie J W, Li T Y et al. 3D data augmentation for driving scenes on camera[EB/OL]. https://arxiv.org/abs/2303.10340

[2] Yariv L, Hedman P, Reiser C et al. BakedSDF: meshing neural SDFs for real-time view synthesis[EB/OL]. https://arxiv.org/abs/2302.14859

[3] Tang J X, Zhou H, Chen X K et al. Delicate textured mesh recovery from NeRF via adaptive surface refinement[EB/OL]. https://arxiv.?org/abs/2303.02091

[4] Deng N C, He Z Y, Ye J N et al. FoV-NeRF: foveated neural radiance fields for virtual reality[J]. IEEE Transactions on Visualization and Computer Graphics, 28, 3854-3864(2022).

[5] Li K, Rolff T, Schmidt S et al. Immersive neural graphics primitives[EB/OL]. https://arxiv.org/abs/2211.13494

[6] Li C J, Li S X, Zhao Y et al. RT-NeRF: real-time on-device neural radiance fields towards immersive AR/VR rendering[C](2022).

[7] Sun S H, Huh M, Liao Y H, Ferrari V, Hebert M, Sminchisescu C et al. Multi-view to novel view: synthesizing novel views with self-learned confidence[M]. Computer vision-ECCV 2018. Lecture notes in computer science, 11207, 162-178(2018).

[8] Kato H, Beker D, Morariu M et al. Differentiable rendering: a survey[EB/OL]. https://arxiv.org/abs/2006.12057

[9] Vicini D, Speierer S, Jakob W. Differentiable signed distance function rendering[J]. ACM Transactions on Graphics, 41, 125(2022).

[10] Li T M, Aittala M, Durand F et al. Differentiable Monte Carlo ray tracing through edge sampling[J]. ACM Transactions on Graphics, 37, 222(2018).

[11] Zhang C, Miller B, Yan K et al. Path-space differentiable rendering[J]. ACM Transactions on Graphics, 39, 143(2020).

[12] Tewari A, Fried O, Thies J et al. Advances in neural rendering[C](2021).

[13] Park J J, Florence P, Straub J et al. DeepSDF: learning continuous signed distance functions for shape representation[C], 165-174(2020).

[14] Genova K, Cole F, Sud A et al. Local deep implicit functions for 3D shape[C], 4856-4865(2020).

[15] Mildenhall B, Srinivasan P P, Tancik M, Vedaldi A, Bischof H, Brox T et al. NeRF: representing scenes as neural radiance fields for view synthesis[M]. Computer vision-ECCV 2020. Lecture notes in computer science, 12346, 405-421(2020).

[16] Zhang K, Riegler G, Snavely N et al. NeRF++: analyzing and improving neural radiance fields[EB/OL]. https://arxiv.org/abs/2010.07492

[17] Martin-Brualla R, Radwan N, Sajjadi M S M et al. NeRF in the wild: neural radiance fields for unconstrained photo collections[C], 7206-7215(2021).

[18] Yu A, Ye V, Tancik M et al. pixelNeRF: neural radiance fields from one or few images[C], 4576-4585(2021).

[19] Zheng Q F. Survey of neural radiance field acceleration technologies[J]. ZTE Technology Journal, 29, 79-86(2023).

[20] Dellaert F, Yen-Chen L. Neural volume rendering: NeRF and beyond[EB/OL]. https://arxiv.org/abs/2101.05204

[21] Chang Y, Gai M. A review on neural radiance fields based view synthesis[J]. Journal of Graphics, 42, 376-384(2021).

[22] Gao K, Gao Y N, He H J et al. NeRF: neural radiance field in 3D vision, a comprehensive review[EB/OL]. https://arxiv.org/abs/2210.00379

[23] Novák J, Georgiev I, Hanika J et al. Monte Carlo methods for volumetric light transport simulation[J]. Computer Graphics Forum, 37, 551-576(2018).

[24] Rahaman N, Baratin A, Arpit D et al. On the spectral bias of neural networks[EB/OL]. https://arxiv.org/abs/1806.08734

[25] Barron J T, Mildenhall B, Tancik M et al. Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields[C], 5835-5844(2022).

[26] Deng B, Barron J T, Srinivasan P P. JaxNeRF: an efficient jax implementation of NeRF[EB/OL]. https://github.?com/google-research/google-research/tree/master/jaxnerf

[27] Wang Z, Bovik A C, Sheikh H R et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 13, 600-612(2004).

[28] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https://arxiv.org/abs/1409.1556

[29] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 60, 84-90(2017).

[30] Zhang R, Isola P, Efros A A et al. The unreasonable effectiveness of deep features as a perceptual metric[C], 586-595(2018).

[31] Mildenhall B, Srinivasan P P, Ortiz-Cayon R et al. Local light field fusion[J]. ACM Transactions on Graphics, 38, 29(2019).

[32] Yücer K, Sorkine-Hornung A, Wang O et al. Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction[J]. ACM Transactions on Graphics, 35, 22(2016).

[33] Knapitsch A, Park J, Zhou Q Y et al. Tanks and temples[J]. ACM Transactions on Graphics, 36, 78(2017).

[34] Barron J T, Mildenhall B, Verbin D et al. Mip-NeRF 360: unbounded anti-aliased neural radiance fields[C], 5460-5469(2022).

[35] Jensen R, Dahl A, Vogiatzis G et al. Large scale multi-view stereopsis evaluation[C], 406-413(2014).

[36] Chang A X, Funkhouser T, Guibas L et al. ShapeNet: an information-rich 3D model repository[EB/OL]. https://arxiv.org/abs/1512.03012

[37] Jin Y H, Mishkin D, Mishchuk A et al. Image matching across wide baselines: from paper to practice[J]. International Journal of Computer Vision, 129, 517-547(2021).

[38] Sun C, Sun M, Chen H T. Improved direct voxel grid optimization for radiance fields reconstruction[EB/OL]. https://arxiv.org/abs/2206.05085

[39] Fridovich-Keil S, Yu A, Tancik M et al. Plenoxels: radiance fields without neural networks[C], 5491-5500(2022).

[40] Turki H, Ramanan D, Satyanarayanan M. Mega-NeRF: scalable construction of large-scale NeRFs for virtual fly- throughs[C], 12912-12921(2022).

[41] Yan H, Liu C L, Ma C et al. Plen-VDB: memory efficient VDB-based radiance fields for fast training and rendering[C], 88-96(2023).

[42] Liu L J, Gu J T, Lin K Z et al. Neural sparse voxel fields[EB/OL]. https://arxiv.?org/?abs/2007.?11571

[43] Reiser C, Szeliski R, Verbin D et al. MERF: memory-efficient radiance fields for real-time view synthesis in unbounded scenes[J]. ACM Transactions on Graphics, 42, 89.

[44] Barron J T, Mildenhall B, Verbin D et al. Zip-NeRF: anti-aliased grid-based neural radiance fields[EB/OL]. https://arxiv.org/abs/2304.06706

[45] Li R L, Gao H, Tancik M et al. NerfAcc: efficient sampling accelerates NeRFs[EB/OL]. https://arxiv.org/abs/2305.04966

[46] Müller T, Evans A, Schied C et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 41, 102(2022).

[47] Bojanowski P, Joulin A, Lopez-Paz D et al. Optimizing the latent space of generative networks[EB/OL]. https://arxiv.org/abs/1707.05776

[48] Chen X Y, Zhang Q, Li X Y et al. Hallucinated neural radiance fields in the wild[C], 12933-12942(2022).

[49] Lee J W, Kim I H, Heo H et al. Semantic-aware occlusion filtering neural radiance fields in the wild[EB/OL]. https://arxiv.org/abs/2303.03966

[50] Tancik M, Casser V, Yan X C et al. Block-NeRF: scalable large scene neural view synthesis[C], 8238-8248(2022).

[51] Deng J, Dong W, Socher R et al. ImageNet: a large-scale hierarchical image database[C], 248-255(2009).

[52] Trevithick A, Yang B. GRF: learning a general radiance field for 3D representation and rendering[C], 15162-15172(2022).

[53] Wang Q Q, Wang Z C, Genova K et al. IBRNet: learning multi-view image-based rendering[C], 4688-4697(2021).

[54] Vaswani A, Shazeer N, Parmar N et al. Attention is all you need[EB/OL]. https://arxiv.org/abs/1706.03762

[55] Chen A P, Xu Z X, Zhao F Q et al. MVSNeRF: fast generalizable radiance field reconstruction from multi-view stereo[C], 14104-14113(2022).

[56] Schönberger J L, Frahm J M. Structure-from-motion revisited[C], 4104-4113(2016).

[57] Wei Y, Liu S H, Rao Y M et al. NerfingMVS: guided optimization of neural radiance fields for indoor multi-view stereo[C], 5590-5599(2022).

[58] Deng K L, Liu A, Zhu J Y et al. Depth-supervised NeRF: fewer views and faster training for free[C], 12872-12881(2022).

[59] Wang G C, Chen Z X, Loy C C et al. SparseNeRF: distilling depth ranking for few-shot novel view synthesis[EB/OL]. https://arxiv.org/abs/2303.16196

[60] Song J, Park S, An H et al. DäRF: boosting radiance fields from sparse inputs with monocular depth adaptation[EB/OL]. https://arxiv.?org/abs/2305.19201

[61] Jain A, Tancik M, Abbeel P. Putting NeRF on a diet: semantically consistent few-shot view synthesis[C], 5865-5874(2022).

[62] Niemeyer M, Barron J T, Mildenhall B et al. RegNeRF: regularizing neural radiance fields for view synthesis from sparse inputs[C], 5470-5480(2022).

[63] Kim M, Seo S, Han B. InfoNeRF: ray entropy minimization for few-shot neural volume rendering[C], 12902-12911(2022).

[64] Kwak M S, Song J, Kim S. GeCoNeRF: few-shot neural radiance fields via geometric consistency[EB/OL]. https://arxiv.org/abs/2301.10941

[65] Yang J W, Pavone M, Wang Y. FreeNeRF: improving few-shot neural rendering with free frequency regularization[C], 8254-8263(2023).

[66] Song L C, Li Z, Gong X et al. Harnessing low-frequency neural fields for few-shot view synthesis[EB/OL]. https://arxiv.org/abs/2303.08370

[67] Xie Y H, Takikawa T, Saito S et al. Neural fields in visual computing and beyond[J]. Computer Graphics Forum, 41, 641-676(2022).

[68] Yu A, Li R L, Tancik M et al. PlenOctrees for real-time rendering of neural radiance fields[C], 5732-5741(2022).

[69] Sun C, Sun M, Chen H T. Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction[C], 5449-5459(2022).

[70] Karnewar A, Ritschel T, Wang O et al. ReLU fields: the little non-linearity that could[C](2022).

[71] Hu T, Liu S, Chen Y L et al. EfficientNeRF-efficient neural radiance fields[C], 12892-12901(2022).

[72] Chen A P, Xu Z X, Geiger A, Avidan S, Brostow G, Cissé M et al. TensoRF: tensorial radiance fields[M]. Computer vision-ECCV 2022. Lecture notes in computer science, 13692, 333-350(2022).

[73] Zhang Y Q, Chen G Y, Cui S G. Efficient large-scale scene representation with a hybrid of high-resolution grid and plane features[EB/OL]. https://arxiv.org/abs/2303.03003

[74] Museth K, Lait J, Johanson J et al. OpenVDB: an open-source data structure and toolkit for high-resolution volumes[C](2013).

Tools

Get Citation

Copy Citation Text

Gaoxiang He, Bin Zhu, Bo Xie, Yi Chen. Progress in Novel View Synthesis Using Neural Radiance Fields[J]. Laser & Optoelectronics Progress, 2024, 61(12): 1200005

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Reviews

Received: Jun. 21, 2023

Accepted: Sep. 4, 2023

Published Online: Jun. 3, 2024

The Author Email: Bin Zhu (zhubineei@163.com)

DOI:10.3788/LOP231578

CSTR:32186.14.LOP231578

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology