3D Reconstruction of Neural Radiation Field Based on Improved Multiple Layer Perceptron

Yaofei Hou; Haisong Huang; Qingsong Fan; Jing Xiao; Zhenggong Han

doi:10.3788/LOP223312

Laser & Optoelectronics Progress, Volume. 61, Issue 4, 0415004(2024)

3D Reconstruction of Neural Radiation Field Based on Improved Multiple Layer Perceptron

Yaofei Hou¹, Haisong Huang^1,2、*, Qingsong Fan¹, Jing Xiao¹, and Zhenggong Han¹

¹Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang 550025, Guizhou, China

²Information Engineering Institute, Chongqing Vocational and Technical University of Mechatronics, Chongqing 402760, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(14)

Fig. 1. Structure of NeRF network

Download full size

Fig. 2. Structure of IP-NeRF network

Download full size

Fig. 3. Multi-feature joint learning module

Download full size

Fig. 4. Gated channel tranformation MLP module

Download full size

Fig. 5. Visualized new view reconstruction results under the modules in Trex and Lego scenes

Download full size

Fig. 6. Visualized new view reconstruction results of different methods in the selected scenes of the three datasets

Download full size

Table 1. Ablation experiment of the new view reconstruction in Trex scene

View table

Table 1. Ablation experiment of the new view reconstruction in Trex scene

Ablation experiment	PSNR/dB	SSIM	LPIPS
A	26.90（+0.37%）	0.881（+0.11%）	0.057
B	28.23（+5.33%）	0.925（+5.11%）	0.051（-10.53%）
C	28.56（+6.56%）	0.930（+5.68%）	0.046（-19.30%）
D	28.74（+7.24%）	0.934（+6.13%）	0.042（-26.31%）

Table 2. Ablation experiment of the new view reconstruction in Lego scene

View table

Table 2. Ablation experiment of the new view reconstruction in Lego scene

Ablation experiment	PSNR/dB	SSIM	LPIPS
A	32.62（+0.24%）	0.961	0.024
B	33.94（+4.33%）	0.968（+0.73%）	0.017（-29.17%）
C	34.37（+5.62%）	0.971（+1.04%）	0.016（-33.33%）
D	34.77（+6.85%）	0.974（+1.35%）	0.015（-37.50%）

Table 3. Parameter comparison of different methods on Realistic Synthetic 360° dataset

View table

Table 3. Parameter comparison of different methods on Realistic Synthetic 360° dataset

Scene	NeRF			NeRF-ID			IP-NeRF
Scene	PSNR/dB	SSIM	LPIPS	PSNR/dB	SSIM	LPIPS	PSNR/dB	SSIM	LPIPS
Mean	31.01	0.947	0.041	32.34	0.957	0.029	32.75	0.960	0.026
Chair	33.00	0.967	0.019	34.54	0.978	0.014	35.17	0.983	0.010
Drums	25.01	0.925	0.058	25.15	0.926	0.057	25.80	0.931	0.051
Ficus	30.13	0.964	0.022	32.24	0.976	0.015	31.86	0.973	0.016
Hotdog	36.18	0.974	0.016	37.26	0.981	0.013	38.48	0.986	0.010
Lego	32.54	0.961	0.024	34.73	0.974	0.015	34.77	0.974	0.015
Materials	29.62	0.949	0.029	30.37	0.956	0.024	31.90	0.977	0.011
Mic	32.91	0.980	0.023	34.71	0.988	0.009	34.21	0.982	0.018
Ship	28.65	0.856	0.119	29.75	0.876	0.081	29.79	0.876	0.081

Table 4. Parameter comparison of different methods on Real Forward-Facing dataset

View table

Table 4. Parameter comparison of different methods on Real Forward-Facing dataset

Scene	NeRF			NeRF-ID			IP-NeRF
Scene	PSNR /dB	SSIM	LPIPS	PSNR /dB	SSIM	LPIPS	PSNR /dB	SSIM	LPIPS
Mean	26.50	0.811	0.073	26.76	0.822	0.070	28.08	0.887	0.061
Fern	25.20	0.792	0.092	25.01	0.800	0.089	27.08	0.868	0.079
Flower	27.40	0.827	0.061	27.85	0.842	0.058	28.82	0.901	0.053
Fortress	31.16	0.881	0.030	31.51	0.888	0.028	32.94	0.933	0.024
Horns	27.45	0.828	0.068	27.88	0.843	0.065	29.30	0.911	0.057
Leaves	20.92	0.690	0.111	21.09	0.708	0.108	22.53	0.825	0.100
Orchids	20.36	0.641	0.121	20.38	0.643	0.120	21.44	0.764	0.100
Room	32.70	0.948	0.041	32.93	0.954	0.039	33.86	0.961	0.035
Trex	26.80	0.880	0.057	27.45	0.897	0.051	28.74	0.934	0.042

Table 5. Parameter comparison of different methods on DTU dataset

View table

Table 5. Parameter comparison of different methods on DTU dataset

Scene	NeRF			NeRF-ID			IP-NeRF
Scene	PSNR /dB	SSIM	LPIPS	PSNR /dB	SSIM	LPIPS	PSNR /dB	SSIM	LPIPS
Scan1 Scan22 Scan55 Scan109	23.49	0.754	0.282	23.80	0.765	0.266	24.47	0.778	0.248
	21.55	0.708	0.238	21.98	0.715	0.226	22.68	0.758	0.196
	26.54	0.794	0.229	26.76	0.800	0.219	27.23	0.812	0.206
	28.33	0.860	0.236	28.63	0.870	0.226	29.46	0.881	0.185
Mean	24.98	0.779	0.246	25.29	0.787	0.234	25.96	0.807	0.208

Table 6. Calculation cost comparison of different methods

View table

Table 6. Calculation cost comparison of different methods

Dataset	NeRF			NeRF-ID			IP-NeRF
Dataset	PSNR /dB	Train-time /h	Render-time /（s/it）	PSNR /dB	Train-time /h	Render-time /（s/it）	PSNR /dB	Train-time /h	Render-time/（s/it）
Realistic Synthetic 360° Real Forward-Facing DTU	31.01	18.4	21.18	32.34	14.9	17.10	32.75	19.5	22.24
	26.50	16.5	20.10	26.76	13.3	16.17	28.08	17.5	21.20
	24.98	19.4	36.60	25.29	15.6	30.48	25.96	20.5	38.63

Table 7. Calculation cost comparison of simplified network

View table

Table 7. Calculation cost comparison of simplified network

Dataset	NeRF-ID			SIP-NeRF
Dataset	PSNR /dB	Train-time /h	Render-time /（s/it）	PSNR /dB	Train-time /h	Render-time /（s/it）
Realistic Synthetic 360° Real Forward-Facing DTU	32.34	14.9	17.10	32.40	15.0	17.08
	26.76	13.3	16.17	27.68	13.3	16.10
	25.29	15.6	30.48	25.64	15.6	30.39

Table 8. Comprehensive performance analysis of different methods

View table

Table 8. Comprehensive performance analysis of different methods

Dataset	Parameter	NeRF^［13］	NSVF^［15］	GRF^［17］	NeuSample^［21］	NeXT^［20］	IP-NeRF
Realistic Synthetic 360°	PSNR /dB	31.01	31.75	32.06	31.15	34.40	32.75
Realistic Synthetic 360°	Train-time/h	18.4	1.5	23.0	14.0	52.7	19.5
Real Forward-Facing	PSNR /dB	26.50	/	26.64	26.83	/	28.08
Real Forward-Facing	Train-time /h	16.5	/	20.6	12.5	/	17.5

Tools

Get Citation

Copy Citation Text

Yaofei Hou, Haisong Huang, Qingsong Fan, Jing Xiao, Zhenggong Han. 3D Reconstruction of Neural Radiation Field Based on Improved Multiple Layer Perceptron[J]. Laser & Optoelectronics Progress, 2024, 61(4): 0415004

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Machine Vision

Received: Dec. 13, 2022

Accepted: Mar. 1, 2023

Published Online: Feb. 26, 2024

The Author Email: Haisong Huang (hshuang@gzu.edu.cn)

DOI:10.3788/LOP223312

CSTR:32186.14.LOP223312

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology