Real-time all-directional 3D recognition and multidistortion correction via prior diffraction neural networks

Min Huang; Bin Zheng; Ruichen Li; Yijun Zou; Xiaofeng Li; Chao Qian; Huan Lu; Rongrong Zhu; Hongsheng Chen

doi:10.1117/1.AP.7.5.056005

Advanced Photonics, Volume. 7, Issue 5, 056005(2025)

Real-time all-directional 3D recognition and multidistortion correction via prior diffraction neural networks

Min Huang^1,2, Bin Zheng^1,3,4, Ruichen Li¹, Yijun Zou¹, Xiaofeng Li⁵, Chao Qian⁶, Huan Lu^1,4, Rongrong Zhu^1,7、*, and Hongsheng Chen^3,4,6、*

Author Affiliations

¹Zhejiang University, Interdisciplinary Center for Quantum Information, State Key Laboratory of Modern Optical Instrumentation, ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou, China

²National University of Defense Technology, College of Electronic Science and Technology, Changsha, China

³Zhejiang University, International Joint Innovation Center, Key Laboratory of Advanced Micro/Nano Electronic Devices & Smart Systems of Zhejiang, The Electromagnetics Academy at Zhejiang University, Haining, China

⁴Zhejiang University, Jinhua Institute of Zhejiang University, Jinhua, China

⁵Air Force Engineering University, Air and Missile Defense College, Xi’an, China

⁶Zhejiang University, ZJU-UIUC Institute, Interdisciplinary Center for Quantum Information, State Key Laboratory of Extreme Photonics and Instrumentation, Hangzhou, China

⁷Hangzhou City University, School of Information and Electrical Engineering, Zhejiang, China

show less

References(49)

[1] Z. Deng, L. Zhou. Detection and recognition of traffic planar objects using colorized laser scan and perspective distortion rectification. IEEE Trans. Intell. Transp., 19, 1485-1495(2018). https://doi.org/10.1109/TITS.2017.2723902

[2] E. U. Samani, A. G. Banerjee. Persistent homology meets object unity: object recognition in clutter. IEEE Trans. Robot., 40, 886-902(2024). https://doi.org/10.1109/TRO.2023.3343994

[3] R. C. Luo, C.-W. Kuo. Intelligent seven-DoF robot with dynamic obstacle avoidance and 3-D object recognition for industrial cyber-physical systems in manufacturing automation. Proc. IEEE, 104, 1102-1113(2016). https://doi.org/10.1109/JPROC.2015.2508598

[4] Y. Tian et al. 3D object recognition method with multiple feature extraction from LiDAR point clouds. J. Supercomput., 75, 4430-4442(2019). https://doi.org/10.1007/s11227-019-02830-9

[5] L.-F. Shi et al. Autonomous multiframe point cloud fusion method for mmWave radar. IEEE Trans. Instrum. Meas., 72, 4505708(2023). https://doi.org/10.1109/TIM.2023.3302936

[6] R. Zheng et al. Deep-neural-network-enabled vehicle detection using high-resolution automotive radar imaging. IEEE Trans. Aero. Elec. Syst., 59, 4815-4830(2023). https://doi.org/10.1109/TAES.2023.3275887

[7] W. Tao et al. A pipeline for 3-D object recognition based on local shape description in cluttered scenes. IEEE Trans. Geosci. Remote, 59, 801-816(2021). https://doi.org/10.1109/TGRS.2020.2998683

[8] L. Tan et al. Projected generative adversarial network for point cloud completion. IEEE Trans. Circ. Syst. Vid., 33, 771-781(2023). https://doi.org/10.1109/TCSVT.2022.3204771

[9] G. Ma, H. Wei. A novel sketch-based framework utilizing contour cues for efficient point cloud registration. IEEE Trans. Geosci. Remote Sens., 61, 5703616(2023). https://doi.org/10.1109/TGRS.2023.3307061

[10] H. Lu et al. Eye accommodation-inspired neuro-metasurface focusing. Nat. Commun., 14, 3301(2023). https://doi.org/10.1038/s41467-023-39070-8

[11] Z. Wang et al. On-chip wavefront shaping with dielectric metasurface. Nat. Commun., 10, 3547(2019). https://doi.org/10.1038/s41467-019-11578-y

[12] B. Zheng et al. Revealing the transformation invariance of full-parameter omnidirectional invisibility cloaks. Electromag. Sci., 1, 0020092(2023). https://doi.org/10.23919/emsci.2023.0009

[13] R. Zhu et al. Metasurface-enabled electromagnetic illusion with generic algorithm. Front. Mater., 10, 1289250(2023). https://doi.org/10.3389/fmats.2023.1289250

[14] M. Huang et al. Evolutionary games-assisted synchronization metasurface for simultaneous multisource invisibility cloaking. Adv. Funct. Mater., 34, 2401909(2024). https://doi.org/10.1002/adfm.202401909

[15] A. Kandwal et al. Designing highly sensitive microwave antenna sensor with novel model for noninvasive glucose measurements. Prog. Electromagn. Res., 176, 129-141(2023). https://doi.org/10.2528/PIER22113002

[16] H. Tian et al. A novel noncontact Ku-band microwave radiometer for human body temperature measurements. Prog. Electromagn. Res., 178, 18-36(2023). https://doi.org/10.2528/PIER23042503

[17] M. Huang et al. Diffraction neural network for multi-source information of arrival sensing. Laser Photonics Rev., 17, 2300202(2023). https://doi.org/10.1002/lpor.202300202

[18] H. Xu et al. Low-altitude UAV detection based on vehicle-mounted wideband programmable metasurface. IEEE Trans. Microw Theory, 72, 7018-7027(2024). https://doi.org/10.1109/TMTT.2024.3406019

[19] M. Huang et al. Machine–learning-enabled metasurface for direction of arrival estimation. Nanophotonics, 11, 2001-2010(2022). https://doi.org/10.1515/nanoph-2021-0663

[20] Z. Li et al. Full-space cloud of random points with a scrambling metasurface. Light Sci. Appl., 7, 63(2018). https://doi.org/10.1038/s41377-018-0064-3

[21] H. Zheng et al. Multichannel meta-imagers for accelerating machine vision. Nat. Nanotechnol., 19, 471-478(2024). https://doi.org/10.1038/s41565-023-01557-2

[22] M. Burfeindt, H. F. Alqadah. Linear sampling method imaging of three-dimensional conducting targets from limited apertures via phase-delay-constrained formulations. Prog. Electromagn. Res., 178, 63-81(2023). https://doi.org/10.2528/PIER23040504

[23] X. Ye et al. Multi-biomarker combination detection system for diagnosis and classification of dry eye disease by imaging of a multi-channel metasurface. Biosens. Bioelectron., 248, 115933(2024). https://doi.org/10.1016/j.bios.2023.115933

[24] Z. Gu et al. Classification of metal handwritten digits based on microwave diffractive deep neural network. Adv. Opt. Mater., 12, 2301938(2024). https://doi.org/10.1002/adom.202301938

[25] L. L. Li et al. Machine-learning reprogrammable metasurface imager. Nat. Commun., 10, 1082(2019). https://doi.org/10.1038/s41467-019-09103-2

[26] H. Yu et al. All-optical image transportation through a multimode fibre using a miniaturized diffractive neural network on the distal facet. Nat. Photonics, 19, 486-493(2025). https://doi.org/10.1038/s41566-025-01621-4

[27] Y. Guo et al. 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Trans. Pattern Anal., 36, 2270-2287(2014). https://doi.org/10.1109/TPAMI.2014.2316828

[28] S. Zhou et al. Recognition-oriented image compressive sensing with deep learning. IEEE Trans. Multimedia, 25, 2022-2032(2023). https://doi.org/10.1109/TMM.2022.3142952

[29] B. Shi, X. Bai, C. Yao. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal., 39, 2298-2304(2017). https://doi.org/10.1109/TPAMI.2016.2646371

[30] X. H. Luo et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci. Appl., 11, 158(2022). https://doi.org/10.1038/s41377-022-00844-2

[31] Z. C. Wang et al. Single-layer spatial analog meta-processor for imaging processing. Nat. Commun., 13, 2188(2022). https://doi.org/10.1038/s41467-022-29732-4

[32] W. W. Fu et al. Ultracompact meta-imagers for arbitrary all-optical convolution. Light Sci. Appl., 11, 62(2022). https://doi.org/10.1038/s41377-022-00752-5

[33] A. M. McNeil et al. Fundamentals and recent developments of free-space optical neural networks. J. Appl. Phys., 136, 030701(2024). https://doi.org/10.1063/5.0215752

[34] Y. Luo et al. Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl., 8, 112(2019). https://doi.org/10.1038/s41377-019-0223-1

[35] M. Veli et al. Terahertz pulse shaping using diffractive surfaces. Nat. Commun., 12, 37(2021). https://doi.org/10.1038/s41467-020-20268-z

[36] J. X. Li et al. Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network. Adv. Photonics, 5, 016003(2023). https://doi.org/10.1117/1.AP.5.1.016003

[37] H. H. Zhu et al. Space-efficient optical computing with an integrated chip diffractive neural network. Nat. Commun., 13, 1044(2022). https://doi.org/10.1038/s41467-022-28702-0

[38] D. Mengu et al. Diffractive interconnects: all-optical permutation operation using diffractive networks. Nanophotonics, 12, 905-923(2023). https://doi.org/10.1515/nanoph-2022-0358

[39] J. X. Li et al. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light Sci. Appl., 11, 153(2022). https://doi.org/10.1038/s41377-022-00849-x

[40] J. K. Weng et al. Meta-neural-network for real-time and passive deep-learning-based object recognition. Nat. Commun., 11, 6309(2020). https://doi.org/10.1038/s41467-020-19693-x

[41] O. Kulce et al. All-optical information-processing capacity of diffractive surfaces. Light Sci. Appl., 10, 25(2021). https://doi.org/10.1038/s41377-020-00439-9

[42] X. Lin et al. All-optical machine learning using diffractive deep neural networks. Science, 361, 1004-1008(2018). https://doi.org/10.1126/science.aat8084

[43] Y. Rivenson et al. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl., 7, 17141(2018). https://doi.org/10.1038/lsa.2017.141

[44] M. Hermans et al. Trainable hardware for dynamical computing using error backpropagation through physical media. Nat. Commun., 6, 6729(2015). https://doi.org/10.1038/ncomms7729

[45] D. Ulyanov, A. Vedaldi, V. Lempitsky. Deep image prior. Int. J. Comput. Vision, 128, 1867-1888(2020). https://doi.org/10.1007/s11263-020-01303-4

[46] R. C. Li et al. Broadband achromatic metasurface holography. Opto-Electron. Eng., 50, 230118(2023). https://doi.org/10.12086/oee.2023.230118

[47] D. Huo et al. Blind image deconvolution using variational deep image prior. IEEE Trans. Pattern Anal., 45, 11472-11483(2023). https://doi.org/10.1109/TPAMI.2023.3283979

[48] F. Han et al. Deep image prior plus sparsity prior: toward single-shot full-Stokes spectropolarimetric imaging with a multiple-order retarder. Adv. Photonics Nexus, 2, 036009(2023). https://doi.org/10.1117/1.APN.2.3.036009

[49] D. Conteduca et al. Dielectric nanohole array metasurface for high-resolution near-field sensing and imaging. Nat. Commun., 12, 3293(2021). https://doi.org/10.1038/s41467-021-23357-9

Tools

Get Citation

Copy Citation Text

Min Huang, Bin Zheng, Ruichen Li, Yijun Zou, Xiaofeng Li, Chao Qian, Huan Lu, Rongrong Zhu, Hongsheng Chen, "Real-time all-directional 3D recognition and multidistortion correction via prior diffraction neural networks," Adv. Photon. 7, 056005 (2025)

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Research Articles

Received: Jan. 22, 2025

Accepted: Jul. 15, 2025

Published Online: Aug. 21, 2025

The Author Email: Rongrong Zhu (rorozhu@hzcu.edu.cn), Hongsheng Chen (hansomchen@zju.edu.cn)

DOI:10.1117/1.AP.7.5.056005

CSTR:32187.14.1.AP.7.5.056005

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology