Chinese Journal of Lasers, Volume. 51, Issue 1, 0119002(2024)

Review of On‑Chip Integrated Optical Neural Networks (Invited)

Tingzhao Fu1,4,5, Run Sun2,3, Yuyao Huang2,3, Jianfa Zhang1,4,5, Sigang Yang2,3, Zhihong Zhu1,4,5, and Hongwei Chen2,3、*
Author Affiliations
  • 1College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha 410073, Hunan, China
  • 2Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
  • 3Beijing National Research Center for Information Science and Technology, Beijing 100084, China
  • 4Hunan Provincial Key Laboratory of Novel Nano-Optoelectronic Information Materials and Devices, National University of Defense Technology, Changsha 410073, Hunan, China
  • 5Nanhu Laser Laboratory, National University of Defense Technology, Changsha 410073, Hunan, China
  • show less
    References(86)

    [1] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 60, 84-90(2017).

    [2] LeCun Y, Bottou L, Bengio Y et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 86, 2278-2324(1998).

    [3] Ananthanarayana T, Srivastava P, Chintha A et al. Deep learning methods for sign language translation[J]. ACM Transactions on Accessible Computing, 14, 22(2021).

    [4] Graves A, Mohamed A R, Hinton G. Speech recognition with deep recurrent neural networks[C], 6645-6649(2013).

    [5] Rausch V, Hansen A, Solowjow E et al. Learning a deep neural net policy for end-to-end control of autonomous vehicles[C], 4914-4919(2017).

    [6] Al-Qizwini M, Barjasteh I, Al-Qassab H et al. Deep learning algorithm for autonomous driving using GoogLeNet[C], 89-96(2017).

    [7] Litjens G, Kooi T, Bejnordi B E et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 42, 60-88(2017).

    [8] Baldi P. Deep learning in biomedical data science[J]. Annual Review of Biomedical Data Science, 1, 181-205(2018).

    [9] Apostolidis K D, Papakostas G A. A survey on adversarial deep learning robustness in medical image analysis[J]. Electronics, 10, 2132(2021).

    [10] Liu Y Q, Qian K, Wang K et al. Effective scaling of blockchain beyond consensus innovations and Moore’s law: challenges and opportunities[J]. IEEE Systems Journal, 16, 1424-1435(2022).

    [11] Lin X, Rivenson Y, Yardimci N T et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 361, 1004-1008(2018).

    [12] Chang J L, Sitzmann V, Dun X et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification[J]. Scientific Reports, 8, 12324(2018).

    [13] Yan T, Wu J M, Zhou T K et al. Fourier-space diffractive deep neural network[J]. Physical Review Letters, 123, 023901(2019).

    [14] Qian C, Lin X, Lin X B et al. Performing optical logic operations by a diffractive neural network[J]. Light: Science & Applications, 9, 59(2020).

    [15] Miscuglio M, Hu Z B, Li S R et al. Massively parallel amplitude-only Fourier neural network[J]. Optica, 7, 1812-1819(2020).

    [16] Dou H K, Deng Y E, Yan T et al. Residual D2NN: training diffractive deep neural networks via learnable light shortcuts[J]. Optics Letters, 45, 2688-2691(2020).

    [17] Zhou T K, Lin X, Wu J M et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit[J]. Nature Photonics, 15, 367-373(2021).

    [18] Wang P P, Xiong W J, Huang Z B et al. Orbital angular momentum mode logical operation using optical diffractive neural network[J]. Photonics Research, 9, 2116-2124(2021).

    [19] Mengu D, Zhao Y F, Tabassum A et al. Diffractive interconnects: all-optical permutation operation using diffractive networks[J]. Nanophotonics, 12, 905-923(2023).

    [20] Chen H L, Huang L Z, Liu T R et al. Fourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior external generalization[J]. Light: Science & Applications, 11, 254(2022).

    [21] Shi W X, Huang Z, Huang H H et al. LOEN: Lensless opto-electronic neural network empowered machine vision[J]. Light: Science & Applications, 11, 121(2022).

    [22] Li J X, Gan T Y, Bai B J et al. Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network[J]. Advanced Photonics, 5, 016003(2023).

    [23] Rahman M S S, Yang X L, Li J X et al. Universal linear intensity transformations using spatially incoherent diffractive processors[J]. Light: Science & Applications, 12, 195(2023).

    [24] Tait A N, Nahmias M A, Shastri B J et al. Broadcast and weight: an integrated network for scalable photonic spike processing[J]. Journal of Lightwave Technology, 32, 4029-4041(2014).

    [25] Shen Y C, Harris N C, Skirlo S et al. Deep learning with coherent nanophotonic circuits[J]. Nature Photonics, 11, 441-446(2017).

    [26] Tait A N, de Lima T F, Zhou E et al. Neuromorphic photonic networks using silicon photonic weight banks[J]. Scientific Reports, 7, 7430(2017).

    [27] Hughes T W, Minkov M, Shi Y U et al. Training of photonic neural networks through in situ backpropagation and gradient measurement[J]. Optica, 5, 864-871(2018).

    [29] Peng H T, Nahmias M A, Lima T F D et al. Neuromorphic photonic integrated circuits[J]. IEEE Journal of Selected Toptics in Quantum Electronics, 24, 6101715(2018).

    [30] Williamson I A D, Hughes T W, Minkov M et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 26, 7700412(2020).

    [31] Feldmann J, Youngblood N, Wright C D et al. All-optical spiking neurosynaptic networks with self-learning capabilities[J]. Nature, 569, 208-214(2019).

    [32] Tait A N, Ferreira de Lima T, Nahmias M A et al. Silicon photonic modulator neuron[J]. Physical Review Applied, 11, 064043(2019).

    [33] Hughes T W, Williamson I A D, Minkov M et al. Wave physics as an analog recurrent neural network[J]. Science Advances, 5, eaay6946(2019).

    [34] Shuhei O, Kasidit T, Shinichi T et al. Si microring resonator crossbar arrays for deep learning accelerator[J]. Japanese Journal of Applied Physics, 59, SGGE04(2020).

    [35] Zarei S, Marzban M R, Khavasi A. Integrated photonic neural network based on silicon metalines[J]. Optics Express, 28, 36668-36684(2020).

    [36] Qu Y R, Zhu H Z, Shen Y C et al. Inverse design of an integrated-nanophotonics optical neural network[J]. Science Bulletin, 65, 1177-1183(2020).

    [37] Shokraneh F, Geoffroy-Gagnon S, Liboiron-Ladouceur O. The diamond mesh, a phase-error-and loss-tolerant field-programmable MZI-based optical processor for optical neural networks[J]. Optics Express, 28, 23495-23508(2020).

    [38] Ong J R, Ooi C C, Ang T Y L et al. Photonic convolutional neural networks using integrated diffractive optics[J]. IEEE Journal of Selected Topics in Quantum Electronics, 26, 7702108(2020).

    [40] Deng H Q, Khajavikhan M. Parity-time symmetric optical neural networks[J]. Optica, 8, 1328-1333(2021).

    [41] Zhang H, Gu M, Jiang X D et al. An optical neural chip for implementing complex-valued neural network[J]. Nature Communications, 12, 457(2021).

    [42] Feldmann J, Youngblood N, Karpov M et al. Parallel convolutional processing using an integrated photonic tensor core[J]. Nature, 589, 52-58(2021).

    [43] Huang C R, Fujisawa S, de Lima T F et al. A silicon photonic-electronic neural network for fibre nonlinearity compensation[J]. Nature Electronics, 4, 837-844(2021).

    [44] Fu T Z, Zang Y B, Huang H H et al. On-chip photonic diffractive optical neural network based on a spatial domain electromagnetic propagation model[J]. Optics Express, 29, 31924-31940(2021).

    [45] Goi E, Chen X, Zhang Q M et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip[J]. Light: Science & Applications, 10, 40(2021).

    [46] Shi Y, Ren J Y, Chen G Y et al. Nonlinear germanium-silicon photodiode for activation and monitoring in photonic neuromorphic networks[J]. Nature Communications, 13, 6048(2022).

    [47] Zhu H H, Zou J, Zhang H et al. Space-efficient optical computing with an integrated chip diffractive neural network[J]. Nature Communications, 13, 1044(2022).

    [48] Wang X Y, Xie P, Chen B H et al. Chip-based high-dimensional optical neural network[J]. Nano-Micro Letters, 14, 221(2022).

    [49] Xu S F, Wang J, Yi S C et al. High-order tensor flow processing using integrated photonic circuits[J]. Nature Communications, 13, 7970(2022).

    [50] Wang Z, Chang L, Wang F F et al. Integrated photonic metasystem for image classifications at telecommunication wavelength[J]. Nature Communications, 13, 2131(2022).

    [51] Zarei S, Khavasi A. Realization of optical logic gates using on-chip diffractive optical neural networks[J]. Scientific Reports, 12, 15747(2022).

    [52] Yan T, Yang R, Zheng Z Y et al. All-optical graph representation learning using integrated diffractive photonic computing units[J]. Science Advances, 8, eabn7630(2022).

    [53] Ashtiani F, Geers A J, Aflatouni F. An on-chip photonic deep neural network for image classification[J]. Nature, 606, 501-506(2022).

    [54] Huang Y, Wang W P, Qiao L et al. Programmable low-threshold optical nonlinear activation functions for photonic neural networks[J]. Optics Letters, 47, 1810-1813(2022).

    [55] Liao K, Li C T, Dai T X et al. Matrix eigenvalue solver based on reconfigurable photonic neural network[J]. Nanophotonics, 11, 4089-4099(2022).

    [56] Luo X H, Hu Y Q, Ou X N et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible[J]. Light: Science & Applications, 11, 158(2022).

    [57] Ohno S, Tang R, Toprasertpong K et al. Si microring resonator crossbar array for on-chip inference and training of the optical neural network[J]. ACS Photonics, 9, 2614-2622(2022).

    [58] Zhang W P, Huang C R, Peng H T et al. Silicon microring synapses enable photonic deep learning beyond 9-bit precision[J]. Optica, 9, 579-584(2022).

    [59] Bai B W, Yang Q P, Shu H W et al. Microcomb-based integrated photonic processing unit[J]. Nature Communications, 14, 66(2023).

    [60] Fu T Z, Zang Y B, Huang Y Y et al. Photonic machine learning with on-chip diffractive optics[J]. Nature Communications, 14, 70(2023).

    [61] Fu T Z, Huang Y Y, Sun R et al. Integrated diffractive optical neural network with space-time interleaving[J]. Chinese Optics Letters, 21, 091301(2023).

    [62] Huang Y, Yue H S, Ma W et al. Easily scalable photonic tensor core based on tunable units with single internal phase shifters[J]. Laser & Photonics Reviews, 17, 2300001(2023).

    [63] Wei M L, Li J Y, Chen Z Q et al. Electrically programmable phase-change photonic memory for optical neural networks with nanoseconds in situ training capability[J]. Advanced Photonics, 5, 046004(2023).

    [64] Huang Y Y, Fu T Z, Huang H H et al. Sophisticated deep learning with on-chip optical diffractive tensor processing[J]. Photonics Research, 11, 1125-1138(2023).

    [65] Liu W C, Fu T Z, Huang Y Y et al. C-DONN: compact diffractive optical neural network with deep learning regression[J]. Optics Express, 31, 22127-22143(2023).

    [66] Poordashtban O, Marzabn M R, Khavasi A. Integrated photonic convolutional neural network based on silicon metalines[J]. IEEE Access, 11, 61728-61737(2023).

    [67] Meng X Y, Zhang G J, Shi N N et al. Compact optical convolution processing unit based on multimode interference[J]. Nature Communications, 14, 3000(2023).

    [68] Wang J H, Rodrigues S P, Dede E M et al. Microring-based programmable coherent optical neural networks[J]. Optics Express, 31, 18871-18887(2023).

    [69] Pai S, Sun Z H, Hughes T W et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks[J]. Science, 380, 398-404(2023).

    [70] Giamougiannis G, Tsakyridis A, Moralis-Pegios M et al. Neuromorphic silicon photonics with 50 GHz tiled matrix multiplication for deep-learning applications[J]. Advanced Photonics, 5, 016004(2023).

    [71] Xu X Y, Tan M X, Corcoran B et al. 11 TOPS photonic convolutional accelerator for optical neural networks[J]. Nature, 589, 44-51(2021).

    [72] Stelzer F, Röhm A, Vicente R et al. Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops[J]. Nature Communications, 12, 5164(2021).

    [73] Zhang L H, Li C Y, He J Y et al. Optical machine learning using time-lens deep neural networks[J]. Photonics, 8, 78(2021).

    [74] Chen H W, Yu Z M, Zhang T et al. Advances and challenges of optical neural networks[J]. Chinese Journal of Lasers, 47, 0500004(2020).

    [75] Cheng J W, Jiang X Y, Zhou H L et al. Advances and challenges of optoelectronic intelligent computing[J]. Chinese Journal of Lasers, 49, 1219001(2022).

    [76] Chen B, Zhang Z Y, Dai T G et al. Photonic neural networks and its applications[J]. Laser & Optoelectronics Progress, 60, 0600001(2023).

    [77] Reck M, Zeilinger A, Bernstein H J et al. Experimental realization of any discrete unitary operator[J]. Physical Review Letters, 73, 58-61(1994).

    [78] Clements W R, Humphreys P C, Metcalf B J et al. Optimal design for universal multiport interferometers[J]. Optica, 3, 1460-1465(2016).

    [79] Ribeiro A, Ruocco A, Vanacker L et al. Demonstration of a 4×4-port universal linear circuit[J]. Optica, 3, 1348-1357(2016).

    [80] Tait A N, Wu A X, de Lima T F et al. Microring weight banks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 22, 312-325(2016).

    [81] Tait A N, Jayatilleka H, De Lima T F et al. Feedback control for microring weight banks[J]. Optics Express, 26, 26422-26443(2018).

    [82] Yu N F, Genevet P, Kats M A et al. Light propagation with phase discontinuities: generalized laws of reflection and refraction[J]. Science, 334, 333-337(2011).

    [83] Zhang J J, Yang J B, Xin H et al. Ultrashort and efficient adiabatic waveguide taper based on thin flat focusing lenses[J]. Optics Express, 25, 19894-19903(2017).

    [84] Wang Z, Li T T, Soman A et al. On-chip wavefront shaping with dielectric metasurface[J]. Nature Communications, 10, 3547(2019).

    [85] Moughames J, Porte X, Thiel M et al. Three-dimensional waveguide interconnects for scalable integration of photonic neural networks[J]. Optica, 7, 640-646(2020).

    [86] Sludds A, Bandyopadhyay S, Chen Z J et al. Delocalized photonic deep learning on the internet’s edge[J]. Science, 378, 270-276(2022).

    Tools

    Get Citation

    Copy Citation Text

    Tingzhao Fu, Run Sun, Yuyao Huang, Jianfa Zhang, Sigang Yang, Zhihong Zhu, Hongwei Chen. Review of On‑Chip Integrated Optical Neural Networks (Invited)[J]. Chinese Journal of Lasers, 2024, 51(1): 0119002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Sep. 21, 2023

    Accepted: Oct. 24, 2023

    Published Online: Jan. 19, 2024

    The Author Email: Chen Hongwei (chenhw@tsinghua.edu.cn)

    DOI:10.3788/CJL231227

    CSTR:32183.14.CJL231227

    Topics