Journals Articles Conferences News About CLP
Submit a paper Email Alert
Search
Search
Articles
Journals
News
Advanced Search
Top Searches
2D MaterialsTransformation opticsQuantum PhotonicsSmall lasersSemiconductor UV PhotonicsSupersymmetric microring laser arrays

Photonics Research, Volume. 8, Issue 6, 940(2020)

In situ optical backpropagation training of diffractive optical neural networks Spotlight on Optics

Tiankuang Zhou1,2,3、†, Lu Fang2,3、†, Tao Yan1,2, Jiamin Wu1,2, Yipeng Li1,2, Jingtao Fan1,2, Huaqiang Wu4,5, Xing Lin1,2,4,7、*, and Qionghai Dai1,2,6,8、*
Author Affiliations
  • 1Department of Automation, Tsinghua University, Beijing 100084, China
  • 2Institute for Brain and Cognitive Science, Tsinghua University, Beijing 100084, China
  • 3Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
  • 4Beijing Innovation Center for Future Chip, Tsinghua University, Beijing 100084, China
  • 5Institute of Microelectronics, Tsinghua University, Beijing 100084, China
  • 6Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
  • 7e-mail: lin-x@tsinghua.edu.cn
  • 8e-mail: qhdai@tsinghua.edu.cn
    • show less
    Full TextGet PDF
    Figures&Tables (10)
    Equations (15)
    References (54)
    Cited By (128)
    Tools
    Paper Information
    Topics
    References(54)

    [1] Y. LeCun, Y. Bengio, G. Hinton. Deep learning. Nature, 521, 436-444(2015).

    [2] A. Krizhevsky, I. Sutskever, G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 1097-1105(2012).

    [3] G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, B. Kingsbury. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag., 29, 82-97(2012).

    [4] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot. Mastering the game of go with deep neural networks and tree search. Nature, 529, 484-489(2016).

    [5] A. Esteva, A. Robicquet, B. Ramsundar, V. Kuleshov, M. DePristo, K. Chou, C. Cui, G. Corrado, S. Thrun, J. Dean. A guide to deep learning in healthcare. Nat. Med., 25, 24-29(2019).

    [6] G. Barbastathis, A. Ozcan, G. Situ. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [7] A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. N. Murthy, M. W. Mathis, M. Bethge. DeepLabCut: Markerless Pose Estimation of User-Defined Body Parts with Deep Learning(2018).

    [8] C. Trabelsi, O. Bilaniuk, Y. Zhang, D. Serdyuk, S. Subramanian, J. F. Santos, S. Mehri, N. Rostamzadeh, Y. Bengio, C. J. Pal. Deep complex networks(2017).

    [9] K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition, 770-778(2016).

    [10] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345, 668-673(2014).

    [11] J. Pei, L. Deng, S. Song, M. Zhao, Y. Zhang, S. Wu, G. Wang, Z. Zou, Z. Wu, W. He. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 572, 106-111(2019).

    [12] B. Marr, B. Degnan, P. Hasler, D. Anderson. Scaling energy per operation via an asynchronous pipeline. IEEE Trans. Very Large Scale Integr. Syst., 21, 147-151(2012).

    [13] J. M. Shainline, S. M. Buckley, R. P. Mirin, S. W. Nam. Superconducting optoelectronic circuits for neuromorphic computing. Phys. Rev. Appl., 7, 034013(2017).

    [14] P. R. Prucnal, B. J. Shastri. Neuromorphic Photonics(2017).

    [15] D. Woods, T. J. Naughton. Optical computing: photonic neural networks. Nat. Phys., 8, 257-259(2012).

    [16] D. R. Solli, B. Jalali. Analog optical computing. Nat. Photonics, 9, 704-706(2015).

    [17] Q. Zhang, H. Yu, M. Barbiero, B. Wang, M. Gu. Artificial neural networks enabled by nanophotonics. Light Sci. Appl., 8, 1(2019).

    [18] X. Luo. Engineering optics 2.0: a revolution in optical materials, devices, and systems. ACS Photon., 5, 4724-4738(2018).

    [19] P. Minzioni, C. Lacava, T. Tanabe, J. Dong, X. Hu, G. Csaba, W. Porod, G. Singh, A. E. Willner, A. Almaiman. Roadmap on all-optical processing. J. Opt., 21, 063001(2019).

    [20] Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund. Deep learning with coherent nanophotonic circuits. Nat. Photonics, 11, 441-446(2017).

    [21] T. W. Hughes, R. J. England, S. Fan. Reconfigurable photonic circuit for controlled power delivery to laser-driven accelerators on a chip. Phys. Rev. Appl., 11, 064014(2019).

    [22] X. Lin, Y. Rivenson, N. T. Yardimci, M. Veli, Y. Luo, M. Jarrahi, A. Ozcan. All-optical machine learning using diffractive deep neural networks. Science, 361, 1004-1008(2018).

    [23] T. Yan, J. Wu, T. Zhou, H. Xie, F. Xu, J. Fan, L. Fang, X. Lin, Q. Dai. Fourier-space diffractive deep neural network. Phys. Rev. Lett., 123, 023901(2019).

    [24] G. Van der Sande, D. Brunner, M. C. Soriano. Advances in photonic reservoir computing. Nanophotonics, 6, 561-576(2017).

    [25] L. Larger, A. Baylón-Fuentes, R. Martinenghi, V. S. Udaltsov, Y. K. Chembo, M. Jacquot. High-speed photonic reservoir computing using a time-delay-based architecture: million words per second classification. Phys. Rev. X, 7, 011015(2017).

    [26] J. Feldmann, N. Youngblood, C. Wright, H. Bhaskaran, W. Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 569, 208-214(2019).

    [27] R. Hamerly, L. Bernstein, A. Sludds, M. Soljačić, D. Englund. Large-scale optical neural networks based on photoelectric multiplication. Phys. Rev. X, 9, 021032(2019).

    [28] I. Chakraborty, G. Saha, K. Roy. Photonic in-memory computing primitive for spiking neural networks using phase-change materials. Phys. Rev. Appl., 11, 014063(2019).

    [29] T. Deng, J. Robertson, Z.-M. Wu, G.-Q. Xia, X.-D. Lin, X. Tang, Z.-J. Wang, A. Huartado. Stable propagation of inhibited spiking dynamics in vertical-cavity surface-emitting lasers for neuromorphic photonic networks. IEEE Access, 6, 67951-67958(2018).

    [30] J. Robertson, T. Deng, J. Javaloyes, A. Hurtado. Controlled inhibition of spiking dynamics in VCSELs for neuromorphic photonics: theory and experiments. Opt. Lett., 42, 1560-1563(2017).

    [31] T. W. Hughes, I. A. Williamson, M. Minkov, S. Fan. Wave physics as an analog recurrent neural network(2019).

    [32] J. Bueno, S. Maktoobi, L. Froehly, I. Fischer, M. Jacquot, L. Larger, D. Brunner. Reinforcement learning in a large-scale photonic recurrent neural network. Optica, 5, 756-760(2018).

    [33] E. Khoram, A. Chen, D. Liu, L. Ying, Q. Wang, M. Yuan, Z. Yu. Nanophotonic media for artificial neural inference. Photon. Res., 7, 823-827(2019).

    [34] A. S. Backer. Computational inverse design for cascaded systems of metasurface optics(2019).

    [35] S. Maktoobi, L. Froehly, L. Andreoli, X. Porte, M. Jacquot, L. Larger, D. Brunner. Diffractive coupling for photonic networks: how big can we go?. IEEE J. Sel. Top. Quantum Electron., 26, 7600108(2019).

    [36] Y. Zuo, B. Li, Y. Zhao, Y. Jiang, Y.-C. Chen, P. Chen, G.-B. Jo, J. Liu, S. Du. All optical neural network with nonlinear activation functions. Optica, 6, 1132-1137(2019).

    [37] J. Chang, V. Sitzmann, X. Dun, W. Heidrich, G. Wetzstein. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep., 8, 12324(2018).

    [38] H. Chen, S. Jayasuriya, J. Yang, J. Stephen, S. Sivaramakrishnan, A. Veeraraghavan, A. Molnar. ASP vision: optically computing the first layer of convolutional neural networks using angle sensitive pixels. IEEE Conference on Computer Vision and Pattern Recognition, 903-912(2016).

    [39] Y. Luo, D. Mengu, N. T. Yardimci, Y. Rivenson, M. Veli, M. Jarrahi, A. Ozcan. Design of task-specific optical systems using broadband diffractive neural networks(2019).

    [40] J. Chang, G. Wetzstein. Deep optics for monocular depth estimation and 3D object detection(2019).

    [41] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86, 2278-2324(1998).

    [42] T. W. Hughes, M. Minkov, Y. Shi, S. Fan. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica, 5, 864-871(2018).

    [43] M. Hermans, J. Dambre, P. Bienstman. Optoelectronic systems trained with backpropagation through time. IEEE Trans. Neural Netw. Learn. Syst., 26, 1545-1550(2014).

    [44] M. Hermans, M. Burm, T. Van Vaerenbergh, J. Dambre, P. Bienstman. Trainable hardware for dynamical computing using error backpropagation through physical media. Nat. Commun., 6, 6729(2015).

    [45] K. Wagner, P. Demetri. Multilayer optical learning networks. Appl. Opt., 26, 5061-5076(1987).

    [46] D. Psaltis, D. Brady, K. Wagner. Adaptive optical networks using photorefractive crystals. Appl. Opt., 27, 1752-1759(1988).

    [47] I. Yamaguchi, T. Zhang. Phase-shifting digital holography. Opt. Lett., 22, 1268-1270(1997).

    [48] O. Mendoza-Yero, G. Mínguez-Vega, J. Lancis. Encoding complex fields by using a phase-only optical element. Opt. Lett., 39, 1740-1743(2014).

    [49] M. W. Matthès, P. del Hougne, J. de Rosny, G. Lerosey, S. M. Popoff. Optical complex media as universal reconfigurable linear operators. Optica, 6, 465-472(2019).

    [50] A. P. Mosk, A. Lagendijk, G. Lerosey, M. Fink. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photonics, 6, 283-292(2012).

    [51] Y. Li, Y. Xue, L. Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

    [52] N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng, L. Waller. DiffuserCam: lensless single-exposure 3D imaging. Optica, 5, 1-9(2018).

    [53] G. K. Shirmanesh, R. Sokhoyan, P. C. Wu, H. A. Atwater. Electro-optically tunable universal metasurfaces(2019).

    [54] Z. Wang, C. Li, P. Lin, M. Rao, Y. Nie, W. Song, Q. Qiu, Y. Li, P. Yan, J. P. Strachan. In situ training of feed-forward and recurrent convolutional memristor networks. Nat. Mach. Intell., 1, 434-442(2019).

    CLP Journals

    [1] Qiuquan Yan, Qinghui Deng, Jun Zhang, Ying Zhu, Ke Yin, Teng Li, Dan Wu, Tian Jiang, "Low-latency deep-reinforcement learning algorithm for ultrafast fiber lasers," Photonics Res. 9, 1493 (2021)

    [2] Xianxin Guo, Thomas D. Barrett, Zhiming M. Wang, A. I. Lvovsky, "Backpropagation through nonlinear units for the all-optical training of neural networks," Photonics Res. 9, B71 (2021)

    [3] Tiankuang Zhou, Lu Fang, Tao Yan, Jiamin Wu, Yipeng Li, Jingtao Fan, Huaqiang Wu, Xing Lin, Qionghai Dai, "In situ optical backpropagation training of diffractive optical neural networks: publisher’s note," Photonics Res. 8, 1323 (2020)

    Tools

    Get Citation

    Copy Citation Text

    Tiankuang Zhou, Lu Fang, Tao Yan, Jiamin Wu, Yipeng Li, Jingtao Fan, Huaqiang Wu, Xing Lin, Qionghai Dai, "In situ optical backpropagation training of diffractive optical neural networks," Photonics Res. 8, 940 (2020)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Set citation alerts for article
    Save article for my favorites
    Paper Information

    Category: Physical Optics

    Received: Feb. 4, 2020

    Accepted: Mar. 27, 2020

    Published Online: May. 20, 2020

    The Author Email: Xing Lin (lin-x@tsinghua.edu.cn), Qionghai Dai (qhdai@tsinghua.edu.cn)

    DOI:10.1364/PRJ.389553

    Topics

    laser devices and laser physics
    Lasers and Laser Optics
    Laser physics
    laser manufacturing
    Instrumentation, Measurement and Metrology