Chinese Journal of Lasers, Volume. 48, Issue 19, 1918004(2021)
Advances in Computational Optics Based on Deep Learning
[1] Al-Qizwini M, Barjasteh I, Al-Qassab H et al. Deep learning algorithm for autonomous driving using GoogLeNet[C]. //2017 IEEE Intelligent Vehicles Symposium (IV), June 11-14, 2017, Los Angeles, CA, USA., 89-96(2017).
[2] Kotz S A, Cappa S F, von Cramon D Y et al. Modulation of the lexical-semantic network by auditory semantic priming: an event-related functional MRI study[J]. NeuroImage, 17, 1761-1772(2002).
[3] Bui H M, Lech M, Cheng E et al. Object recognition using deep convolutional features transformed by a recursive network structure[J]. IEEE Access, 4, 10059-10066(2016).
[4] Leibig C, Allken V, Ayhan M S et al. Leveraging uncertainty information from deep neural networks for disease detection[J]. Scientific Reports, 7, 17816(2017).
[5] Rashevsky N. Some remarks on the Boolean algebra of nervous nets in mathematical biophysics[J]. The Bulletin of Mathematical Biophysics, 7, 203-211(1945).
[6] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psychological Review, 65, 386-408(1958).
[7] Cortes C, Vapnik V. Support-vector networks[J]. Machine Learning, 20, 273-297(1995).
[8] Smirnov E A, Timoshenko D M, Andrianov S N. Comparison of regularization methods for ImageNet classification with deep convolutional neural networks[J]. AASRI Procedia, 6, 89-94(2014).
[9] Tran D, Bourdev L, Fergus R et al. Learning spatiotemporal features with 3D convolutional networks[C]. //2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile., 4489-4497(2015).
[10] Ledig C, Theis L, Huszár F et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. //2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA., 105-114(2017).
[11] Genty G, Salmela L, Dudley J M et al. Machine learning and applications in ultrafast photonics[J]. Nature Photonics, 15, 91-101(2021).
[12] Wiecha P R, Arbouet A, Girard C et al. Deep learning in nano-photonics: inverse design and beyond[J]. Photonics Research, 9, B182-B200(2021).
[13] Liu Z, Zhu D, Raju L et al. Tackling photonic inverse design with machine learning[J]. Advanced Science, 8, 2002923(2021).
[15] Dardikman-Yoffe G, Eldar Y C . Learned SPARCOM: unfolded deep super-resolution microscopy[J]. Optics Express, 28, 27736-27763(2020).
[16] Dong C, Loy C C, He K M et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307(2016).
[17] Dong Xu, Yu Luo, Jun Luo et al. Efficient design of a dielectric metasurface with transfer learning and genetic algorithm[J]. Optical Materials Express, 11, 1852-1862(2021).
[18] Jiao L C, Zhao J. A survey on the new generation of deep learning in image processing[J]. IEEE Access, 7, 172231-172263(2019).
[19] Horisaki R, Takagi R, Tanida J. Deep-learning-generated holography[J]. Applied Optics, 57, 3859-3863(2018).
[20] Qu Y R, Zhu H Z, Shen Y C et al. Inverse design of an integrated-nanophotonics optical neural network[J]. Science Bulletin, 65, 1177-1183(2020).
[21] Oskooi A F, Roundy D, Ibanescu M et al. Meep: a flexible free-software package for electromagnetic simulations by the FDTD method[J]. Computer Physics Communications, 181, 687-702(2010).
[22] Deslandes D, Wu K. Integrated microstrip and rectangular waveguide in planar form[J]. IEEE Microwave and Wireless Components Letters, 11, 68-70(2001).
[23] Kogelnik H. Coupled wave theory for thick hologram gratings[J]. The Bell System Technical Journal, 48, 2909-2947(1969).
[24] Wiecha P R, Muskens O L. Deep learning meets nanophotonics: a generalized accurate predictor for near fields and far fields of arbitrary 3D nano-structures[J]. Nano Letters, 20, 329-338(2020).
[25] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[M]. //Navab N, Hornegger J, Wells W M, et al. Medical image computing and computer-assisted intervention-MICCAI 2015. Lecture notes in computer science, 9351, 234-241(2015).
[26] Rudy S H, Brunton S L, Proctor J L et al. Data-driven discovery of partial differential equations[J]. Science Advances, 3, e1602614(2017).
[27] Liu D J, Tan Y X, Khoram E et al. Training deep neural networks for the inverse design of nanophotonic structures[C]. //2019 Conference on Lasers and Electro-Optics (CLEO), May 5-10, 2019, San Jose, California, United States, 7b01377(2019).
[28] Peurifoy J, Shen Y, Jing L et al. Nanophotonic particle simulation and inverse design using artificial neural networks[J]. Science Advances, 4, eaar4206(2018).
[29] Ma W, Cheng F, Liu Y. Deep-learning-enabled on-demand design of chiral metamaterials[J]. ACS Nano, 12, 6326-6334(2018).
[30] Malkiel I, Mrejen M, Nagler A et al. Plasmonic nano-structure design and characterization via Deep Learning[J]. Light, Science & Applications, 7, 60(2018).
[31] An S S, Fowler C, Zheng B W et al. A deep learning approach for objective-driven all-dielectric metasurface design[J]. ACS Photonics, 6, 3196-3207(2019).
[32] Wang H, Jia W, Wang J et al. Learning graph representation with generative adversarial nets[J]. IEEE Transactions on Knowledge and Data Engineering, 33, 3090-3103(2021).
[33] Liu C, Yu W M, Ma Q et al. Intelligent coding metasurface holograms by physics-assisted unsupervised generative adversarial network[J]. Photonics Research, 9, B159-B167(2021).
[34] Liu Z, Zhu D, Rodrigues S P et al. Generative model for the inverse design of metasurfaces[J]. Nano Letters, 18, 6570-6576(2018).
[35] Zhu D, Liu Z, Raju L et al. Building multifunctional metasystems via algorithmic construction[J]. ACS Nano, 15, 2318-2326(2021).
[36] Ma W, Cheng F, Xu Y et al. Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy[J]. Advanced Materials, 31, e1901111(2019).
[37] Kiarashinejad Y, Abdollahramezani S, Adibi A. Deep learning approach based on dimensionality reduction for designing electromagnetic nano-structures[J]. Npj Computational Materials, 6, 12(2020).
[38] Jorge L P, Meeks W H. The topology of complete minimal surfaces of finite total Gaussian curvature[J]. Topology, 22, 203-221(1983).
[39] Camayd-Muñoz P, Ballew C, Roberts G et al. Multifunctional volumetric meta-optics for color and polarization image sensors[J]. Optica, 7, 280-283(2020).
[41] Augenstein Y, Rockstuhl C. Inverse design of nanophotonic devices with structural integrity[J]. ACS Photonics, 7, 2190-2196(2020).
[42] Zhan A, Gibson R, Whitehead J et al. Controlling three-dimensional optical fields via inverse Mie scattering[J]. Science Advances, 5, eaax4769(2019).
[43] Mansouree M, McClung A, Samudrala S et al. Large-scale parametrized metasurface design using adjoint optimization[J]. ACS Photonics, 8, 455-463(2021).
[44] Chen M, Jiang J, Fan J. Design space reparameterization enforces hard geometric constraints in inverse-designed nanophotonic devices[J]. ACS Photonics, 7, 3141-3151(2020).
[45] Kubota T. 48 years with holography[J]. Optical Review, 21, 883-892(2014).
[46] Wan M, Healy J J, Sheridan J T. Terahertz phase imaging and biomedical applications[J]. Optics & Laser Technology, 122, 105859(2020).
[47] Cuche E, Marquet P, Depeursinge C. Spatial filtering for zero-order and twin-image elimination in digital off-axis holography[J]. Applied Optics, 39, 4070-4075(2000).
[48] Tanaka K, Hara M, Tokuyama K et al. Improved performance in coaxial holographic data recording[J]. Optics Express, 15, 16196-16209(2007).
[49] Tanaka Y, Tani S, Murata S. Phase retrieval method for digital holography with two cameras in particle measurement[J]. Optics Express, 24, 25233-25241(2016).
[50] Zhou W J, Guan X F, Liu F F et al. Phase retrieval based on transport of intensity and digital holography[J]. Applied Optics, 57, A229-A234(2017).
[51] Khan A, Zhang Z J, Yu Y J et al. GAN-holo: generative adversarial networks-based generated holography using deep learning[J]. Complexity, 2021, 1-7(2021).
[52] Rivenson Y, Zhang Y, Günaydın H et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light, Science & Applications, 7, 17141(2018).
[53] Schnars U, J ptner W P O. Digital recording and numerical reconstruction of holograms[J]. Measurement Science and Technology, 13, R85-R101(2002).
[54] Chang C, Xia J, Yang L et al. Speckle-suppressed phase-only holographic three-dimensional display based on double-constraint Gerchberg-Saxton algorithm[J]. Applied Optics, 54, 6994-7001(2015).
[55] Jin X Y, Gui J B, Liu C et al. Progress of fast generation algorithm of computer-generated hologram based on point source model[J]. Laser & Optoelectronics Progress, 55, 100005(2018).
[56] Shi L, Li B, Kim C et al. Towards real-time photorealistic 3D holography with deep neural networks[J]. Nature, 591, 234-239(2021).
[57] Wu J, Liu K, Sui X et al. High-speed computer-generated holography using an autoencoder-based deep neural network[J]. Optics Letters, 46, 2908-2911(2021).
[58] Edgar M P, Gibson G M, Padgett M J. Principles and prospects for single-pixel imaging[J]. Nature Photonics, 13, 13-20(2019).
[59] Duarte M F, Davenport M A, Takhar D et al. Single-pixel imaging via compressive sampling[J]. IEEE Signal Processing Magazine, 25, 83-91(2008).
[60] Zhang Z, Ma X, Zhong J. Single-pixel imaging by means of Fourier spectrum acquisition[J]. Nature Communications, 6, 6225(2015).
[61] Lyu M, Wang W, Wang H et al. Deep-learning-based ghost imaging[J]. Scientific Reports, 7, 17865(2017).
[62] He Y, Wang G, Dong G et al. Ghost imaging based on deep learning[J]. Scientific Reports, 8, 6469(2018).
[63] Rizvi S, Cao J, Zhang K Y et al. Deringing and denoising in extremely under-sampled Fourier single pixel imaging[J]. Optics Express, 28, 7360-7374(2020).
[64] Higham C F, Murray-Smith R, Padgett M J et al. Deep learning for real-time single-pixel video[J]. Scientific Reports, 8, 2369(2018).
[65] Radwell N, Johnson S D, Edgar M P et al. Deep learning optimized single-pixel LiDAR[J]. Applied Physics Letters, 115, 231101(2019).
[66] Wang F, Wang H, Wang H C et al. Learning from simulation: an end-to-end deep-learning approach for computational ghost imaging[J]. Optics Express, 27, 25560-25572(2019).
[67] Wu H, Wang R Z, Zhao G P et al. Sub-Nyquist computational ghost imaging with deep learning[J]. Optics Express, 28, 3846-3853(2020).
[68] Luo Z, Yurt A, Stahl R et al. Pixel super-resolution for lens-free holographic microscopy using deep learning neural networks[J]. Optics Express, 27, 13581(2019).
[69] Liu X Y, Wang Y H, Liu Q J. Remote sensing image fusion based on two-stream fusion network[M]. //Schoeffmann K, Chalidabhongse T H, Ngo C W, et al. MultiMedia modeling. Lecture notes in computer science, 10704, 428-439(2018).
[70] Liu Y, Chen X, Peng H et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 36, 191-207(2017).
[71] Ma J Y, Yu W, Liang P W et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 48, 11-26(2019).
[72] Naik N, Zhao S, Velten A et al. Single view reflectance capture using multiplexed scattering and time-of-flight imaging[C]. //Proceedings of the 2011 SIGGRAPH Asia Conference on - SA ’11, December 12-15, 2011, Hong Kong, China., 1-10(2011).
[73] Repasi E, Lutzmann P, Steinvall O et al. Advanced short-wavelength infrared range-gated imaging for ground applications in monostatic and bistatic configurations[J]. Applied Optics, 48, 5956-5969(2009).
[74] Gariepy G, Tonolini F, Henderson R et al. Detection and tracking of moving objects hidden from view[J]. Nature Photonics, 10, 23-26(2016).
[75] Vellekoop I M, Lagendijk A, Mosk A P. Exploiting disorder for perfect focusing[J]. Nature Photonics, 4, 320-322(2010).
[76] Judkewitz B, Horstmeyer R, Vellekoop I M et al. Translation correlations in anisotropically scattering media[J]. Nature Physics, 11, 684-689(2015).
[77] Li L, Li Q, Sun S et al. Imaging through scattering layers exceeding memory effect range with spatial-correlation-achieved point-spread-function[J]. Optics Letters, 43, 1670-1673(2018).
[78] Li Y Z, Xue Y J, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media[J]. Optica, 5, 1181-1190(2018).
[79] Yaqoob Z, Psaltis D, Feld M S et al. Optical phase conjugation for turbidity suppression in biological samples[J]. Nature Photonics, 2, 110-115(2008).
[81] Schott S, Bertolotti J, Léger J F et al. Characterization of the angular memory effect of scattered light in biological tissues[J]. Optics Express, 23, 13505-13516(2015).
[82] Lyu M, Wang H, Li G W et al. Learning-based lensless imaging through optically thick scattering media[J]. Advanced Photonics, 1, 036002(2019).
[83] Zhou Z Y, Xia J, Wu J et al. Learning-based phase imaging using a low-bit-depth pattern[J]. Photonics Research, 8, 1624-1633(2020).
[84] Rivenson Y, Göröcs Z, Günaydin H et al. Deep learning microscopy[J]. Optica, 4, 1437-1443(2017).
[85] Chen X, Li Y, Wyman N et al. Deep learning provides high accuracy in automated chondrocyte viability assessment in articular cartilage using nonlinear optical microscopy[J]. Biomedical Optics Express, 12, 2759-2722(2021).
[86] Zhang H, Cao L C, Jin G F et al. Progress on lensless digital holography imaging based on compressive holographic algorithm[J]. Laser & Optoelectronics Progress, 57, 080001(2020).
[88] Moon I, Jaferzadeh K, Kim Y et al. Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network[J]. Optics Express, 28, 26284-26301(2020).
[89] Zhang H, Liu S W, Cao L C et al. Noise suppression for ballistic-photons based on compressive in-line holographic imaging through an inhomogeneous medium[J]. Optics Express, 28, 10337-10349(2020).
[90] Zaharia M, Chowdhury M, Das T et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing[C]. // Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, April 25-27, 2012, San Jose, CA, USA(2012).
[91] Schwabe R J, Zelinger S, Key T S et al. Electronic lighting interference[J]. IEEE Industry Applications Magazine, 4, 43-48(1998).
[92] Hu W, Li X, Yang J et al. Crosstalk analysis of aligned and misaligned free-space optical interconnect systems[J]. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 27, 200-205(2010).
[93] Lin X, Rivenson Y, Yardimci N T et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 361, 1004-1008(2018).
[94] Veli M, Mengu D, Yardimci N T et al. Terahertz pulse shaping using diffractive surfaces[J]. Nature Communications, 12, 37(2021).
[95] Qian C, Lin X, Lin X et al. Performing optical logic operations by a diffractive neural network[J]. Light, Science & Applications, 9, 59(2020).
[96] Goi E, Chen X, Zhang Q et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip[J]. Light, Science & Applications, 10, 40(2021).
[97] Zhou T K, Lin X, Wu J M et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit[J]. Nature Photonics, 15, 367-373(2021).
[98] Li J X, Mengu D, Yardimci N T et al. Spectrally encoded single-pixel machine vision using diffractive networks[J]. Science Advances, 7, eabd7690(2021).
[99] Shen Y C, Harris N C, Skirlo S et al. Deep learning with coherent nanophotonic circuits[C]. //2017 IEEE Photonics Society Summer Topical Meeting Series (SUM), July 10-12, 2017, San Juan, PR, USA., 189-190(2017).
[100] Hughes T W, Williamson I A D, Minkov M et al. Wave physics as an analog recurrent neural network[J]. Science Advances, 5, eaay6946(2019).
[101] Qian C, Zheng B, Shen Y C et al. Deep-learning-enabled self-adaptive microwave cloak without human intervention[J]. Nature Photonics, 14, 383-390(2020).
[102] Liu H L, Xie D, Shen H Y et al. Functional micro-nano structure with variable colour: applications for anti-counterfeiting[J]. Advances in Polymer Technology, 2019, 1-26(2019).
[103] Yang H, Niu Z K, Xiao S L et al. Fast and accurate optical fiber channel modeling using generative adversarial network[J]. Journal of Lightwave Technology, 39, 1322-1333(2021).
[104] Ibáñez M B, Di Serio Á, Villarán D et al. Experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness[J]. Computers & Education, 71, 1-13(2014).
[105] Zhang L P, Zhang L F, Du B. Deep learning for remote sensing data: a technical tutorial on the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 4, 22-40(2016).
[106] Zhou Y, Tuzel O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]. //2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA, 4490-4499(2018).
[107] Liu J M, Wang P P, Zhang X K et al. Deep learning based atmospheric turbulence compensation for orbital angular momentum beam distortion and communication[J]. Optics Express, 27, 16671-16688(2019).
[108] Zhou H Q, Huang L L, Wang Y T. Deep learning algorithm and its application in optics[J]. Infrared and Laser Engineering, 48, 1226004(2019).
Get Citation
Copy Citation Text
Yitong Wang, Hongqiang Zhou, Jingxiao Yan, Cong He, Lingling Huang. Advances in Computational Optics Based on Deep Learning[J]. Chinese Journal of Lasers, 2021, 48(19): 1918004
Received: Jul. 2, 2021
Accepted: Aug. 16, 2021
Published Online: Sep. 30, 2021
The Author Email: Huang Lingling (huanglingling@bit.edu.cn)