Laser & Optoelectronics Progress, Volume. 60, Issue 22, 2220001(2023)

Influence of Hyperparameters on Performance of Optical Neural Network Training Algorithms

Wen Cao, Meiyu Liu, Minghao Lu, Xiaofeng Shao, Qifa Liu, and Jin Wang*
Author Affiliations
  • School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, Jiangsu , China
  • show less

    An optical neural network (ONN) based on fast Fourier transform (FFT) is constructed for digital image recognition in optical devices. Herein, ONN uses Mach-Zehnder interferometer (MZI) as its linear optical processing unit. These MZIs are connected in a grid-like layout and modulate the passing optical signals to achieve multiplication and addition. Subsequently, MZIs achieve classification and recognition for images. In this study, the influence of main hyperparameters (e.g., momentum coefficient and learning rate of the training algorithm) on the performance of ONN in recognizing handwritten digital images is investigated. First, the performance of ONN with four training algorithms in recognizing handwritten digital images under different learning rates is compared. These algorithms connect with different nonlinear functions and different number of hidden layers, namely, stochastic gradient descent (SGD), root mean square prop (RMSprop), adaptive moment estimation (Adam), and adaptive gradient (Adagrad). Additionally, the accuracy, running memory, and training time of ONN with the SGD algorithm connected with different nonlinear functions and different number of hidden layers are analyzed under different momentum coefficients. The recognition performance of ONN with SGD and RMSprop training algorithms is also compared after the introduction of momentum, where the learning rate is 0.05 and 0.005. The experimental results show that when the learning rate changes from 0.5 to 5 × 10-5, the FFT-typed ONN with the RMSprop training algorithm, two hidden layers, and the nonlinear function of Softplus has the highest recognition accuracy, reaching 97.4%. Furthermore, for the momentum coefficient of 0, the ONN with two hidden layers and the nonlinear function of Softplus trained by the SGD algorithm exhibits the highest recognition accuracy of 96%, when the momentum coefficient increases to 0.9, the accuracy of ONN is improved to 96.9%. However, the RMSprop algorithm with momentum leads to nonconvergence or slow convergence of network recognition accuracy.

    Tools

    Get Citation

    Copy Citation Text

    Wen Cao, Meiyu Liu, Minghao Lu, Xiaofeng Shao, Qifa Liu, Jin Wang. Influence of Hyperparameters on Performance of Optical Neural Network Training Algorithms[J]. Laser & Optoelectronics Progress, 2023, 60(22): 2220001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Optics in Computing

    Received: Jan. 30, 2023

    Accepted: Feb. 27, 2023

    Published Online: Nov. 6, 2023

    The Author Email: Wang Jin (jinwang@njupt.edu.cn)

    DOI:10.3788/LOP230535

    Topics