Infrared and Laser Engineering, Volume. 51, Issue 3, 20210253(2022)

High efficient activation function design for CNN model image classification task

Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, and Baiting Zhao
Author Affiliations
  • School of Electrical and Information Engineering, Anhui University of Science and Technolog, Huainan 232000, China
  • show less

    Activation Functions (AF) play a very important role in learning and fitting complex function models of convolutional neural networks. In order to enable neural networks to complete various learning tasks better and faster, a new efficient activation function EReLU was designed in this paper. By introducing the natural logarithm function, EReLU effectively alleviated the problems of neuronal "necrosis" and gradient dispersion. Through the analysis of the activation function and its derivative function in the feedforward and feedback process of the mathematical model of the EReLU function exploration and design, the specific design of the EReLU function was determined through test, and finally the effect of improving the accuracy and accelerating training was achieved; Subsequently, EReLU was tested on different networks and data sets, and the results show that compared with ReLU and its improved function, the accuracy of EReLU is improved by 0.12%-6.61%, and the training efficiency is improved by 1.02%-6.52%, which strongly proved the superiority of EReLU function in accelerating training and improving accuracy.

    Tools

    Get Citation

    Copy Citation Text

    Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, Baiting Zhao. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Image processing

    Received: Dec. 10, 2021

    Accepted: --

    Published Online: Apr. 8, 2022

    The Author Email:

    DOI:10.3788/IRLA20210253

    Topics