Infrared and Laser Engineering, Volume. 51, Issue 3, 20210253(2022)
High efficient activation function design for CNN model image classification task
Figures & Tables(12)
Fig. 2. Derivatives of (a)
Fig. 3. Test accuracy (a) and training time (b) of different activation functions using ResNet18 network on CIFAR10
Fig. 4. Test accuracy (a) and training time (b) of different activation functions using VGG16 network on CIFAR10
Fig. 5. Test accuracy (a) and training time (b) of different activation functions using ResNet18 network on CIFAR100
Fig. 6. Test accuracy (a) and training time (b) of different activation functions using VGG16 network on CIFAR100
Fig. 7. Test accuracy (a) and training time (b) of different activation functions using ResNet18 network on Fer2013
Fig. 8. Test accuracy (a) and training time (b) of different activation functions using VGG16 network on Fer2013
Table 1. Mathematical models of four activation functions
View tableView in ArticleTable 1. Mathematical models of four activation functions
Function Function model f1 ${f_1}(x) = \left\{ { f2 ${f_2}(x) = \left\{ { f3 ${f_3}(x) = \left\{ { f4 ${f_4}(x) = \left\{ {
Table 2. Four kinds of activation function derivative function model
View tableView in ArticleTable 2. Four kinds of activation function derivative function model
Derived function Function model f1’ ${f_1}^\prime (x) = \left\{ { f2’ ${f_2}^\prime (x) = \left\{ { f3’ ${f_3}^\prime (x) = \left\{ { f4’ ${f_4}^\prime (x) = \left\{ {
Table 3. Performance of different activation functions on the ResNet18 network
View tableView in ArticleTable 3. Performance of different activation functions on the ResNet18 network
Results Methods Datasets CIFAR10 CIFAR100 ACC T/h ACC T/h f1 93.11% 1.332 74.82% 1.332 f2 93.03% 1.335 74.27% 1.335 f3 93.66% 1.290 75.23% 1.290 f4 93.78% 1.262 75.87% 1.262 ReLU 92.90% 1.325 73.68% 1.325
Table 4. Performance of different activation functions on the VGG16 network
View tableView in ArticleTable 4. Performance of different activation functions on the VGG16 network
Results Methods Datasets CIFAR10 CIFAR100 ACC T/h ACC T/h f1 91.31% 1.225 58.91% 1.225 f2 91.24% 1.248 58.35% 1.248 f3 91.86% 1.243 59.23% 1.243 f4 91.98% 1.175 59.95% 1.175 ReLU 91.15% 1.238 56.24% 1.238
Tools
Get Citation
Copy Citation Text
Shengjie Du, Xiaofen Jia, Yourui Huang, Yongcun Guo, Baiting Zhao. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253
Paper Information
Category: Image processing
Received: Dec. 10, 2021
Accepted: --
Published Online: Apr. 8, 2022
The Author Email:
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20