Journal of Electronic Science and Technology, Volume. 22, Issue 1, 100246(2024)
Machine learning model based on non-convex penalized huberized-SVM
Figures & Tables(11)
Fig. 1. Comparison of the hinge loss with the huberized hinge loss.
Table 1. Desciption of the resulted algorithm for solving the model (1).
View tableView in ArticleTable 1. Desciption of the resulted algorithm for solving the model (1).
Algorithm 1. Initialize $ {\tilde \beta _0} $ and$ \tilde {\Cambriabfont\text{β}} $ .2. Iterate 1) and 2) until convergence: 1) Cyclic coordinate descent: j = 1, 2, ···, p. i) Calculate $ {r_i} = {y_i}({\tilde \beta _0} + {\mathbf{x}}_i^{\text{T}}\tilde {\Cambriabfont\text{β}}) $ ;ii) Calculate $ \hat \beta _j^{{\text{new}}} = \left( {{\delta \mathord{\left/ {\vphantom {\delta 2}} \right. } 2}} \right)S\left( {z{\rm{,}}{\text{ }}{P_{\lambda _1}'}\left( {\left| {{{\tilde \beta }_j}} \right|} \right)} \right) $ ;iii) Let $ {\tilde \beta _j} = \hat \beta _j^{{\text{new}}}{\rm{,}}{\text{ }}j = j + 1 $ .2) Update the intercept term: i) Calculate $ {r_i} = {y_i}\left( {{{\tilde \beta }_0} + {\mathbf{x}}_i^{\text{T}}\tilde {\Cambriabfont\text{β}}} \right) $ ;ii) Calculate $ \hat \beta _0^{{\text{new}}} = {\tilde \beta _0} - \left( {{\delta \mathord{\left/ {\vphantom {\delta {2n}}} \right. } {2n}}} \right)\sum\limits_{i = 1}^n {{l_\delta' }\left( {{r_i}} \right){y_i}} $ ;iii) Let $ {\tilde \beta _j} = \hat \beta _j^{{\text{new}}}{\rm{,}}{\text{ }}j = j + 1 $ .3. If $\mathop {{\rm{max}} }\limits_{0 \leq j \leq p} \left| {\hat \beta _j^{{\text{new}}} - {{\tilde \beta }_j}} \right| < \varepsilon $ , stop the iteration; otherwise, return to step 2.
Table 2. Results of evaluation indicators for the classification prediction of each model.
View tableView in ArticleTable 2. Results of evaluation indicators for the classification prediction of each model.
Indicators Methods Min 1st Qu Median Mean 3rd Qu Max ACC SVM 0.7016 0.7435 0.8010 0.8000 0.8534 0.8952 LASSO-SVM 0.7612 0.8114 0.8680 0.8653 0.9025 0.9589 MCP-HSVM 0.7773 0.8260 0.8721 0.8749 0.9180 0.9708 SCAD-HSVM 0.7842 0.8396 0.8862 0.8884 0.9372 0.9804 AUC SVM 0.7840 0.8170 0.8627 0.8608 0.9021 0.9335 LASSO-SVM 0.8436 0.8916 0.9378 0.9290 0.9657 0.9902 MCP-HSVM 0.8489 0.8801 0.9239 0.9239 0.9625 0.9969 SCAD-HSVM 0.8502 0.8867 0.9328 0.9256 0.9626 0.9974 TPR SVM 0.6691 0.7075 0.7546 0.7602 0.8055 0.8645 LASSO-SVM 0.7553 0.7988 0.8370 0.8463 0.8809 0.9500 MCP-HSVM 0.7602 0.8068 0.8434 0.8512 0.8978 0.9581 SCAD-HSVM 0.7595 0.7925 0.8525 0.8542 0.9102 0.9555 FPR SVM 0.2101 0.2342 0.2535 0.2571 0.2807 0.3097 LASSO-SVM 0.1470 0.1710 0.1974 0.1960 0.2195 0.2456 MCP-HSVM 0.1643 0.1868 0.2100 0.2121 0.2381 0.2634 SCAD-HSVM 0.1683 0.1982 0.2239 0.2209 0.2457 0.2670
Table 3. Results of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
View tableView in ArticleTable 3. Results of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
Indicators Methods Min 1st Qu Median Mean 3rd Qu Max NV LASSO-SVM 5.0000 10.0000 17.0000 16.6400 24.2500 38.0000 MCP-HSVM 6.0000 9.0000 15.0000 19.6000 25.7500 47.0000 SCAD-HSVM 8.0000 9.0000 10.0000 10.2600 11.0000 13.0000 FNR LASSO-SVM 0.0176 0.1029 0.2294 0.2512 0.3247 0.5112 MCP-HSVM 0.0765 0.1252 0.2000 0.2182 0.3059 0.4824 SCAD-HSVM 0.1029 0.2294 0.2512 0.2824 0.3471 0.5647 FDR LASSO-SVM 0.0000 0.0764 0.2248 0.2679 0.3564 0.4578 MCP-HSVM 0.0000 0.1193 0.2556 0.2880 0.3810 0.4610 SCAD-HSVM 0.0000 0.1000 0.1577 0.1735 0.2662 0.3500
Table 4. Results of evaluation indicators for the classification prediction of each model under different correlations.
View tableView in ArticleTable 4. Results of evaluation indicators for the classification prediction of each model under different correlations.
ρ Indicators SVM LASSO-SVM MCP-HSVM SCAD-HSVM ρ = 0.3 ACC 0.7758 0.8339 0.8436 0.8508 AUC 0.8351 0.8802 0.9050 0.8956 TPR 0.7290 0.7996 0.8306 0.8260 FPR 0.2827 0.2103 0.2206 0.2230 ρ = 0.5 ACC 0.7496 0.8188 0.8348 0.8310 AUC 0.7828 0.8650 0.8861 0.8853 TPR 0.6939 0.7647 0.8245 0.8130 FPR 0.3238 0.2357 0.2394 0.2345 ρ = 0.8 ACC 0.6760 0.7786 0.8200 0.8196 AUC 0.7209 0.8212 0.8701 0.8635 TPR 0.6331 0.7211 0.8135 0.7992 FPR 0.3301 0.2567 0.2412 0.2451
Table 5. Means of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
View tableView in ArticleTable 5. Means of evaluation indicators for variable selection of LASSO-SVM, MCP-HSVM, and SCAD-HSVM.
ρ Indicators LASSO-SVM MCP-HSVM SCAD-HSVM ρ = 0.3 NV 15.8200 18.4800 14.2600 FNR 0.2424 0.1612 0.2118 FDR 0.2360 0.2057 0.1885 ρ = 0.5 NV 12.9000 16.0800 13.7600 FNR 0.3117 0.2294 0.2941 FDR 0.3298 0.2325 0.2916 ρ = 0.8 NV 10.2400 15.7600 12.9800 FNR 0.3729 0.2800 0.3329 FDR 0.1655 0.2397 0.3244
Table 6. Financial indicators used in this paper.
View tableView in ArticleTable 6. Financial indicators used in this paper.
Level 1 indicators Level 2 indicators Solvency Current ratio: X1 Quick ratio: X2 Equity ratio: X3 Total equity/liabilities attributable to shareholders of the parent company: X4 Earnings before interest, taxes, depreciation, and amortization/total liabilities: X5 Net cash flow from operating activities/total liabilities: X6 Interest coverage ratio (earnings before interest and taxes/interest expense): X7 Indicators per share Earnings per share (RMB): X8 Operating cash flow per share (RMB): X9 Net assets per share (net of minority interests) (RMB): X10 Revenue quality Net income from operating activities/total profit (%): X11 Net gain from changes in value/total profit (%): X12 Net non-operating income and expenses/total profit (%): X13 Net income after deducting non-recurring gain and losses/net income (%): X14 Profitability Gross profit margin on sales (%): X15 Net sales margin (%): X16 Net return on assets (%): X17 Net margin on total assets (%): X18 Operating capacity Inventory turnover ratio (times): X19 Accounts receivable turnover ratio (times): X20 Current assets turnover ratio (times): X21 Fixed assets turnover ratio (times): X22 Total assets turnover ratio (times): X23 Capital structure Asset-liability ratio (%): X24 Equity multiplier: X25 Current assets/total assets (%): X26 Non-current assets/total assets (%): X27 Current liabilities/total liabilities (%): X28 Non-current liabilities/total liabilities (%): X29
Table 7. Prediction results of each method (average of 100 predictions).
View tableView in ArticleTable 7. Prediction results of each method (average of 100 predictions).
Methods ACC AUC TPR NV SVM 0.8776 0.7676 0.7057 / Lasso-SVM 0.8632 0.7967 0.7301 17.8 MCP-HSVM 0.8652 0.8187 0.7682 23.6 SCAD-HSVM 0.8754 0.8385 0.7494 22.1
Table 8. Selection frequency of each financial indicator.
View tableView in ArticleTable 8. Selection frequency of each financial indicator.
Variables LASSO-SVM MCP-HSVM SCAD-HSVM Variables LASSO-SVM MCP-HSVM SCAD-HSVM X1 0 15 11 X16 100 100 100 X2 100 100 100 X17 0 93 100 X3 40 50 45 X18 100 100 100 X4 0 8 0 X19 0 0 55 X5 100 100 100 X20 100 100 100 X6 50 100 58 X21 100 100 100 X7 0 0 5 X22 100 100 100 X8 100 100 100 X23 95 100 55 X9 100 100 100 X24 91 95 100 X10 100 100 100 X25 0 40 0 X11 92 95 100 X26 100 100 100 X12 50 95 100 X27 100 100 100 X13 0 0 3 X28 85 95 100 X14 100 100 100 X29 0 55 76 X15 0 51 0
Table 9. Classification of variables by the selection frequency of MCP-HSVM.
View tableView in ArticleTable 9. Classification of variables by the selection frequency of MCP-HSVM.
Selection frequency Level 2 indicators 100 X2 (quick ratio), X5 (earnings before interest, taxes, depreciation, and amortization/total liabilities), X6 (net cash flow from operating activities/total liabilities), X8 (earnings per share), X9 (operating cash flow per share), X10 (net assets per share), X14 (net income after deducting non-recurring gain and losses/net income), X16 (net sales margin), X18 (net margin on total assets), X20 (accounts receivable turnover ratio), X21 (current assets turnover ratio), X22 (fixed assets turnover ratio), X23 (total assets turnover ratio), X26 (current assets/total assets), and X27 (non-current assets/total assets) 90−100 X11 (net income from operating activities/total profit), X12 (net gain from changes in value/total profit), X17 (return on net assets), X24 (asset-liability ratio), and X28 (current liabilities/total liabilities) 60−90 None 40−60 X3 (equity ratio), X15 (gross profit margin on sales), X25 (equity multiplier), and X29 (non-current liabilities/total liabilities) 20−40 None 1−20 X1 (current ratio) and X4 (total equity/liabilities attributable to shareholders of the parent company) 0 X7 (interest coverage ratio (earnings before interest and taxes/interest expense)), X13 (net non-operating income and expenses/total profit), and X19 (inventory turnover ratio)
Tools
Get Citation
Copy Citation Text
Peng Wang, Ji Guo, Lin-Feng Li. Machine learning model based on non-convex penalized huberized-SVM[J]. Journal of Electronic Science and Technology, 2024, 22(1): 100246
Paper Information
Category:
Received: May. 21, 2023
Accepted: Mar. 15, 2024
Published Online: Jul. 5, 2024
The Author Email: Peng Wang (pwang@xzmu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20