Advertisement

Parallel one-class extreme learning machine for imbalance learning based on Bayesian approach

  • Yanjiao Li
  • Sen Zhang
  • Yixin Yin
  • Wendong Xiao
  • Jie Zhang
Original Research
  • 16 Downloads

Abstract

Extreme learning machine (ELM) is a competitive machine learning approach for training single hidden layer feedforward neural networks with fast learning speed and good generalization performance. However, ELM also suffers from the imbalanced classification problem, which exists in many fields and degrades the performance of classifier significantly. In order to address this issue, in this paper, we propose a novel parallel one-class ELM based on Bayesian approach for imbalanced classification problem, named P-ELM. In P-ELM, the training dataset is firstly divided into k (k is the number of classes) components according to the class attribution of the samples, and then the divided training datasets will be fed into the corresponding k Kernel-based one-class ELM classifiers. Estimation of the probability density based on the output function of Kernel-based one-class ELM classifier is constructed. Thus, the class attribution of the samples can be judged directly by comparing the values of the output function of each Kernel-based one-class ELM classifier of P-ELM based on the Bayesian approach. The detailed performance comparison and discussions of P-ELM are performed with several related class imbalance learning approaches on some selected benchmark datasets including both the binary classification and the multiclass classification problems, as well as a real world application, i.e., the blast furnace status diagnosis problem. The experimental results show that the proposed P-ELM can achieve better performance.

Keywords

Extreme learning machine Imbalanced classification One-class extreme learning machine Bayesian approach Blast furnace status diagnosis 

Notes

Acknowledgements

This work has been supported in part by the National Natural Science Foundation of China (NSFC Grants Nos. 61673056, 61333002 and 61673055) and Beijing Natural Science Foundation (No. 4182039).

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

References

  1. Ali A, Shamsuddin SM, Ralescu AL (2015) Classification with class imbalance problem: a review. Int J Adv Soft Comput Appl 7(3):176–204Google Scholar
  2. Arkedev AG, Braverman EM (1996) Computer pattern recognition. Thompson, Washington, DCGoogle Scholar
  3. Cao JW, Zhao YF, Lai XP, Ong MEH, Yin C, Koh ZX, Lin N (2015) Landmark recognition with sparse representation classification and extreme learning machine. J Franklin Inst 352:4528–4545MathSciNetCrossRefGoogle Scholar
  4. Cao JW, Chen T, Fan JY (2016) Landmark recognition compact bow histogram and ensemble elm. Multimed Tools Appl 75(5):2839–2857CrossRefGoogle Scholar
  5. Chen C, Vong CM, Wong CM, Wang W, Wong PK (2018) Effcient extreme learning machine via very sparse random projection. Soft Comput 22:3563–3574CrossRefGoogle Scholar
  6. Ding S, Bilal M, Lin Z, Cao J, Lai X, Tam VN, Jose S (2018) Kernel based online learning for imbalance multiclass classification. Neurocomputing 277:139–148CrossRefGoogle Scholar
  7. Domingos P, Pazzani M (1997) On the optimality of the simple bayesian classifier under zero-one loss. Mach Learn 29(2):103–130CrossRefGoogle Scholar
  8. Du J, Vong CM, Pun CM, Wong PK, Ip WF (2017) Posting-boosting of classification boundary for imbalanced data using geometric mean. Neural Netw 96:101–114CrossRefGoogle Scholar
  9. Fernandez A, Jesus MJ, Herrera F (2009) Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets. Int J Approx Reason 50(3):561–577CrossRefGoogle Scholar
  10. Fu AM, Dong CR, Wang LS (2015) An experimental study on stability and generalization of extreme learning machines. Int J Mach Learn Cyber 6:129–135CrossRefGoogle Scholar
  11. Galar M, Fernandez A, Barrebechea E, Bustince H, Herrera F (2013) A review on ensembles for the class imbalance problem: bagging, boosting, and hybrid-based-approaches. IEEE Trans Syst Man Cybern C Appl Rev 46(12):3460–3471Google Scholar
  12. Gautam C, Tiwari A, Ravindran S (2016) Construction of multi-class classifiers by extreme learning machine based one-class classifiers. In: Proceedings of 2016 international joint conference on neural networks, pp 2001–2007Google Scholar
  13. Gautam C, Tiwari A, Leng Q (2017) On the construction of extreme learning machine for online and offline one-class classification-an expaned toolbox. Neurocomputing 261:126–143CrossRefGoogle Scholar
  14. Hadjadji B, Chibani Y, Guerbai Y (2014) Multiple one-class classifier combination for multi-class classification. In: Proceedings of the 22nd international conference on pattern recognition, pp 2832–2837Google Scholar
  15. Han HG, Qiao JF (2013) A structure optimisation algorithm for feedforward neural network construction. Neurocomputing 99:347–357CrossRefGoogle Scholar
  16. He HB, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRefGoogle Scholar
  17. Huang GB, Zhu QY, Siew CK (2004) Extreme learing machine: a new learning scheme of feedforward neural networks. In: Proceedings of 2004 IEEE internal joint conference on neural networks, pp 985–990Google Scholar
  18. Huang GB, Chen L, Siew CK (2006a) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRefGoogle Scholar
  19. Huang GB, Zhu QY, Siew CK (2006b) Extreme learing machine: theory and applications. Neurocomputing 70:489–501CrossRefGoogle Scholar
  20. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomptuing 70:3056–3062CrossRefGoogle Scholar
  21. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomptuing 71:3460–3468CrossRefGoogle Scholar
  22. Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomptuing 71:576–583CrossRefGoogle Scholar
  23. Huang GB, Zhou HM, Ding XJ, Zhang R (2012) Extreme learing machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529CrossRefGoogle Scholar
  24. Huang GB (2014) An insight into extreme learing machine: random neurons, random features and Kernel. Cogn Comput 6:376–390CrossRefGoogle Scholar
  25. Huang G, Song SJ, Gupta JND, Wu C (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44(12):2405–2417CrossRefGoogle Scholar
  26. Huang G, Huang GB, Song SJ, You KY (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48CrossRefGoogle Scholar
  27. Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6:429–450zbMATHGoogle Scholar
  28. Kasun LLC, Zhou HM, Huang GB, Vong CM (2013) Representational learning with elms for big data. IEEE Intell Syst 28(6):31–34Google Scholar
  29. Leng Q, Qi HG, Miao J, Zhu WT, Su GP (2015) On-class classification with extreme learning machine. Math Probl Eng.  https://doi.org/10.1155/2015/412957 MathSciNetCrossRefzbMATHGoogle Scholar
  30. Li K, Kong XF, Lu Z, Liu WY, Yin JP (2014) Boosting weighted elm for imbalanced learning. Neurocomputing 128:15–21CrossRefGoogle Scholar
  31. Li YJ, Zhang S, Yin YX, Xiao WD, Zhang J (2017a) A novel online sequential extreme learning machine for gas utilization ratio prediction in blast furnaces. Sensors 17(8):1847–1870CrossRefGoogle Scholar
  32. Li YJ, Zhang S, Yin YX, Zhang J, Xu ZG (2017b) Quantitative association rule mining for blast furnace production data. In: Proceedings of 2017 Chinese automation congress, pp 2275–2280Google Scholar
  33. Liang NY, Huang GB, Saratchandran P, Sundarajan N (2006) A fast and accurate online squential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423CrossRefGoogle Scholar
  34. Liu LM, Wang AN, Sha M, Sun XY, Li YL (2011) Optional svm for fault diagnosis of blast furnace with imbalanced data. ISIJ Int 51(9):1474–1479CrossRefGoogle Scholar
  35. Lopez V, Fernandez A, Moreno-Torres JG, Herrera F (2012) Analysis of preprocessing vs cost-sensitive learning for imbalanced classification, open problems on intrinsic data characteristics. Expert Syst Appl 39(7):6585–6608CrossRefGoogle Scholar
  36. Lopez V, Fernandez A, Garcia S, Palade V, Herrera F (2013) An insight into classification with imbalanced data: empirical results and current trend on using data intrinsic characteristics. Inf Sci 258:113–141CrossRefGoogle Scholar
  37. Mena L, Gonzalez JA (2009) Symbolic one-class learning from imbalanced datasets: application in medical diagnosis. Int J Artif Intell T 18(2):273–309CrossRefGoogle Scholar
  38. Miche Y, Sorjamaa A, Simula O, Jutten C, Lendasse A (2010) Op-elm: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRefGoogle Scholar
  39. Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329CrossRefGoogle Scholar
  40. Raskutti B, Kowalczyk A (2006) Extreme re-balancing for svms: a case study. SIGKDD Explor 6:60–69CrossRefGoogle Scholar
  41. Rong HJ, Ong YS, Tan AH, Zhu ZX (2008) A fast pruned-extreme learning machine for classification problem. Neurocomptuing 72:359–366CrossRefGoogle Scholar
  42. Scholkopf B, Platt JC, Shawe-Taylor JC, Smola AJ, Williamson RC (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471CrossRefGoogle Scholar
  43. Sen A, Islam MM, Murase K, Yao X (2016) Binarization with boosting and oversampling for multiclass classifications. IEEE Trans Cybern 46(5):1078–1091CrossRefGoogle Scholar
  44. Sun Y, Tang K, Minku LL, Wang S, Yao X (2016) Online ensemble learning of data streams with gradually evolved classes. IEEE Trans Knowl Data En 28(6):1532–1545CrossRefGoogle Scholar
  45. Tax DMJ (2014) The data description toolbox for matlab, version 2.1.1. http://prlab.Tudelft.nl/david-tax/ddtools.html. Accessed 12 Sept 2018
  46. Tax DMJ, Duin RPW (2001) Combining one-class classifiers. In: Proceeding of the 2nd international workshop on multiple classifier systems, pp 299–308CrossRefGoogle Scholar
  47. Tax DMJ, Muller KR (2004) A consistency-based model selection for one-class classification. In: Proceedings of the 17th international conference on pattern recognition, pp 363–366Google Scholar
  48. Toh KA (2008) Deterministic neural classification. Neural Comput 20(6):1565–1595MathSciNetCrossRefGoogle Scholar
  49. Vong CM, Ip WF, Wong PK, Chiu CC (2014) Predicting minority class for suspended particulate matters level by extreme learning machine. Neurocomputing 128:136–144CrossRefGoogle Scholar
  50. Wang S, Minku LL, Yao X (2013) Online class imbalance learning and its applicationos in fault detection. Int J Comput Intell Appl 12(4):1–19CrossRefGoogle Scholar
  51. Wang SH, Muhammad K, Phillips P, Dong ZC, Zhang YD (2017) Ductal carcinoma in situ detection in breast thermography by extreme learning machine and combination of statistical measure and fractal dimension. J Ambient Intell Human Comput.  https://doi.org/10.1007/s12652-017-0639-5 CrossRefGoogle Scholar
  52. Wong PK, Zhong JH, Yang ZX, Vong CM (2016) Sparse bayesian extreme learning committee machine for engine simultaneous fault diagnosis. Neurocomputing 174:331–343CrossRefGoogle Scholar
  53. Wong CM, Vong CM, Wong PK, Cao JW (2018) Kernel-based multilayer extreme learning machines for represent learning. IEEE Trans Neural Netw Learn Syst 29(3):757–762MathSciNetCrossRefGoogle Scholar
  54. Xiao WD, Zhang J, Li YJ, Zhang S, Yang WD (2017) Class-specific cost regulation extreme learning machine for imbalanced classification. Neurocomputing 261:70–82CrossRefGoogle Scholar
  55. Yang YM, Wu QMJ (2016) Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Trans Cybern 46(11):2570–2583CrossRefGoogle Scholar
  56. Yu HL, Sun CY, Yang XB, Yang WK, Shen JF, Qi YS (2016) Odoc-elm: Optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data. Knowl Based Syst 92:55–70CrossRefGoogle Scholar
  57. Zhang L, Dai HH (2006) Parameter optimization of Kernel-based one-class classifier on imbalance learning. J Comput 1(7):32–40Google Scholar
  58. Zhang WB, Ji HB (2013) Fuzzy extreme learning machine for classification. IET Electron Lett 49:448–450CrossRefGoogle Scholar
  59. Zhang R, Lan Y, Huang GB, Xu ZB, Soh YC (2013) Dynamic extreme learning machine and its approximation capability. IEEE Trans Cybern 43(6):2054–2065CrossRefGoogle Scholar
  60. Zhang J, Xiao WD, Zhang S, Huang SD (2017) Device-free localization via an extreme learning machine with parameterized geometrical feature extraction. Sensors 17(4):879–890CrossRefGoogle Scholar
  61. Zhang J, Xiao WD, Li YJ, Zhang S (2018) Residual compensation extreme learning machine for regression. Neurcomputing 311:126–136CrossRefGoogle Scholar
  62. Zong WW, Huang GB, Chen YQ (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Yanjiao Li
    • 1
    • 2
  • Sen Zhang
    • 1
    • 2
  • Yixin Yin
    • 1
    • 2
  • Wendong Xiao
    • 1
    • 2
  • Jie Zhang
    • 1
    • 2
  1. 1.School of Automation and Electrical EngineeringUniversity of Science and Technology BeijingBeijingChina
  2. 2.Key Laboratory of Knowledge Automation for Industrial ProcessesMinistry of EducationBeijingChina

Personalised recommendations