Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Parallel one-class extreme learning machine for imbalance learning based on Bayesian approach

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM) is a competitive machine learning approach for training single hidden layer feedforward neural networks with fast learning speed and good generalization performance. However, ELM also suffers from the imbalanced classification problem, which exists in many fields and degrades the performance of classifier significantly. In order to address this issue, in this paper, we propose a novel parallel one-class ELM based on Bayesian approach for imbalanced classification problem, named P-ELM. In P-ELM, the training dataset is firstly divided into k (k is the number of classes) components according to the class attribution of the samples, and then the divided training datasets will be fed into the corresponding k Kernel-based one-class ELM classifiers. Estimation of the probability density based on the output function of Kernel-based one-class ELM classifier is constructed. Thus, the class attribution of the samples can be judged directly by comparing the values of the output function of each Kernel-based one-class ELM classifier of P-ELM based on the Bayesian approach. The detailed performance comparison and discussions of P-ELM are performed with several related class imbalance learning approaches on some selected benchmark datasets including both the binary classification and the multiclass classification problems, as well as a real world application, i.e., the blast furnace status diagnosis problem. The experimental results show that the proposed P-ELM can achieve better performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Ali A, Shamsuddin SM, Ralescu AL (2015) Classification with class imbalance problem: a review. Int J Adv Soft Comput Appl 7(3):176–204

    Google Scholar 

  • Arkedev AG, Braverman EM (1996) Computer pattern recognition. Thompson, Washington, DC

    Google Scholar 

  • Cao JW, Zhao YF, Lai XP, Ong MEH, Yin C, Koh ZX, Lin N (2015) Landmark recognition with sparse representation classification and extreme learning machine. J Franklin Inst 352:4528–4545

    Article  MathSciNet  Google Scholar 

  • Cao JW, Chen T, Fan JY (2016) Landmark recognition compact bow histogram and ensemble elm. Multimed Tools Appl 75(5):2839–2857

    Article  Google Scholar 

  • Chen C, Vong CM, Wong CM, Wang W, Wong PK (2018) Effcient extreme learning machine via very sparse random projection. Soft Comput 22:3563–3574

    Article  Google Scholar 

  • Ding S, Bilal M, Lin Z, Cao J, Lai X, Tam VN, Jose S (2018) Kernel based online learning for imbalance multiclass classification. Neurocomputing 277:139–148

    Article  Google Scholar 

  • Domingos P, Pazzani M (1997) On the optimality of the simple bayesian classifier under zero-one loss. Mach Learn 29(2):103–130

    Article  Google Scholar 

  • Du J, Vong CM, Pun CM, Wong PK, Ip WF (2017) Posting-boosting of classification boundary for imbalanced data using geometric mean. Neural Netw 96:101–114

    Article  PubMed  Google Scholar 

  • Fernandez A, Jesus MJ, Herrera F (2009) Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets. Int J Approx Reason 50(3):561–577

    Article  Google Scholar 

  • Fu AM, Dong CR, Wang LS (2015) An experimental study on stability and generalization of extreme learning machines. Int J Mach Learn Cyber 6:129–135

    Article  Google Scholar 

  • Galar M, Fernandez A, Barrebechea E, Bustince H, Herrera F (2013) A review on ensembles for the class imbalance problem: bagging, boosting, and hybrid-based-approaches. IEEE Trans Syst Man Cybern C Appl Rev 46(12):3460–3471

    Google Scholar 

  • Gautam C, Tiwari A, Ravindran S (2016) Construction of multi-class classifiers by extreme learning machine based one-class classifiers. In: Proceedings of 2016 international joint conference on neural networks, pp 2001–2007

  • Gautam C, Tiwari A, Leng Q (2017) On the construction of extreme learning machine for online and offline one-class classification-an expaned toolbox. Neurocomputing 261:126–143

    Article  Google Scholar 

  • Hadjadji B, Chibani Y, Guerbai Y (2014) Multiple one-class classifier combination for multi-class classification. In: Proceedings of the 22nd international conference on pattern recognition, pp 2832–2837

  • Han HG, Qiao JF (2013) A structure optimisation algorithm for feedforward neural network construction. Neurocomputing 99:347–357

    Article  Google Scholar 

  • He HB, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Article  Google Scholar 

  • Huang GB, Zhu QY, Siew CK (2004) Extreme learing machine: a new learning scheme of feedforward neural networks. In: Proceedings of 2004 IEEE internal joint conference on neural networks, pp 985–990

  • Huang GB, Chen L, Siew CK (2006a) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  PubMed  Google Scholar 

  • Huang GB, Zhu QY, Siew CK (2006b) Extreme learing machine: theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  • Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomptuing 70:3056–3062

    Article  Google Scholar 

  • Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomptuing 71:3460–3468

    Article  Google Scholar 

  • Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomptuing 71:576–583

    Article  Google Scholar 

  • Huang GB, Zhou HM, Ding XJ, Zhang R (2012) Extreme learing machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  • Huang GB (2014) An insight into extreme learing machine: random neurons, random features and Kernel. Cogn Comput 6:376–390

    Article  Google Scholar 

  • Huang G, Song SJ, Gupta JND, Wu C (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44(12):2405–2417

    Article  PubMed  Google Scholar 

  • Huang G, Huang GB, Song SJ, You KY (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    Article  PubMed  Google Scholar 

  • Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6:429–450

    Article  Google Scholar 

  • Kasun LLC, Zhou HM, Huang GB, Vong CM (2013) Representational learning with elms for big data. IEEE Intell Syst 28(6):31–34

    Google Scholar 

  • Leng Q, Qi HG, Miao J, Zhu WT, Su GP (2015) On-class classification with extreme learning machine. Math Probl Eng. https://doi.org/10.1155/2015/412957

    Article  MathSciNet  Google Scholar 

  • Li K, Kong XF, Lu Z, Liu WY, Yin JP (2014) Boosting weighted elm for imbalanced learning. Neurocomputing 128:15–21

    Article  Google Scholar 

  • Li YJ, Zhang S, Yin YX, Xiao WD, Zhang J (2017a) A novel online sequential extreme learning machine for gas utilization ratio prediction in blast furnaces. Sensors 17(8):1847–1870

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  • Li YJ, Zhang S, Yin YX, Zhang J, Xu ZG (2017b) Quantitative association rule mining for blast furnace production data. In: Proceedings of 2017 Chinese automation congress, pp 2275–2280

  • Liang NY, Huang GB, Saratchandran P, Sundarajan N (2006) A fast and accurate online squential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  PubMed  Google Scholar 

  • Liu LM, Wang AN, Sha M, Sun XY, Li YL (2011) Optional svm for fault diagnosis of blast furnace with imbalanced data. ISIJ Int 51(9):1474–1479

    Article  CAS  Google Scholar 

  • Lopez V, Fernandez A, Moreno-Torres JG, Herrera F (2012) Analysis of preprocessing vs cost-sensitive learning for imbalanced classification, open problems on intrinsic data characteristics. Expert Syst Appl 39(7):6585–6608

    Article  Google Scholar 

  • Lopez V, Fernandez A, Garcia S, Palade V, Herrera F (2013) An insight into classification with imbalanced data: empirical results and current trend on using data intrinsic characteristics. Inf Sci 258:113–141

    Article  Google Scholar 

  • Mena L, Gonzalez JA (2009) Symbolic one-class learning from imbalanced datasets: application in medical diagnosis. Int J Artif Intell T 18(2):273–309

    Article  Google Scholar 

  • Miche Y, Sorjamaa A, Simula O, Jutten C, Lendasse A (2010) Op-elm: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Article  PubMed  Google Scholar 

  • Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329

    Article  Google Scholar 

  • Raskutti B, Kowalczyk A (2006) Extreme re-balancing for svms: a case study. SIGKDD Explor 6:60–69

    Article  Google Scholar 

  • Rong HJ, Ong YS, Tan AH, Zhu ZX (2008) A fast pruned-extreme learning machine for classification problem. Neurocomptuing 72:359–366

    Article  Google Scholar 

  • Scholkopf B, Platt JC, Shawe-Taylor JC, Smola AJ, Williamson RC (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471

    Article  CAS  PubMed  Google Scholar 

  • Sen A, Islam MM, Murase K, Yao X (2016) Binarization with boosting and oversampling for multiclass classifications. IEEE Trans Cybern 46(5):1078–1091

    Article  PubMed  Google Scholar 

  • Sun Y, Tang K, Minku LL, Wang S, Yao X (2016) Online ensemble learning of data streams with gradually evolved classes. IEEE Trans Knowl Data En 28(6):1532–1545

    Article  Google Scholar 

  • Tax DMJ (2014) The data description toolbox for matlab, version 2.1.1. http://prlab.Tudelft.nl/david-tax/ddtools.html. Accessed 12 Sept 2018

  • Tax DMJ, Duin RPW (2001) Combining one-class classifiers. In: Proceeding of the 2nd international workshop on multiple classifier systems, pp 299–308

  • Tax DMJ, Muller KR (2004) A consistency-based model selection for one-class classification. In: Proceedings of the 17th international conference on pattern recognition, pp 363–366

  • Toh KA (2008) Deterministic neural classification. Neural Comput 20(6):1565–1595

    Article  MathSciNet  PubMed  Google Scholar 

  • Vong CM, Ip WF, Wong PK, Chiu CC (2014) Predicting minority class for suspended particulate matters level by extreme learning machine. Neurocomputing 128:136–144

    Article  Google Scholar 

  • Wang S, Minku LL, Yao X (2013) Online class imbalance learning and its applicationos in fault detection. Int J Comput Intell Appl 12(4):1–19

    Article  ADS  Google Scholar 

  • Wang SH, Muhammad K, Phillips P, Dong ZC, Zhang YD (2017) Ductal carcinoma in situ detection in breast thermography by extreme learning machine and combination of statistical measure and fractal dimension. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-017-0639-5

    Article  Google Scholar 

  • Wong PK, Zhong JH, Yang ZX, Vong CM (2016) Sparse bayesian extreme learning committee machine for engine simultaneous fault diagnosis. Neurocomputing 174:331–343

    Article  Google Scholar 

  • Wong CM, Vong CM, Wong PK, Cao JW (2018) Kernel-based multilayer extreme learning machines for represent learning. IEEE Trans Neural Netw Learn Syst 29(3):757–762

    Article  MathSciNet  PubMed  Google Scholar 

  • Xiao WD, Zhang J, Li YJ, Zhang S, Yang WD (2017) Class-specific cost regulation extreme learning machine for imbalanced classification. Neurocomputing 261:70–82

    Article  Google Scholar 

  • Yang YM, Wu QMJ (2016) Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Trans Cybern 46(11):2570–2583

    Article  PubMed  Google Scholar 

  • Yu HL, Sun CY, Yang XB, Yang WK, Shen JF, Qi YS (2016) Odoc-elm: Optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data. Knowl Based Syst 92:55–70

    Article  Google Scholar 

  • Zhang L, Dai HH (2006) Parameter optimization of Kernel-based one-class classifier on imbalance learning. J Comput 1(7):32–40

    Google Scholar 

  • Zhang WB, Ji HB (2013) Fuzzy extreme learning machine for classification. IET Electron Lett 49:448–450

    Article  ADS  Google Scholar 

  • Zhang R, Lan Y, Huang GB, Xu ZB, Soh YC (2013) Dynamic extreme learning machine and its approximation capability. IEEE Trans Cybern 43(6):2054–2065

    Article  ADS  PubMed  Google Scholar 

  • Zhang J, Xiao WD, Zhang S, Huang SD (2017) Device-free localization via an extreme learning machine with parameterized geometrical feature extraction. Sensors 17(4):879–890

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  • Zhang J, Xiao WD, Li YJ, Zhang S (2018) Residual compensation extreme learning machine for regression. Neurcomputing 311:126–136

    Article  Google Scholar 

  • Zong WW, Huang GB, Chen YQ (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported in part by the National Natural Science Foundation of China (NSFC Grants Nos. 61673056, 61333002 and 61673055) and Beijing Natural Science Foundation (No. 4182039).

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Yanjiao Li, Sen Zhang, Yixin Yin, Wendong Xiao & Jie Zhang

  2. Key Laboratory of Knowledge Automation for Industrial Processes, Ministry of Education, Beijing, 100083, China

    Yanjiao Li, Sen Zhang, Yixin Yin, Wendong Xiao & Jie Zhang

Authors
  1. Yanjiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yixin Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wendong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sen Zhang.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

The IR is defined:

For binary classification: \(IR = \frac{{\# ({\mathrm{minority\,class}})}}{{\# ({{\mathrm{majority\,class}}})}}\),

$$\begin{aligned} {\mathrm{For\,majority\,classification}}{:}IR = \frac{{\min (\# ({y_i}))}}{{\max (\# ({y_i}))}},i = 1,2, \ldots ,k, \end{aligned}$$

where, \(\# ({\mathrm{minority\,class}} )\) and \(\# ({\mathrm{majority\,class}})\) are the sample number of minority class and the sample number of majority class in the binary classification problem, \(\# ({y_i})\) is the sample number of class i.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Zhang, S., Yin, Y. et al. Parallel one-class extreme learning machine for imbalance learning based on Bayesian approach. J Ambient Intell Human Comput 15, 1745–1762 (2024). https://doi.org/10.1007/s12652-018-0994-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-018-0994-x

Keywords

Search

Navigation