Abstract
The term “concept drift” refers to a change in statistical distribution of the data. In machine learning and predictive analysis, a fundamental assumption exits which reasons that the data is a random variable which is being generated independently from an underlying stationary distribution. In this chapter we present discussions on concept drifts that are inherent in the context big data. We discuss different forms of concept drifts that are evident in streaming data and outline different techniques for handling them. Handling concept drift is important for big data where the data flow occurs continuously causing existing learned models to lose their predictive accuracy. This chapter will serve as a reference to academicians and industry practitioners who are interested in the niche area of handling concept drift for big data applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zang W, Zhang P, Zhou C, Guo L (2015) Comparative study between incremental and ensemble learning on data streams: case study. J Big Data 1:5
Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. Johns Hopkins University, Baltimore
Ross DA et al (2008) Incremental learning for robust visual tracking. Int J Computer Vision 77(1–3):125–141
Losing V, Wersing BHH (2018) Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275:1261–1274
Oza NC (2001) Online ensemble learning. University of California, Berkeley
Liao J-W, Dai B-R (2014) An ensemble learning approach for concept drift. In: Information science and applications (ICISA), 2014 international conference on. IEEE
Gomes HM (2017) A survey on ensemble learning for data stream classification. ACM Computing Surveys (CSUR) 50(2):23
Yoo PD, Ho YS, Zhou BB, Zomaya AY (2008) SiteSeek: post-translational modification analysis using adaptive locality-effective kernel methods and new profiles. BMC Bioinformatics 9:272
Lee W, Stolfo S, Mok K (2000) Adaptive intrusion detection: a data mining approach. Artif Intell Rev 14(6):533–567
Hilas CS (2009) Designing an expert system for fraud detection in private telecommunications networks. Expert Syst Appl 36(9):11559–11569
Mazhelis O, Puuronen S (2007) Comparing classifier combining techniques for mobile-masquerader detection. In: The second international conference on availability, reliability and security
Aminikhanghahi S, Cook DJ (2017) A survey of methods for time series change point detection. Knowl Inf Syst 51(2):339–367
Kawahara Y (2009) Change-point detection in time-series data by direct density-ratio estimation. In: Proceedings of the 2009 SIAM international conference on data mining. Society for Industrial and Applied Mathematics
Ghourchian N, Allegue-Martinez M, Precup D (2017) Real-time indoor localization in smart homes using semi-supervised learning. In: AAAI
Cohn D, Atlas L, Ladner R (1994) Improving generalization with active learning. Mach Learn 15(2):201–221
Zliobaite I, Bifet A, Holmes G, Pfahringer B (2011) MOA concept drift active learning strategies for streaming data. In: Proceedings of the second workshop on applications of pattern analysis
Saurav S (2018) Online anomaly detection with concept drift adaptation using recurrent neural networks. In: Proceedings of the ACM India joint international conference on data science and management of data, ACM
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Cho K (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation, In: arXiv preprint arXiv:1406.1078
Gerstner W, Kistler WM (2002) Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, Cambridge
Lobo JL et al (2018) Evolving spiking neural networks for online learning over drifting data streams. Neural Netw 108:1–19
Budiman A, Fanany MI, Basaruddin C (2016) Adaptive convolutional ELM for concept drift handling in online stream data. In: arXiv preprint arXiv:1610.02348
Sethi TS, Kantardzic M (2018) Handling adversarial concept drift in streaming data. Expert Syst Appl 97:18–40
Niyaz Q, Sun W, Javaid AY, Alam M (2016) A deep learning approach for network intrusion detection system. In: Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS) ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)
Abramson M (2015) Oward adversarial online learning and the science of deceptive machines. In: AAAI fall symposium series
Chinavle D et al (2009), Ensembles in adversarial classification for spam. In: Proceedings of the 18th ACM conference on Information and knowledge management, ACM
Grosse K et al (2017) On the (statistical) detection of adversarial examples. In: arXiv preprint arXiv
Kantchelian A et al (2013) Approaches to adversarial drift. In: Proceedings of the 2013 ACM workshop on artificial intelligence and security, ACM
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Seraj, R., Ahmed, M. (2020). Concept Drift for Big Data. In: Fadlullah, Z., Khan Pathan, AS. (eds) Combating Security Challenges in the Age of Big Data. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-030-35642-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-35642-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35641-5
Online ISBN: 978-3-030-35642-2
eBook Packages: Computer ScienceComputer Science (R0)