Abstract
An often used approach for detecting and adapting to concept drift when doing classification is to treat the data as i.i.d. and use changes in classification accuracy as an indication of concept drift. In this paper, we take a different perspective and propose a framework, based on probabilistic graphical models, that explicitly represents concept drift using latent variables. To ensure efficient inference and learning, we resort to a variational Bayes inference scheme. As a proof of concept, we demonstrate and analyze the proposed framework using synthetic data sets as well as a real financial data set from a Spanish bank.
H. Borchani, A.M. Martínez, and A.R. Masegosa—These authors are considered as first authors and contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
AMIDST is an open source toolbox available at http://amidst.github.io/toolbox/ under the Apache Software License.
- 2.
For now, we shall assume that the total number of instances \(N_t\) does not vary with time t; this assumption is lifted in Sect. 5 when we consider the financial data set.
References
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer New York Inc., New York (2001)
Gaber, M.M., Zaslavsky, A.B., Krishnaswamy, S.: A survey of classification methods in data streams. In: Aggarwal, C.C. (ed.) Data Streams - Models and Algorithms. Advances in Database Systems, vol. 31, pp. 39–59. Springer, Berlin (2007)
Schlimmer, J.C., Granger, R.H.: Incremental learning from noisy data. Mach. Learn. 1, 317–354 (1986)
Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Mach. Learn. 23(1), 69–101 (1996)
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. 46, 44:1–44:37 (2014)
Jensen, F.V., Nielsen, T.D.: Bayesian Networks and Decision Graphs. Springer, Berlin (2007)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, New York (1973)
Zhong, S., Langseth, H., Nielsen, T.D.: A classification-based approach to monitoring the safety of dynamic systems. Reliab. Eng. Syst. Safety 121, 61–71 (2014)
Bach, S., Maloof, M.: A Bayesian approach to concept drift. In: Advances in Neural Information Processing Systems, pp. 127–135 (2010)
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37, 183–233 (1999)
Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010)
Martínez, M., Sucar, L.E.: Learning dynamic naive Bayesian classifiers. In: Proceedings of the Twenty-First International Florida Artificial Intelligence Research Symposium Conference, 655–659 (2008)
Cooper, G.F.: The computational complexity of probabilistic inference using Bayesian belief networks. Artif. Intell. 42, 393–405 (1990)
Beal, M.J.: Variational algorithms for approximate Bayesian inference. Ph.D. thesis, Gatsby Computational Neuroscience Unit, University College London (2003)
Winn, J.M., Bishop, C.M.: Variational message passing. J. Mach. Learn. Res. 6, 661–694 (2005)
Street, N., Kim, Y.: A streaming ensemble algorithm (SEA) for large-scale classification. In: 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 377–382 (2001)
Hulten, G., Spencer, L., Domingos, P.: Mining time changing data streams. In: Proceedings of the Seventh International Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)
Acknowledgments
This work was performed as part of the AMIDST project. AMIDST has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 619209. The data set has been provided by Banco de Crédito Cooperativo.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Borchani, H. et al. (2015). Modeling Concept Drift: A Probabilistic Graphical Model Based Approach. In: Fromont, E., De Bie, T., van Leeuwen, M. (eds) Advances in Intelligent Data Analysis XIV. IDA 2015. Lecture Notes in Computer Science(), vol 9385. Springer, Cham. https://doi.org/10.1007/978-3-319-24465-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-24465-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24464-8
Online ISBN: 978-3-319-24465-5
eBook Packages: Computer ScienceComputer Science (R0)