Advertisement

Drift Detection over Non-stationary Data Streams Using Evolving Spiking Neural Networks

  • Jesus L. Lobo
  • Javier Del Ser
  • Ibai Laña
  • Miren Nekane Bilbao
  • Nikola Kasabov
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 798)

Abstract

Drift detection in changing environments is a key factor for those active adaptive methods which require trigger mechanisms for drift adaptation. Most approaches are relied on a base learner that provides accuracies or error rates to be analyzed by an algorithm. In this work we propose the use of evolving spiking neural networks as a new form of drift detection, which resorts to the own architectural changes of this particular class of models to estimate the drift location without requiring any external base learner. By virtue of its inherent simplicity and lower computational cost, this embedded approach can be suitable for its adoption in online learning scenarios with severe resource constraints. Experiments with synthetic datasets show that the proposed technique is very competitive when compared to other drift detection techniques.

Keywords

Online learning Concept drift Spiking neural networks 

Notes

Acknowledgements

This work was supported by the EU project Pacific Atlantic Network for Technical Higher Education and Research - PANTHER (grant number 2013-5659/004-001 EMA2), and by the Basque Government through the EMAITEK program.

References

  1. 1.
    Zhou, Z.H., Chawla, N.V., Jin, Y., Williams, G.J.: Big data opportunities and challenges: discussions from data analytics perspectives. IEEE Comput. Intell. Mag. 9(4), 62–74 (2014)CrossRefGoogle Scholar
  2. 2.
    Alippi, C.: Intelligence for Embedded Systems. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  3. 3.
    Domingos, P., Hulten, G.: A general framework for mining massive data streams. J. Comput. Graph. Stat. 12(4), 945–949 (2003)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Ditzler, G., Roveri, M., Alippi, C., Polikar, R.: Learning in nonstationary environments: a survey. IEEE Comput. Intell. Mag. 10(4), 12–25 (2015)CrossRefGoogle Scholar
  5. 5.
    Khamassi, I., Sayed-Mouchaweh, M., Hammami, M., Ghédira, K.: Discussion and review on evolving data streams and concept drift adapting. Evolving Syst. 9(1), 1–23 (2018)CrossRefGoogle Scholar
  6. 6.
    Gonçalves Jr., P.M., de Carvalho Santos, S.G., Barros, R.S., Vieira, D.C.: A comparative study on concept drift detectors. Expert Syst. Appl. 41(18), 8144–8156 (2014)CrossRefGoogle Scholar
  7. 7.
    Demšar, J., Bosnić, Z.: Detecting concept drift in data streams using model explanation. Expert Syst. Appl. 92, 546–559 (2018)CrossRefGoogle Scholar
  8. 8.
    Minku, L.L., Yao, X.: DDD: a new ensemble approach for dealing with concept drift. IEEE Trans. Knowl. Data Eng. 24(4), 619–633 (2012)CrossRefGoogle Scholar
  9. 9.
    Gonçalves Jr., P.M., De Barros, R.S.M.: RCD: a recurring concept drift framework. Pattern Recogn. Lett. 34(9), 1018–1025 (2013)CrossRefGoogle Scholar
  10. 10.
    Dehghan, M., Beigy, H., ZareMoodi, P.: A novel concept drift detection method in data streams using ensemble classifiers. Intell. Data Anal. 20(6), 1329–1350 (2016)CrossRefGoogle Scholar
  11. 11.
    Brzezinski, D., Stefanowski, J.: Ensemble diversity in evolving data streams. In: International Conference on Discovery Science, pp. 229–244. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  12. 12.
    Lobo, J.L., Del Ser, J., Bilbao, M.N., Perfecto, C., Salcedo-Sanz, S.: DRED: an evolutionary diversity generation method for concept drift adaptation in online learning environments. Appl. Soft Comput. 68, 693–709 (2017)CrossRefGoogle Scholar
  13. 13.
    Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar
  14. 14.
    Soltic, S., Kasabov, N.: Knowledge extraction from evolving spiking neural networks with rank order population coding. Int. J. Neural Syst. 20(06), 437–445 (2010)CrossRefGoogle Scholar
  15. 15.
    Schliebs, S., Kasabov, N.: Evolving spiking neural network: a survey. Evolving Syst. 4(2), 87–98 (2013)CrossRefGoogle Scholar
  16. 16.
    Gama, J., Zliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4), 44 (2014)CrossRefGoogle Scholar
  17. 17.
    Wald, A.: Sequential Analysis. Courier Corporation, New York City (1973)zbMATHGoogle Scholar
  18. 18.
    Page, E.S.: Continuous inspection schemes. Biometrika 41(1/2), 100–115 (1954)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ross, G.J., Adams, N.M., Tasoulis, D.K., Hand, D.J.: Exponentially weighted moving average charts for detecting concept drift. Pattern Recogn. Lett. 33(2), 191–198 (2012)CrossRefGoogle Scholar
  20. 20.
    Bifet, A., Gavalda, R.: Learning from time-changing data with adaptive windowing. In: Proceedings of the 2007 SIAM International Conference on Data Mining, SIAM, pp. 443–448 (2007)CrossRefGoogle Scholar
  21. 21.
    Minku, L.L.: Online ensemble learning in the presence of concept drift. Ph.D. thesis, University of Birmingham (2011)Google Scholar
  22. 22.
    Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Brazilian symposium on artificial intelligence, pp. 286–295. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Baena-García, M., del Campo-Ávila, J., Fidalgo, R., Bifet, A., Gavaldà, R., Morales-Bueno, R.: Early drift detection method. In: Fourth International Workshop on Knowledge Discovery from Data Streams (2006)Google Scholar
  24. 24.
    Bach, S.H., Maloof, M.A.: Paired learners for concept drift. In: Eighth IEEE International Conference on Data Mining, ICDM 2008, pp. 23–32. IEEE (2008)Google Scholar
  25. 25.
    Sobhani, P., Beigy, H.: New drift detection method for data streams. In: Adaptive and intelligent systems, pp. 88–97. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  26. 26.
    Nishida, K., Yamauchi, K.: Detecting concept drift using statistical testing. In: International Conference on Discovery Science, pp. 264–269. Springer, Heidelberg (2007)Google Scholar
  27. 27.
    Barros, R.S., Cabral, D.R., Gonçalves Jr., P.M., Santos, S.G.: RDDM: reactive drift detection method. Expert Syst. Appl. 90, 344–355 (2017)CrossRefGoogle Scholar
  28. 28.
    Wang, J., Belatreche, A., Maguire, L., Mcginnity, T.M.: An online supervised learning method for spiking neural networks with adaptive structure. Neurocomputing 144, 526–536 (2014)CrossRefGoogle Scholar
  29. 29.
    Wang, J., Belatreche, A., Maguire, L., McGinnity, M.: Online versus offline learning for spiking neural networks: a review and new strategies. In: 2010 IEEE 9th International Conference on Cybernetic Intelligent Systems (CIS), pp. 1–6. IEEE (2010)Google Scholar
  30. 30.
    Wang, J., Belatreche, A., Maguire, L.P., McGinnity, T.M.: SpikeTemp: an enhanced rank-order-based learning approach for spiking neural networks with adaptive structure. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 30–43 (2017)CrossRefGoogle Scholar
  31. 31.
    Kasabov, N.K.: Evolving Connectionist Systems: The Knowledge Engineering Approach. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  32. 32.
    Thorpe, S.J., Gautrais, J.: Rapid visual processing using spike asynchrony. In: Advances in Neural Information Processing Systems, pp. 901–907 (1997)Google Scholar
  33. 33.
    Bohte, S.M., Kok, J.N., La Poutre, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4), 17–37 (2002)CrossRefGoogle Scholar
  34. 34.
    Thorpe, S., Gautrais, J.: Rank order coding. In: Computational Neuroscience, pp. 113–118. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  35. 35.
    Minku, L.L., White, A.P., Yao, X.: The impact of diversity on online ensemble learning in the presence of concept drift. IEEE Trans. Knowl. Data Eng. 22(5), 730–742 (2010)CrossRefGoogle Scholar
  36. 36.
    Frías-Blanco, I., del Campo-Ávila, J., Ramos-Jiménez, G., Morales-Bueno, R., Ortiz-Díaz, A., Caballero-Mota, Y.: Online and non-parametric drift detection methods based on hoeffdings bounds. IEEE Trans. Knowl. Data Eng. 27(3), 810–823 (2015)CrossRefGoogle Scholar
  37. 37.
    Gao, J., Ding, B., Fan, W., Han, J., Philip, S.Y.: Classifying data streams with skewed class distributions and concept drifts. IEEE Internet Comput. 12(6) (2008)CrossRefGoogle Scholar
  38. 38.
    Pears, R., Sakthithasan, S., Koh, Y.S.: Detecting concept change in dynamic data streams. Mach. Learn. 97(3), 259–293 (2014)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Jesus L. Lobo
    • 1
  • Javier Del Ser
    • 1
    • 2
    • 3
  • Ibai Laña
    • 1
  • Miren Nekane Bilbao
    • 2
  • Nikola Kasabov
    • 4
  1. 1.TECNALIADerioSpain
  2. 2.University of the Basque Country UPV/EHUBilbaoSpain
  3. 3.Basque Center for Applied Mathematics (BCAM)BilbaoSpain
  4. 4.KEDRI - Auckland University of Technology (AUT)AucklandNew Zealand

Personalised recommendations