Skip to main content
SpringerLink
Log in

Parallel Computing TEDA for High Frequency Streaming Data Clustering

  • Conference paper
  • First Online:
Advances in Big Data (INNS 2016)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 529))

Included in the following conference series:

Abstract

In this paper, a novel online clustering approach called Parallel_TEDA is introduced for processing high frequency streaming data. This newly proposed approach is developed within the recently introduced TEDA theory and inherits all advantages from it. In the proposed approach, a number of data stream processors are involved, which collaborate with each other efficiently to achieve parallel computation as well as a much higher processing speed. A fusion center is involved to gather the key information from the processors which work on chunks of the whole data stream and generate the overall output. The quality of the generated clusters is being monitored within the data processors all the time and stale clusters are being removed to ensure the correctness and timeliness of the overall clustering results. This, in turn, gives the proposed approach a stronger ability of handling shifts/drifts that may take place in live data streams. The numerical experiments performed with the proposed new approach Parallel_TEDA on benchmark datasets present higher performance and faster processing speed when compared with the alternative well-known approaches. The processing speed has been demonstrated to fall exponentially with more data processors involved. This new online clustering approach is very suitable and promising for real-time high frequency streaming processing and data analytics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fukunaga, K., Hostetler, L.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theor. 21(1), 32–40 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  2. MacQueen, J.: Some methods for classification and analysis of multi-variate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Statistics, vol. 1, Berkeley, pp. 281–297 (1967)

    Google Scholar 

  3. Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm. Comput. Geosci. 10(2), 191–203 (1984)

    Article  Google Scholar 

  4. Johnson, S.: Hierarchical clustering schemes. Psychometrika 32(3), 241–254 (1967)

    Article  Google Scholar 

  5. de Oliveira, J.V., Pedrycz, W. (eds.): Advances in Fuzzy Clustering and Its Applications. Wiley, New York (2007)

    Google Scholar 

  6. Yager, R., Filev, D.: Generation of fuzzy rules by mountain clustering. J. Intell. Fuzzy Syst. 2(3), 209–219 (1994)

    Google Scholar 

  7. Chiu, S.L.: Fuzzy model identification based on cluster estimation. J. Intell. Fuzzy Syst. 2(3), 1064–1246 (1994)

    Google Scholar 

  8. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: 2nd International Conference on Knowledge Discovery and Data Mining, vol. 96(34), Portland, Oregan, pp. 226–231 (1996)

    Google Scholar 

  9. Wang, C., Lai, J., Huang, D., Zheng, W.: SVStream: a support vector-based algorithm for clustering data streams. IEEE Trans. Knowl. Data Eng. 25(6), 1410–1424 (2013)

    Article  Google Scholar 

  10. Baruah, R., Angelov, P.: Evolving local means method for clustering of streaming data. In: IEEE Congress on Computational Intelligence, Brisbane, Australia, pp. 2161–2168 (2012)

    Google Scholar 

  11. Hyde, R., Angelov, P.: A fully autonomous data density based clustering technique. In: IEEE Symposium on Evolving and Autonomous Learning Systems, Orlando, USA, pp. 116–123 (2014)

    Google Scholar 

  12. Angelov, P., Gu, X., Gutierrez, G., Iglesias, J.A., Sanchis, A.: Autonomous data density based clustering method. In: 2016 IEEEWorld Congress on Computational Intelligence, Vancouver, Canada, pp. 2405-2413 (2016)

    Google Scholar 

  13. Guha, S., Mishra, N., Motwani, R., O’Callaghan, L.: Clustering data streams. In: Proceedings of the Annual Symposium on Foundations of Computer Scuebce (FOCS), Redondo Beach, CA, pp. 359–366 (2000)

    Google Scholar 

  14. Guha, S., Meyerson, A., Mishra, N., Motwani, R., O’Callaghan, L.: Clustering data streams: theory and practice. IEEE Trans. Knowl. Data Eng. 15(3), 515–528 (2003)

    Article  Google Scholar 

  15. Aggarwal, C., Han, J., Wang, J., Yu, S.: A framework for clustering evolving data streams. In: Proceedings of the International Conference on Very Large Data Bases, Berlin, Germany, pp. 81–92 (2003)

    Google Scholar 

  16. Comode, G., Muthukrishnan, S., Zhang, W.: Conquering the divide: continuous clustering of distributed data streams. In: Proceedings of the International Conference on Data Engineering, Istanbul, pp. 1036–1045 (2007)

    Google Scholar 

  17. Gama, J., Rodrigues, P., Sebastio, R.: Evaluating algorithms that learn from data streams. In: Proceedings of the ACM Symposium on Applied Computing, Hawaii, pp. 1496–1500 (2009)

    Google Scholar 

  18. Angelov, P.: Outside the box: an alternative data analytics framework. J. Autom. Mob. Rob. Intell. Syst. 8(2), 53–59 (2014)

    Google Scholar 

  19. Angelov, P.: Typicality distribution function - a new density-based data analytics tool. In: IEEE International Joint Conference on Neural Networks (IJCNN), Killarney, pp. 1–8 (2015)

    Google Scholar 

  20. Angelov, P., Gu, X., Kangin, D., Principe, J.: Empirical data analysis: a new tool for data analytics. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics, Budapest, Hungary (2016, to appear)

    Google Scholar 

  21. Angelov, P., Filev, D.: Simple_TS: a simplified method for learning evolving TakagiSugeno fuzzy models. In: IEEE International Conference on Fuzzy Systems, Reno, USA, pp. 1068–1073 (2005)

    Google Scholar 

  22. Angelov, P.: Autonomous Learning Systems from Data Stream to Knowledge in Real Time. John Wiley & Sons, Ltd., West Sussex (2012)

    Book  Google Scholar 

  23. Lughofer, E., Angelov, P.: Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl. Soft Comput. J. 11(2), 2057–2068 (2011)

    Article  Google Scholar 

  24. Saw, J., Yang, M., Mo, T.: Chebyshev inequality with estimated mean and variance. Am. Stat. 38(2), 130–132 (1984)

    MathSciNet  Google Scholar 

  25. Clustering Datasets - University of Eastern Finland (2016). http://cs.joensuu.fi/sipu/datasets/. Access 12 May 2016

Download references

Acknowledgements

The second author would like to acknowledge the partial support through The Royal Society grant IE141329/2014 Novel Machine Learning Paradigms to Address Big Data Streams. The third, fourth, and fifth authors would like to acknowledge the support by the Spanish Goverment under the project TRA2013-48314-C3-1-R and the project TRA2015-63708-R.

Author information

Authors and Affiliations

  1. Data Science Group, School of Computing and Communications, Lancaster University, Lancaster, LA1 4WA, UK

    Xiaowei Gu & Plamen P. Angelov

  2. CAOS Research Group, Computer Science Department, Carlos III University of Madrid, 28911, Leganes, Spain

    German Gutierrez, Jose Antonio Iglesias & Araceli Sanchis

Authors
  1. Xiaowei Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Plamen P. Angelov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. German Gutierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jose Antonio Iglesias
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Araceli Sanchis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Plamen P. Angelov .

Editor information

Editors and Affiliations

  1. School of Computing and Communications, Lancaster University , Lancaster, United Kingdom

    Plamen Angelov

  2. Data Engineering Lab, Dept. of Informatics, Aristotle University of Thessaloniki , Thessaloniki, Greece

    Yannis Manolopoulos

  3. Lab of Forest Informatics (FiLAB), Democritus University of Thrace , Orestiada, Greece

    Lazaros Iliadis

  4. WPC Information Systems Faculty, Arizona State University , Tempe, Arizona, USA

    Asim Roy

  5. Electrical Engineering Dept, (ICA), Pontifical Catholic Univ of Rio de Janei , Rio de Janeiro, Rio de Janeiro, Brazil

    Marley Vellasco

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Gu, X., Angelov, P.P., Gutierrez, G., Iglesias, J.A., Sanchis, A. (2017). Parallel Computing TEDA for High Frequency Streaming Data Clustering. In: Angelov, P., Manolopoulos, Y., Iliadis, L., Roy, A., Vellasco, M. (eds) Advances in Big Data. INNS 2016. Advances in Intelligent Systems and Computing, vol 529. Springer, Cham. https://doi.org/10.1007/978-3-319-47898-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47898-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47897-5

  • Online ISBN: 978-3-319-47898-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation