Advertisement

PAS3-HSID: a Dynamic Bio-Inspired Approach for Real-Time Hot Spot Identification in Data Streams

  • Rebecca Tickle
  • Isaac TrigueroEmail author
  • Grazziela P. Figueredo
  • Mohammad Mesgarpour
  • Robert I. John
Article
  • 24 Downloads

Abstract

Hot spot identification is a very relevant problem in a wide variety of areas such as health care, energy or transportation. A hot spot is defined as a region of high likelihood of occurrence of a particular event. To identify hot spots, location data for those events is required, which is typically collected by telematics devices. These sensors are constantly gathering information, generating very large volumes of data. Current state-of-the-art solutions are capable of identifying hot spots from big static batches of data by means of variations of clustering or instance selection techniques that pre-process the original input data, providing the most relevant locations. However, these approaches neglect to address changes in hot spots over time. This paper presents a dynamic bio-inspired approach to detect hot spots in big data streams. This computational intelligence method is designed and applied to the transportation sector as a case study to identify incidents in the roads caused by heavy goods vehicles. We adapt an immune-based algorithm to account for the temporary aspect of hot spots inspired by the idea of pheromones, which is then subsequently implemented using Apache Spark Streaming. Experimental results on real datasets with up to 4.5 million data points—provided by a telematics company—show that the algorithm is capable of quickly processing large streaming batches of data, as well as successfully adapting over time to detect hot spots. The outcome of this method is twofold, both reducing data storage requirements and demonstrating resilience to sudden changes in the input data (concept drift).

Keywords

Hot spots Road incidents Instance selection Telematics data Big data streams Computational intelligence 

Notes

Acknowledgements

The authors would like to thank the Soft Computing and Intelligent Information Systems research group from the University of Granada, for allowing us to use their big data infrastructure to carry out the experiments.

Compliance with Ethical Standards

Conflict of Interests

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. 1.
    Alpaydin E. Introduction to machine learning. Cambridge: The MIT Press; 2014.Google Scholar
  2. 2.
    Anderson TK. Kernel density estimation and k-means clustering to profile road accident hotspots. Accid Anal Prev 2009;41(3):359–64.CrossRefGoogle Scholar
  3. 3.
    Barros RSM, Santos SGTC. A large-scale comparison of concept drift detectors. Inf Sci 2018;451-452:348–70.CrossRefGoogle Scholar
  4. 4.
    Beringer J, Hüllermeier E. Efficient instance-based learning on data streams. Intelligent Data Analysis 2007; 11(6):627–50.CrossRefGoogle Scholar
  5. 5.
    Braithwaite A, Li Q. Transnational terrorism hot spots: identification and impact evaluation. Conflict Management and Peace Science 2007;24(4):281–96.CrossRefGoogle Scholar
  6. 6.
    Cambria E, Chattopadhyay A, Linn E, Mandal B, White B. Storages are not forever. Cogn Comput 2017;9(5):646–58.CrossRefGoogle Scholar
  7. 7.
    Cheng W, Washington SP. Experimental evaluation of hotspot identification methods. Accid Anal Prev 2005;37(5):870–81.CrossRefGoogle Scholar
  8. 8.
    Chu F, Zaniolo C. Fast and light boosting for adaptive mining of data streams. Advances in Knowledge Discovery and Data Mining, p 282–92. In: Dai H, Srikant R, and Zhang C, editors; 2004.Google Scholar
  9. 9.
    Dean J, Ghemawat S. MapReduce: a flexible data processing tool. Commun ACM 2010;53(1):72–7.CrossRefGoogle Scholar
  10. 10.
    Ding S, Zhang J, Jia H, Qian J. An adaptive density data stream clustering algorithm. Cogn Comput 2016;8(1):30–8.CrossRefGoogle Scholar
  11. 11.
    Dorigo M, Di Caro G. Ant colony optimization: a new meta-heuristic. Proceedings of the 1999 congress on evolutionary computation, 1999. IEEE; 1999. p. 1470–7.Google Scholar
  12. 12.
    Dorigo M, Maniezzo V, Colorni A. Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern B (Cybernetics) 1996;26(1):29–41.CrossRefGoogle Scholar
  13. 13.
    Elen B, Peters J, van Poppel M, Bleux N, Theunis J, Reggente M, Standaert A. The Aeroflex: a bicycle for mobile air quality measurements. Sensors (Switzerland) 2013;13(1):221–40.CrossRefGoogle Scholar
  14. 14.
    Ester M, Kriegel HP, Sander J, Xu X, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. Kdd; 1996. p. 226–31.Google Scholar
  15. 15.
    Figueredo GP, Ebecken NFF, Augusto DA, Barbosa HJC. An immune-inspired instance selection mechanism for supervised classification. Memetic Computing 2012;4:135–47.CrossRefGoogle Scholar
  16. 16.
    Figueredo GP, Ebecken NFF, Barbosa HJC. The SUPRAIC algorithm: a suppression immune based mechanism to find a representative training set in data classification tasks. ICARIS, Lecture notes in computer science. Berlin: Springer; 2007. p. 59–70.Google Scholar
  17. 17.
    Figueredo GP, Triguero I, Mesgarpour M, Guerra AM, Garibaldi JM, John RI. An immune-inspired technique to identify heavy goods vehicles incident hot spots. IEEE Transactions on Emerging Topics in Computational Intelligence 2017;1(4):248–58.CrossRefGoogle Scholar
  18. 18.
    Gama J. Knowledge discovery from data streams, 1st ed. Boca Raton: Chapman & hall/CRC; 2010.CrossRefGoogle Scholar
  19. 19.
    García S, Derrac J, Cano J, Herrera F. Prototype selection for nearest neighbor classification: taxonomy and empirical study. IEEE Trans Pattern Anal Mach Intell 2012;34(3):417–35.CrossRefGoogle Scholar
  20. 20.
    García S, Luengo J, Herrera F. Data preprocessing in data mining. Berlin: Springer Publishing Company, Incorporated; 2014.Google Scholar
  21. 21.
    Han J, Kamber M, Tung AKH. Spatial clustering methods in data mining: a survey. In: Miller HJ and Han J, editors. Milton Park: Taylor and Francis; 2001.Google Scholar
  22. 22.
    Han J, Pei J, Kamber M. Data mining: concepts and techniques. Amsterdam: Elsevier; 2011.Google Scholar
  23. 23.
    Heylighen F. Stigmergy as a universal coordination mechanism I: definition and components. Cogn Syst Res 2016; 38:4–13.  https://doi.org/10.1016/j.cogsys.2015.12.002. Special Issue of Cognitive Systems Research – Human-Human Stigmergy.CrossRefGoogle Scholar
  24. 24.
    Hulten G, Spencer L, Domingos P. Mining time-changing data streams. Proceedings of the Seventh ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’01. New York: ACM; 2001. p. 97–106.Google Scholar
  25. 25.
    Klinkenberg R. Learning drifting concepts: example selection vs. example weighting. Intelligent Data Analysis 2004;8(3):281–300.CrossRefGoogle Scholar
  26. 26.
    Krawczyk B. Active and adaptive ensemble learning for online activity recognition from data streams. Knowl-Based Syst 2017;138:69–78.CrossRefGoogle Scholar
  27. 27.
    Krawczyk B, Cano A. Online ensemble learning with abstaining classifiers for drifting and noisy data streams. Appl Soft Comput 2018;68:677–92.CrossRefGoogle Scholar
  28. 28.
    Krawczyk B, Minku LL, Gama J, Stefanowski J, Woźniak M. Ensemble learning for data stream analysis: a survey. Information Fusion 2017;37:132–56.CrossRefGoogle Scholar
  29. 29.
    Krempl G, žliobaite I, Brzeziński D, Hüllermeier E, Last M, Lemaire V, Noack T, Shaker A, Sievi S, Spiliopoulou M, Stefanowski J. Open challenges for data stream mining research. SIGKDD Explor Newsl 2014;16(1):1–0.CrossRefGoogle Scholar
  30. 30.
    Mesgarpour M, Landa-Silva D, Dickinson I. Overview of telematics-based prognostics and health management systems for commercial vehicles. Activities of Transport Telematics 2013;395:123–30.CrossRefGoogle Scholar
  31. 31.
    Molina D, LaTorre A, Herrera F. An insight into bio-inspired and evolutionary algorithms for global optimization: Review, analysis, and lessons learnt over a decade of competitions. Cogn Comput 2018;10(4):517–44.  https://doi.org/10.1007/s12559-018-9554-0.CrossRefGoogle Scholar
  32. 32.
    Montella A. A comparative analysis of hotspot identification methods. Accid Anal Prev 2010;42(2):571–81.CrossRefGoogle Scholar
  33. 33.
    Passini MLC, Estébanez KB, Figueredo GP, Ebecken NFF. A strategy for training set selection in text classification problems. Int J Adv Comput Sci Appl 2013;4(6):54–60.Google Scholar
  34. 34.
    Perallos A, Hernandez-Jayo U, Onieva E, García-zuazola IJ. Intelligent transport systems: technologies and applications, 1st ed. Hoboken: Wiley Publishing; 2015.CrossRefGoogle Scholar
  35. 35.
    Ramírez-Gallego S, Krawczyk B, García S, Woźniak M, Herrera F. A survey on data preprocessing for data stream mining: current status and future directions. Neurocomputing 2017;239:39–57.CrossRefGoogle Scholar
  36. 36.
    Shen YY, Liu CL. Incremental adaptive learning vector quantization for character recognition with continuous style adaptation. Cogn Comput 2018;10(2):334–46.CrossRefGoogle Scholar
  37. 37.
    Shirkhorshidi AS, Aghabozorgi S, Wah TY, Herawan T. 2014. Big data clustering: a review. In: International conference on computational science and its applications, Springer; p. 707–20.Google Scholar
  38. 38.
    Siddique N, Adeli H. Nature inspired computing: an overview and some future directions. Cogn Comput 2015;7 (6):706–14.CrossRefGoogle Scholar
  39. 39.
    Sousa R, Gama J. Multi-label classification from high-speed data streams with adaptive model rules and random rules. Progress in Artificial Intelligence 2018;7(3):177–87.CrossRefGoogle Scholar
  40. 40.
    Street WN, Kim Y. 2001. A streaming ensemble algorithm (SEA) for large-scale classification. In: Proceedings of the Seventh ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’01, p. 377–82.Google Scholar
  41. 41.
    Triguero I, Figueredo GP, Mesgarpour M, Garibaldi JM, John RI. 2017. Vehicle incident hot spots identification: an approach for big data. In: 2017 IEEE Trustcom/bigdataSE/ICESS, p. 901–8.Google Scholar
  42. 42.
    Van Brummelen G. Heavenly mathematics: the forgotten art of spherical trigonometry. Princeton: Princeton University Press; 2012.CrossRefGoogle Scholar
  43. 43.
    Zaharia M, Chowdhury M, Das T, Dave A, Ma J, McCauley M, Franklin MJ, Shenker S, Stoica I. 2012. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX conference on networked systems design and implementation, NSDI’12, p. 15–28.Google Scholar
  44. 44.
    Zaharia M, Das T, Li H, Shenker S, Stoica I. 2012. Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. In: Proceedings of the 4th USENIX conference on hot topics in cloud computing, p. 10–0.Google Scholar
  45. 45.
    Zhao L, Wang L, Xu Q. Data stream classification with artificial endocrine system. Appl Intell 2012;37 (3):390–404.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.The Automated Scheduling Optimisation and Planning Research Group, School of Computer ScienceUniversity of NottinghamNottinghamUK
  2. 2.The Advanced Data Analysis Centre, School of Computer ScienceUniversity of NottinghamNottinghamUK
  3. 3.MicroliseEastwoodUK

Personalised recommendations