Skip to main content

A Heuristically Optimized Complex Event Processing Engine for Big Data Stream Analytics

  • Conference paper
  • First Online:
Harmony Search Algorithm (ICHSA 2017)

Abstract

This paper describes a Big Data stream analytics platform developed within the DEWI project for processing upcoming events from wireless sensors installed in a truck. The platform consists of a Complex Event Processing (CEP) engine capable of triggering alarms from a predefined set of rules. In general these rules are characterized by multiple parameters, for which finding their optimal value usually yields a challenging task. In this paper we explain a methodology based on a meta-heuristic solver that is used as a wrapper to obtain optimal parametric rules for the CEP engine. In particular this approach optimizes CEP rules through the refinement of the parameters controlling their behavior based on an alarm detection improvement criterion. As a result the proposed scheme retrieves the rules parameterized in a detection-optimal fashion. Results for a certain use case – i.e. fuel level of the vehicle – are discussed towards assessing the performance gains provided by our method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dewi project. http://www.dewiproject.eu. Accessed 15 Oct 2016

  2. Espertech. http://www.espertech.com/. Accessed 15 Oct 2016

  3. F1 score. https://en.wikipedia.org/wiki/F1_score. Accessed 15 Nov 2016

  4. Bäck, T., Schwefel, H.P.: An overview of evolutionary algorithms for parameter optimization. Evol. Comput. 1(1), 1–23 (1993)

    Article  Google Scholar 

  5. Björne, J., Heimonen, J., Ginter, F., Airola, A., Pahikkala, T., Salakoski, T.: Extracting complex biological events with rich graph-based feature sets. In: Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task, pp. 10–18. Association for Computational Linguistics (2009)

    Google Scholar 

  6. Bruns, R., Dunkel, J., Billhardt, H., Lujak, M., Ossowski, S.: Using complex event processing to support data fusion for ambulance coordination. In: 2014 17th International Conference on Information Fusion (FUSION), pp. 1–7. IEEE (2014)

    Google Scholar 

  7. Ding, L., Chen, S., Rundensteiner, E.A., Tatemura, J., Hsiung, W.P., Candan, K.S.: Runtime semantic query optimization for event stream processing. In: 2008 IEEE 24th International Conference on Data Engineering, pp. 676–685. IEEE (2008)

    Google Scholar 

  8. Geem, Z.W., Kim, J.H., Loganathan, G.: A new heuristic optimization algorithm: harmony search. Simulation 76(2), 60–68 (2001)

    Article  Google Scholar 

  9. Hirzel, M., Andrade, H., Gedik, B., Jacques-Silva, G., Khandekar, R., Kumar, V., Mendell, M., Nasgaard, H., Schneider, S., Soulé, R., et al.: IBM streams processing language: analyzing big data in motion. IBM J. Res. Dev. 57(3/4), 7:1–7:11 (2013)

    Article  Google Scholar 

  10. Liu, H.L., Chen, Q., Li, Z.H.: Optimization techniques for RFID complex event processing. J. Comput. Sci. Technol. 24(4), 723–733 (2009)

    Article  Google Scholar 

  11. Lu, N., Cheng, N., Zhang, N., Shen, X., Mark, J.W.: Connected vehicles: solutions and challenges. IEEE Internet Things J. 1(4), 289–299 (2014)

    Article  Google Scholar 

  12. Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, pp. 407–418. ACM (2006)

    Google Scholar 

  13. Yan, Y., Yang, Y., Meng, D., Liu, G., Tong, W., Hauptmann, A.G., Sebe, N.: Event oriented dictionary learning for complex event detection. IEEE Trans. Image Process. 24(6), 1867–1878 (2015)

    Article  MathSciNet  Google Scholar 

  14. Zhang, H., Diao, Y., Immerman, N.: On complexity and optimization of expensive queries in complex event processing. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 217–228. ACM (2014)

    Google Scholar 

Download references

Acknowledgments

The research from DEWI project (www.dewi-project.eu) [1] leading to these results has received funding from ARTEMIS Joint Undertaking under grant agreement no. 621353. The authors want to specially thank Parthasarathy Dhasarathy, from Volvo Technology AB, who helped in the use cases and the platform infrastructure definition.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ignacio (Iñaki) Olabarrieta .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Olabarrieta, I.(., Torre-Bastida, A.I., Laña, I., Campos-Cordobes, S., Del Ser, J. (2017). A Heuristically Optimized Complex Event Processing Engine for Big Data Stream Analytics. In: Del Ser, J. (eds) Harmony Search Algorithm. ICHSA 2017. Advances in Intelligent Systems and Computing, vol 514. Springer, Singapore. https://doi.org/10.1007/978-981-10-3728-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-3728-3_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-3727-6

  • Online ISBN: 978-981-10-3728-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics