Abstract
Many applications processing dynamic data require to filter, aggregate, join as well as to recognize event patterns in streams of data in an online fashion. However, data analysis and complex event processing (CEP) on high volume and/or high rate streams are challenging tasks. Typically, partitioning techniques are leveraged for achieving low latency and scalable processing. Unfortunately, sequence-based operations such as CEP operations as well as long-running continuous queries make partitioning much more difficult than for batch-oriented approaches.
In this paper, we address this challenge by presenting partitioning strategies for CEP queries. We discuss two strategies for stream and pattern partitioning and we present a cost-based optimization approach for determining the number of partitions as well as the split points in the queries to achieve better load balancing and avoid congestions of processing nodes in a cluster environment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Brenna, L., Gehrke, J., Hong, M., Johansen, D.: Distributed event stream processing with non-deterministic finite automata. In: DEBS 2009, pp. 3:1–3:12 (2009)
Condie, T., Conway, N., Alvaro, P., Hellerstein, J.M., Elmeleegy, K., Sears, R.: MapReduce online. Technical Report UCB/EECS-2009-136, EECS Department, UC, Berkeley (October 2009)
Hirzel, M.: Partition and compose: Parallel complex event processing. In: DEBS Conference, pp. 191–200. ACM (2012)
Krämer, J., Seeger, B.: Semantics and implementation of continuous sliding window queries over data streams. ACM Trans. Database Syst. 34(1) (2009)
Neumeyer, L., Robbins, B., Nair, A., Kesari, A.: S4: Distributed stream computing platform. In: ICDMW 2010, pp. 170–177 (2010)
Schultz-Møller, N.P., Migliavacca, M., Pietzuch, P.: Distributed complex event processing with query rewriting. In: DEBS Conference, pp. 4:1–4:12. ACM (2009)
Viglas, S., Naughton, J.F.: Rate-based query optimization for streaming information sources. In: SIGMOD Conference, pp. 37–48 (2002)
Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: SIGMOD Conference, pp. 407–418 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Saleh, O., Betz, H., Sattler, KU. (2015). Partitioning for Scalable Complex Event Processing on Data Streams. In: Bassiliades, N., et al. New Trends in Database and Information Systems II. Advances in Intelligent Systems and Computing, vol 312. Springer, Cham. https://doi.org/10.1007/978-3-319-10518-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-10518-5_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10517-8
Online ISBN: 978-3-319-10518-5
eBook Packages: EngineeringEngineering (R0)