Advertisement

Efficient Pattern Detection Over a Distributed Framework

  • Ahmed Khan LeghariEmail author
  • Martin Wolf
  • Yongluan Zhou
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 206)

Abstract

In recent past, work has been done to parallelize pattern detection queries over event stream, by partitioning the event stream on certain keys or attributes. In such partitioning schemes the degree of parallelization totally relies on the available partition keys. A limited number of partitioning keys, or unavailability of such partitioning attributes noticeably affect the distribution of data among multiple nodes, and is a reason of potential data skew and improper resource utilization. Moreover, majority of the past implementations of complex event detection are based on a single machine, hence, they are immune to potential data skew that could be seen in a real distributed environment. In this study, we propose an event stream partitioning scheme that without considering any key attributes partitions the stream over time-windows. This scheme efficiently distributes the event stream partitions across network, and detects pattern sequences in distributed fashion. Our scheme also provides an effective means to minimize potential data skew and handles a substantial number of pattern queries across network.

References

  1. 1.
    Diao, Y., Immerman, N., Gyllstrom, D.: Sase+: An Agile Language for Kleene Closure Over Event Streams. ACM Press, New York (2007)Google Scholar
  2. 2.
    Agrawal, J., Diao, Y., Gyllstrom, D., Immerman, N.: Efficient pattern matching over event streams. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 147–160. ACM (2008)Google Scholar
  3. 3.
    Mei, Y., Madden, S.: Zstream: a cost-based query processor for adaptively detecting composite events. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pp. 193–206. ACM (2009)Google Scholar
  4. 4.
    Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, pp. 407–418. ACM (2006)Google Scholar
  5. 5.
    Liu, M., Ray, M., Rundensteiner, E.A., Dougherty, D.J., Gupta, C., Wang, S., Ari, I., Mehta, A.: Processing nested complex sequence pattern queries over event streams. In: Proceedings of the Seventh International Workshop on Data Management for Sensor Networks, pp. 14–19. ACM (2010)Google Scholar
  6. 6.
    Ramakrishnan, R., Cheng, M., Livny, M., Seshadri, P.: What’s next? sequence queries. In: Proceedings of International Conferene Management of Data. Citeseer (1994)Google Scholar
  7. 7.
    Liu, M., Li, M., Golovnya, D., Rundensteiner, E.A., Claypool, K.: Sequence pattern query processing over out-of-order event streams. In: IEEE 25th International Conference on Data Engineering, ICDE 2009, pp. 784–795. IEEE (2009)Google Scholar
  8. 8.
    Law, Y.N., Wang, H., Zaniolo, C.: Query languages and data models for database sequences and data streams. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases, vol. 30, pp. 492–503. VLDB Endowment (2004)Google Scholar
  9. 9.
    Seshadri, P., Livny, M., Ramakrishnan, R.: Sequence query processing. ACM SIGMOD Rec. 23, 430–441 (1994). ACMCrossRefGoogle Scholar
  10. 10.
    Zuo, X., Zhou, Y., Zhao, C.-H.: Elastic non-contiguous sequence pattern detection for data stream monitoring. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 599–608. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  11. 11.
    Gao, C., Wei, J., Xu, C., Cheung, S.: Sequential event pattern based context-aware adaptation. In: Proceedings of the Second Asia-Pacific Symposium on Internetware, p. 3. ACM (2010)Google Scholar
  12. 12.
    Gyllstrom, D., Agrawal, J., Diao, Y., Immerman, N.: On supporting kleene closure over event streams. In: ICDE, vol. 8, pp. 1391–1393 (2008)Google Scholar
  13. 13.
    Demers, A.J., Gehrke, J., Panda, B., Riedewald, M., Sharma, V., White, W.M., et al.: Cayuga: a general purpose event monitoring system. In: CIDR, vol. 7, pp. 412–422 (2007)Google Scholar
  14. 14.
    Balkesen, C., Dindar, N., Wetter, M., Tatbul, N.: Rip: run-based intra-query parallelism for scalable complex event processing. In: Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems, pp. 3–14. ACM (2013)Google Scholar
  15. 15.
    Brenna, L., Gehrke, J., Hong, M., Johansen, D.: Distributed event stream processing with non-deterministic finite automata. In: Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, p. 3. ACM (2009)Google Scholar
  16. 16.
    Chakravarthy, S., Krishnaprasad, V., Anwar, E., Kim, S.K.: Composite events for active databases: semantics, contexts and detection. VLDB 94, 606–617 (1994)Google Scholar
  17. 17.
    Peng, S., Li, Z., Li, Q., Chen, Q., Pan, W., Liu, H., Nie, Y.: Event detection over live and archived streams. In: Wang, H., Li, S., Oyama, S., Hu, X., Qian, T. (eds.) WAIM 2011. LNCS, vol. 6897, pp. 566–577. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  18. 18.
    Zdonik, S., Sibley, P., Rasin, A., Sweetser, V., Montgomery, P., Turner, J., Wicks, J., Zgolinski, A., Snyder, D., Humphrey, M., Williamson, C.: Streaming for dummies (2004)Google Scholar
  19. 19.
    Hirzel, M.: Partition and compose: parallel complex event processing. In: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, pp. 191–200. ACM (2012)Google Scholar
  20. 20.
    Schultz-Møller, N.P., Migliavacca, M., Pietzuch, P.: Distributed complex event processing with query rewriting. In: Proceedings of the Third ACM International Conference on Distributed Event-Based Systems, p. 4. ACM (2009)Google Scholar
  21. 21.
    Sadoghi, M., Singh, H., Jacobsen, H.A.: Towards highly parallel event processing through reconfigurable hardware. In: Proceedings of the Seventh International Workshop on Data Management on New Hardware, pp. 27–32. ACM (2011)Google Scholar
  22. 22.
    Hirzel, M., Andrade, H., Gedik, B., Kumar, V., Losa, G., Nasgaard, M., Soule, R., Wu, K.: Spl stream processing language specification. IBM Research, Yorktown Heights, NY, USA, Technical report RC24 897 (2009)Google Scholar
  23. 23.
    Stonebraker, M., Çetintemel, U., Zdonik, S.: The 8 requirements of real-time stream processing. ACM SIGMOD Rec. 34(4), 42–47 (2005)CrossRefGoogle Scholar
  24. 24.
    Golab, L., Özsu, M.T.: Issues in data stream management. ACM SIGMOD Rec. 32(2), 5–14 (2003)CrossRefGoogle Scholar
  25. 25.
    Wang, Y., Cao, K., Zhang, X.: Complex event processing over distributed probabilistic event streams. Comput. Math. Appl. 66(10), 1808–1821 (2013)CrossRefGoogle Scholar
  26. 26.
    Mani, M.: Efficient event stream processing: handling ambiguous events and patterns with negation. In: Xu, J., Yu, G., Zhou, S., Unland, R. (eds.) DASFAA Workshops 2011. LNCS, vol. 6637, pp. 415–426. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  27. 27.
    Kawashima, H., Kitagawa, H., Li, X.: Complex event processing over uncertain data streams. In: 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC), pp. 521–526. IEEE (2010)Google Scholar
  28. 28.
    Jiang, Q., Chakravarthy, S.: Scheduling strategies for a data stream management system. Computer Science & Engineering, BNCOD, pp. 16–30 (2004)Google Scholar
  29. 29.
    Sharaf, M.A., Labrinidis, A., Chrysanthis, P.K.: Scheduling continuous queries in data stream management systems. Proc. VLDB Endow. 1(2), 1526–1527 (2008)CrossRefGoogle Scholar
  30. 30.
    Wu, J., Tan, K.-L., Zhou, Y.: QoS-oriented multi-query scheduling over data streams. In: Zhou, X., Yokota, H., Deng, K., Liu, Q. (eds.) DASFAA 2009. LNCS, vol. 5463, pp. 215–229. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  31. 31.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pp. 1–16. ACM (2002)Google Scholar
  32. 32.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Thomas, D.: Operator scheduling in data stream systems. VLDB J. Int. J. Very Large Data Bases 13(4), 333–353 (2004)CrossRefGoogle Scholar
  33. 33.
    Babcock, B., Babu, S., Motwani, R., Datar, M.: Chain: operator scheduling for memory minimization in data stream systems. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp. 253–264. ACM (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Ahmed Khan Leghari
    • 1
    Email author
  • Martin Wolf
    • 1
  • Yongluan Zhou
    • 1
  1. 1.Institute of Mathematics and Computer Science (IMADA)University of Southern DenmarkOdenseDenmark

Personalised recommendations