Skip to main content
SpringerLink
Log in

Aggressive Complex Event Processing with Confidence over Out-of-Order Streams

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In recent years, there has been a growing need for complex event processing (CEP), ranging from supply chain management to security monitoring. In many scenarios events are generated in different sources but arrive at the central server out of order, due to the differences of network latencies. Most state-of-the-art techniques process out-of-order events by buffering the events until the total event order within a specified range can be guaranteed. Their main problems are leading to increasing response time and reducing system throughput. This paper aims to build a high performance out-of-order event processing mechanism, which can match events as soon as they arrive instead of buffering them till all arrive. A suffix-automaton-based event matching algorithm is proposed to speed up query processing, and a confidence-based accuracy evaluation is proposed to control the query result quality. The performance of our approach is evaluated through detailed accuracy and response time analysis. As experimental results show, our approach can obviously speed up the query matching time and produce reasonable query results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chakravarthy S, Krishnaprasad V, Anwar E, Kim S K. Composite events for active databases: Semantics, contexts and detection. In Proc. VLDB, Santiago de Chile, Chile, Sept. 12-15, 1994, pp.606-617.

  2. Ré C, Letchner J, Balazinska M, Suciu D. Event queries on correlated probabilistic streams. In Proc. SIGMOD, Vancouver, Canada, Jun. 10–12, 2008, pp.715–728.

  3. Liu M, Li M, Golovnya D, Rundensteiner E A, Claypoo K. Sequence pattern query processing over out-of-order event streams. In Proc. ICDE, Shanghai, China, Mar. 29-Apr. 2, 2009, pp.784–795.

  4. Wu E, Diao Y, Rizvi S. High-performance complex event processing over streams. In Proc. SIGMOD, Chicago, USA, Jun. 27–29, 2006, pp.407–418.

  5. Barga R S, Goldstein J, Ali M, Hong M. Consistent streaming through time: A vision for event stream processing. In Proc. CIDR, Asilomar, USA, Jan. 7–10, 2007, pp.363–374.

  6. Gyllstrom D, Agrawal J, Diao Y, Immerman N. On supporting kleene closure over event streams. In Proc. ICDE, Cancun, Mexico, Apr. 7–12, 2008, pp.1391–1393.

  7. Demers A, Gehrke J, Panda B, Riedewald M, Sharma V, White W. Cayuga: A general purpose event monitoring system. In Proc. CIDR, Asilomar, USA, 2007, pp.412–422.

  8. Abadi D, Carney D, Cetintemel U et al. Aurora: A data stream management system. In Proc. the 2003 ACM SIGMOD International Conference on Management of Data, San Diego, USA, Jun. 9–12, 2003, pp.666–677.

  9. Babu S, Srivastava U, Widom J. Exploiting k-constraints to reduce memory overhead in continuous queries over data streams. IEEE Trans. Knowl. Data Eng., 2003, 13(3): 545–580.

    Google Scholar 

  10. Tucker P A, Maie D, Sheard T, Fegaras L. Exploiting punctuation semantics in continuous data streams. TKDE, 2003, 15(3): 555–568.

    Google Scholar 

  11. Babu S, Motwani R, Munagala K et al. Adaptive ordering of pipelined stream filters. In Proc. the 2004 ACM SIGMOD Int. Conf. Management of Data, Paris, France, Jun. 13–18, 2004, pp.407–418.

  12. Li J, Maier D, Tufte K, Papadimos V, Tucker P. Semantics and evaluation techniques for window aggregates in data streams. In Proc. the 2005 ACM SIGMOD International Conference on Management of Data, Baltimore, USA, Jun. 14–16, 2005, pp.311–322.

  13. Hwang J, Balazinska M, Rasin A, Cetintemel U, Stonebraker M, Zdonik S. High-availability algorithms for distributed stream processing. In Proc. the 21st International Conference on Data Engineering, 2005 (ICDE 2005), Tokyo, Japan, Apr. 5–8, 2005, pp.779–790.

  14. Ding L, Mehta N, Rundensteiner E A, Heineman G T. Joining punctuated streams. In Proc. EDBT, Heraklion, Greece, Mar. 14–18, 2004, pp.587–604.

  15. Cormode G, Korn F, Tirthapura S. Time-decaying aggregates in out-of-order streams. In Proc. PODS, Vancouver, Canada, Jun. 9–11, 2008, pp.89–98.

  16. Shrivastava N, Buragohain C, Agrawal D et al. Medians and beyond: New aggregation techniques for sensor networks. In Proc. SenSys, Baltimore, USA, Nov. 3–5, 2004, pp.239–249.

  17. Golab L, Ozsu M. Update-pattern-aware modeling and processing of continuous queries. In Proc. the 2005 ACM SIGMOD International Conference on Management of Data, Baltimore, USA, 2005, pp.658–669.

  18. Ryvkina E, Maskey A, Cherniack M, Zdonik S. Revision processing in a stream processing engine: A high-level design. In Proc. the 22nd International Conference on Data Engineering (ICDE 2006), Atlanta, USA, Apr. 3–8, 2006, pp.141–153.

  19. Aguilera M K, Strom R E, Sturman D C et al. Matching events in a content-based subscription system. In Proc. PODC, Atlanta, USA, Apr. 29-May 6, 1999, pp.53–61.

  20. Seshadri P, Livny M, Ramakrishnan R. Sequence query processing. ACM SIGMOD Record, 1994, 23(2): 430–441.

    Article  Google Scholar 

  21. Zhuge Y, Garcia-Molina H, Hammer J, Widom J. View maintenance in a warehousing environment. In Proc. the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, USA, May 22–25, 1995, pp.316–327.

  22. Crochemore M, Hancart C. Automata for matching patterns. Handbook of Formal Languages, Rozenberg G, Solomaa S (eds.), Vol. Linear Modeling: Background and Application, Heidelberg: Springer-Verlag, 1997, pp.399–462.

  23. Cormen T H, Leiserson C E, Rivest R L, Stein C. Perfect hashing. Introduction to Algorithms, Vol. Hash Tables, MIT Press and McGraw-Hill, 2001, pp.245–253.

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Northeastern University, Shenyang, 110004, China

    Chuan-Wen Li (Student Member, CCF, Member, ACM), Yu Gu (Member, CCF, ACM) & Ge Yu (Senior Member, CCF, Member, ACM, IEEE)

  2. Department of Computer Science and Engineering, Pusan National University, Busan, Korea

    Bonghee Hong (Member, ACM, IEEE)

Authors
  1. Chuan-Wen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ge Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bonghee Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ge Yu.

Additional information

This research was supported by the National Natural Science Foundation of China under Grant Nos. 61003058, 60933001 and the Fundamental Research Funds for the Central Universities under Grant No. N090104001.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 80.6 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, CW., Gu, Y., Yu, G. et al. Aggressive Complex Event Processing with Confidence over Out-of-Order Streams. J. Comput. Sci. Technol. 26, 685–696 (2011). https://doi.org/10.1007/s11390-011-1168-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-011-1168-x

Keywords

Search

Navigation