Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Adaptive Stream Processing

  • Zachary G. Ives
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_11

Synonyms

Adaptive query processing

Definition

When querying long-lived data streams, the characteristics of the data may change over time or data may arrive in bursts – hence, the traditional model of optimizing a query prior to executing it is insufficient. As a result, most data stream management systems employ feedback-driven adaptive stream processing, which continuously re-optimizes the query execution plan based on data and stream properties, in order to meet certain performance or resource consumption goals. Adaptive stream processing is a special case of the more general problem of adaptive query processing, with the special property that intermediate results are bounded in size (by stream windows), but where query processing may have quality-of-service constraints.

Historical Background

The field of adaptive stream processing emerged in the early 2000s, as two separate developments converged. Adaptivetechniques for database query processing had become an area of increasing...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Abadi DJ, Carney D, Cetintemel U, Cherniack M, Convey C, Lee S, Stonebraker M, Tatbul N, Zdonik S. Aurora: a new model and architecture for data stream management. VLDB J. 2003;12(2):120–39.CrossRefGoogle Scholar
  2. 2.
    Avnur R, Hellerstein JM. Eddies: continuously adaptive query processing. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2000. p. 261–72.Google Scholar
  3. 3.
    Babcock B, Babu S, Datar M, Motwani R. Chain: operator scheduling for memory minimization in data stream systems. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 253–64.Google Scholar
  4. 4.
    Babcock B, Datar M, Motwani R. Load shedding for aggregation queries over data streams. In: Proceedings of the 20th International Conference on Data Engineering; 2004. p. 350.Google Scholar
  5. 5.
    Babu S, Motwani R, Munagala K, Nishizawa I, Widom J. Adaptive ordering of pipelined stream filters. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2004. p. 407–18.Google Scholar
  6. 6.
    Balazinska M, BalaKrishnan H, Stonebraker M. Demonstration: load management and high availability in the Medusa distributed stream processing system. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2004. p. 929–30.Google Scholar
  7. 7.
    Bizarro P, Babu S, De Witt DJ, Widom J. Content-based routing: different plans for different data In: Proceedings of the 31st International Conference on Very Large Data; 2005. p. 757–68.Google Scholar
  8. 8.
    Chandrasekaran S, Cooper O, Deshpande A, Franklin MJ, Hellerstein JM, Hong W, Krishnamurthy S, Madden S, Raman V, Reiss F, Shah MA. TelegraphCQ: continuous dataflow processing for an uncertain world. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research; 2003.Google Scholar
  9. 9.
    Deshpande A. An initial study of overheads of eddies. ACM SIGMOD Rec. 2004;33(1):44–9.CrossRefGoogle Scholar
  10. 10.
    Deshpande A, Ives Z, Raman V. Adaptive query processing. Found. Trends Databases. 2007;1(1):1–140.zbMATHCrossRefGoogle Scholar
  11. 11.
    Madden S, Shah MA, Hellerstein JM, Raman V. Continuously adaptive continuous queries over streams. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2002. p. 49–60.Google Scholar
  12. 12.
    Motwani R, Widom J, Arasu A, Babcock B, Babu S, Datar M, Manku G, Olston C, Rosenstein J, Varma R. Query processing, resource management, and approximation in a data stream management system. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research; 2003.Google Scholar
  13. 13.
    Olston C, Jiang J, Widom J. Adaptive filters for continuous queries over distributed data streams. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 563–74.Google Scholar
  14. 14.
    Raman V, Deshpande A, Hellerstein JM. Using state modules for adaptive query processing. In: Proceedings of the 19th International Conference on Data Engineering; 2003. p. 353–66.Google Scholar
  15. 15.
    Tatbul N, Cetintemel U, Zdonik SB, Cherniack M, Stonebraker M. Load shedding in a data stream manager. In: Proceedings of the 29th International Conference on Very Large Data Bases; 2003. p. 309–20.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Computer and Information Science DepartmentUniversity of PennsylvaniaPhiladelphiaUSA

Section editors and affiliations

  • Ugur Cetintemel
    • 1
  1. 1.Department of Computer ScienceBrown UniversityProvidenceUSA