Definition
Data stream processing architectures perform database-style query processing on a set of continuously arriving input streams. The core query executor in this type of architecture is designed to process continuous queries, rather than ad hoc queries, by pushing inputs through a series of operators functioning in a pipelined and potentially non-blocking manner. Stream processing applications perform explicit read and write operations to access storage via asynchronous disk I/O operations. Other architectural components that differ significantly from standard database designs include the stream processor’s scheduler, storage manager, and queue manager.
Historical Background
Support database-style query processing for long-running applications that operate in high (data) volume environments and impose high throughput and low latency requirements on the system. There have been several efforts from both the academic and industrial communities at developing functional prototypes of...
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Recommended Reading
Abadi DJ, Carney D, Çetintemel U, Cherniack M, Convey C, Lee S, Stonebraker M, Tatbul N, Zdonik S. Aurora: a new model and architecture for data stream management. VLDB J. 2003; 12(2):120–39.
Babcock B, Babu S, Datar M, Motwani R, Widom J. Models and issues in data stream systems. In: Proceedings of the 21st ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems; 2002.
Chandrasekaran S, Deshpande A, Franklin M, Hellerstein J. TelegraphCQ: continuous dataflow processing for an uncertain world. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research; 2003.
Arasu A, Babcock B, Babu S, McAlister J, Widom J. Characterizing memory requirements for queries over continuous data streams. ACM Trans Database Syst. 2004;29(1):162–94.
Carney D, Çetintemel U, Rasin A, Zdonik SB, Cherniack M, Stonebraker M. Operator scheduling in a data stream manager. In: Proceedings of the 29th International Conference on Very Large Data Bases; 2003. p. 838–49.
Tatbul N, Çetintemel U, Zdonik SB, Cherniack M, Stonebraker M. Load shedding in a data stream manager. In: Proceedings of the 29th International Conference on Very Large Data Bases; 2003. p. 309–20.
Babu S, Motwani R, Munagala K, Nishizawa I, Widom J Adaptive ordering of pipelined stream filters. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2004. p. 407–18.
Abadi D, Ahmad Y, Balazinska M, Çetintemel U, Cherniack M, Hwang J.-H, Lindner W, Maskey AS, Rasin A, Ryvkina E, Tatbul N, Xing Y, Zdonik S. The design of the Borealis stream processing engine. In: Proceedings of the 2nd Biennial Conference on Innovative Data Systems Research; 2005.
Cranor CD, Johnson T, Spatscheck O, Shkapenyuk V. Gigascope: a stream database for network applications. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2003. p. 647–51.
Hammad MA, Mokbel MF, Ali MH, Aref WG, Catlin AC, Elmagarmid AK, Eltabakh MY, Elfeky MG, Ghanem TM, Gwadera R, Ilyas IF, Marzouk MS, Xiong X. Nile: a query processing engine for data streams. In: Proceedings of the 20th International Conference on Data Engineering; 2004. p. 851.
Gedik B, Andrade H, Wu K-L, Yu PS, Doo M. SPADE: the systems S declarative stream processing engine. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2008.
Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. Matei Zaharia, Tathagata Das, Haoyuan Li, Scott Shenker, Ion Stoica. HotCloud 2012. June 2012.
Das T, Dave A, Ma J, McCauley M, Franklin MJ, Shenker S, Stoica I. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. Matei Zaharia, Mosharaf Chowdhury. NSDI 2012. April 2012.
Cetintemel U, Du J, Kraska T, Madden S, Maier D, Meehan J, Pavlo A, Stonebraker M, Sutherland E, Tatbul N, Tufte K, Wang H, Zdonik S. S-Store: a streaming NewSQL system for big velocity applications (demonstration). In: Proceedings of the 40th International Conference on Very Large Data Bases; 2014.
Kallman R, Kimura H, Natkins J, Pavlo A, Rasin A, Zdonik S, Jones EPC, Madden S, Stonebraker M, Zhang Y, Hugg J, Abadi DJ. H-Store: a high-performance, distributed main memory transaction processing system. Proc VLDB Endow. 2008;1(2):1496–9.
Babcock B, Babu S, Datar M, Motwani R, Thomas D. Operator scheduling in data stream systems. VLDB J. 2004;13(4):333–53.
Chen J, DeWitt DJ, Tian F, Wang Y. NiagaraCQ: a scalable continuous query system for internet databases. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2000. p. 379–90.
Gehrke J. Data stream processing. IEEE Data Eng Bull. 2003;26(1).
Golab L, Özsu MT. Issues in data stream management. ACM SIGMOD Rec. 2003;32(2):5–14.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer Science+Business Media, LLC, part of Springer Nature
About this entry
Cite this entry
Ahmad, Y., Cetintemel, U. (2018). Data Stream Management Architectures and Prototypes. In: Liu, L., Özsu, M.T. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8265-9_24
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8265-9_24
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8266-6
Online ISBN: 978-1-4614-8265-9
eBook Packages: Computer ScienceReference Module Computer Science and Engineering