Abstract
Traditional database management systems are best equipped to run one-time queries over finite stored data sets. However, many modern applications such as network monitoring, financial analysis, manufacturing, and sensor networks require long-running, or continuous, queries over continuous unbounded streams of data. In the STREAM project at Stanford, we are investigating data management and query processing for this class of applications. As part of the project we are building a general-purpose prototype Data Stream Management System (DSMS), also called STREAM, that supports a large class of declarative continuous queries over continuous streams and traditional stored data sets. The STREAM prototype targets environments where streams may be rapid, stream characteristics and query loads may vary over time, and system resources may be limited.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M.K. Aguilera, R.E. Strom, D.C. Sturman, M. Astley, T.D. Chandra, Matching events in a content-based subscription system, in Proc. of the 18th Annual ACM Symp. on Principles of Distributed Computing (1999), pp. 53–61
A. Arasu, B. Babcock, S. Babu, J. McAlister, J. Widom, Characterizing memory requirements for queries over continuous data streams. ACM Trans. Database Syst. 29(1), 1–33 (2004)
A. Arasu, S. Babu, J. Widom, The CQL continuous query language: semantic foundations and query execution. VLDB J. 15(2), 121–142 (2006)
B. Babcock, S. Babu, M. Datar, R. Motwani, Chain: operator scheduling for memory minimization in data stream systems, in Proc. of the 2003 ACM SIGMOD Intl. Conf. on Management of Data (2003), pp. 253–264
B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in Proc. of the 21st ACM SIGACT–SIGMOD–SIGART Symp. on Principles of Database Systems (2002), pp. 1–16
B. Babcock, M. Datar, R. Motwani, Load shedding for aggregation queries over data streams, in Proc. of the 20th Intl. Conf. on Data Engineering (2004)
S. Babu, R. Motwani, K. Munagala, I. Nishizawa, J. Widom, Adaptive ordering of pipelined stream filters, in Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data (2004)
S. Babu, K. Munagala, J. Widom, R. Motwani, Adaptive caching for continuous queries, in Proc. of the 21st Intl. Conf. on Data Engineering (2005), pp. 118–129
S. Babu, U. Srivastava, J. Widom, Exploiting \(k\)-constraints to reduce memory overhead in continuous queries over data streams. ACM Trans. Database Syst. 29(3), 545–580 (2004)
S. Babu, J. Widom, StreaMon: an adaptive engine for stream query processing, in Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data (2004). Demonstration description
B.H. Bloom, Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)
K. Chakrabarti, M.N. Garofalakis, R. Rastogi, K. Shim, Approximate query processing using wavelets, in Proc. of the 26th Intl. Conf. on Very Large Data Bases (2000), pp. 111–122
J. Gehrke (ed.), Data stream processing. IEEE Comput. Soc. Bull. Technical Comm. Database Eng. 26(1) (2003)
F. Fabret, H.-.A. Jacobsen, F. Llirbat, J. Pereira, K.A. Ross, D. Shasha, Filtering algorithms and implementation for very fast publish/subscribe, in Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data (2001), pp. 115–126
R.E. Gruber, B. Krishnamurthy, E. Panagos, READY: a high performance event notification system, in Proc. of the 16th Intl. Conf. on Data Engineering (2000), pp. 668–669
W. Hoeffding, Probability inequalities for sums of bounded random variables. J. Am. Stat. Soc. 58(301), 13–30 (1963)
U. Srivastava, J. Widom, Flexible time management in data stream systems, in Proc. of the 23rd ACM SIGACT-SIGMOD-SIGART Symp. on Principles of Database Systems (2004)
N. Tatbul, U. Cetintemel, S.B. Zdonik, M. Cherniak, M. Stonebraker, Load shedding in a data stream manager, in Proc. of the 29th Intl. Conf. on Very Large Data Bases (2003), pp. 309–320
N. Thaper, S. Guha, P. Indyk, N. Koudas, Dynamic multidimensional histograms, in Proc. of the 2002 ACM SIGMOD Intl. Conf. on Management of Data (2002), pp. 428–439
D. Thomas, R. Motwani, Caching queues in memory buffers, in Proc. of the 15th Annual ACM–SIAM Symp. on Discrete Algorithms (2004)
P.A. Tucker, D. Maier, T. Sheard, L. Fegaras, Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng. 15(3), 555–568 (2003)
S. Viglas, J.F. Naughton, J. Burger, Maximizing the output rate of multi-way join queries over streaming information sources, in Proc. of the 29th Intl. Conf. on Very Large Data Bases (2003), pp. 285–296
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Arasu, A. et al. (2016). STREAM: The Stanford Data Stream Management System. In: Garofalakis, M., Gehrke, J., Rastogi, R. (eds) Data Stream Management. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28608-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-28608-0_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28607-3
Online ISBN: 978-3-540-28608-0
eBook Packages: Computer ScienceComputer Science (R0)