Skip to main content
SpringerLink
Log in
Book cover

Data Stream Management pp 87–102Cite as

Join Sizes, Frequency Moments, and Applications

  • Chapter
  • First Online:

Part of the book series: Data-Centric Systems and Applications ((DCSA))

Abstract

We focus on a set of problems chiefly inspired by the problem of estimating the size of the (equi-)join between two relational data streams. This problem is at the heart of a wide variety of other problems, both in databases/data streams and beyond, including approximating range-query aggregates, quantiles, and heavy-hitter elements, and building approximate histograms and wavelet representations. Our discussion focuses on efficient, sketch-based streaming algorithms for join-size and self-join-size estimation problems, based on the influential papers of Alon, Matias, Gibbons, and Szegedy.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. Achlioptas, Database-friendly random projections, in Proceedings of ACM Principles of Database Systems (2001), pp. 274–281

    Google Scholar 

  2. N. Alon, P. Gibbons, Y. Matias, M. Szegedy, Tracking join and self-join sizes in limited storage, in Proceedings of the Eighteenth ACM Symposium on Principles of Database Systems (1999), pp. 10–20

    Google Scholar 

  3. N. Alon, Y. Matias, M. Szegedy, The space complexity of approximating the frequency moments, in Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing. (1996), pp. 20–29. Journal version in: J. Comput. Syst. Sci. 58, 137–147 (1999)

    Google Scholar 

  4. Z. Bar-Yossef, R. Kumar, D. Sivakumar, Reductions in streaming algorithms, with an application to counting triangles in graphs, in Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms (2002), pp. 623–632

    Google Scholar 

  5. J.L. Carter, M.N. Wegman, Universal classes of hash functions. J. Comput. Syst. Sci. 18(2), 143–154 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  6. M. Charikar, K. Chen, M. Farach-Colton, Finding frequent items in data streams, in Proceedings of the International Colloquium on Automata, Languages and Programming (ICALP) (2002), pp. 693–703

    Chapter  Google Scholar 

  7. G. Cormode, S. Muthukrishnan, An improved data stream summary: the count-min sketch and its applications, in Latin American Informatics (2004), pp. 29–38

    Google Scholar 

  8. G. Cormode, M. Garofalakis, Approximate continuous querying over distributed streams. ACM Trans. Database Syst. 33(2) (2008)

    Google Scholar 

  9. G. Cormode, M. Garofalakis, P.J. Haas, C. Jermaine, Synopses for massive data: samples, histograms, wavelets, sketches. Found. Trends® Databases 4(1–3) (2012)

    Google Scholar 

  10. A. Das, J. Gehrke, M. Riedewald, Approximation techniques for spatial data, in Proc. of the 2004 ACM SIGMOD Intl. Conference on Management of Data (2004)

    Google Scholar 

  11. J. Feigenbaum, S. Kannan, M. Strauss, M. Viswanathan, An approximate \({L}_{1}\)-difference algorithm for massive data streams, in Proceedings of the 40th Annual Symposium on Foundations of Computer Science (1999), pp. 501–511

    Google Scholar 

  12. M. Garofalakis, P.B. Gibbons, Approximate query processing: taming the terabytes, in 27th Intl. Conf. on Very Large Data Bases, Rome, Italy (2001). Tutorial

    Google Scholar 

  13. A.C. Gilbert, Y. Kotidis, S. Muthukrishnan, M. Strauss, How to summarize the universe: dynamic maintenance of quantiles, in Proceedings of the International Conference on Very Large Data Bases (2002), pp. 454–465

    Chapter  Google Scholar 

  14. I.J. Good, Surprise indexes and \(p\)-values. J. Stat. Comput. Simul. 32, 90–92 (1989)

    Article  Google Scholar 

  15. Y.E. Ioannidis, S. Christodoulakis, Optimal histograms for limiting worst-case error propagation in the size of join results. ACM Trans. Database Syst. 18(4), 709–748 (1993)

    Article  Google Scholar 

  16. W.B. Johnson, J. Lindenstrauss, Extensions of Lipschitz mapping into Hilbert space. Contemp. Math. 26, 189–206 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  17. B. Krishnamurthy, S. Sen, Y. Zhang, Y. Chen, Sketch-based change detection: methods, evaluation and applications, in Proceedings of the ACM SIGCOMM Conference on Internet Measurement (2003), pp. 234–247

    Google Scholar 

  18. E. Kushilevitz, N. Nisan, Communication Complexity (Cambridge University Press, Cambridge, 1997)

    Book  MATH  Google Scholar 

  19. R. Motwani, P. Raghavan, Randomized Algorithms (Cambridge University Press, Cambridge, 1995)

    Book  MATH  Google Scholar 

  20. S. Muthukrishnan, Data streams: algorithms and applications, in Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (2003)

    Google Scholar 

  21. M. Thorup, Even strongly universal hashing is pretty fast, in Proceedings of the 11th Annual ACM-SIAM Symposium on Discrete Algorithms (2000), pp. 496–497

    Google Scholar 

  22. M. Thorup, Y. Zhang, Tabulation based 4-universal hashing with applications to second moment estimation, in Proceedings of the 15th Annual ACM–SIAM Symposium on Discrete Algorithms (2004), pp. 615–624

    Google Scholar 

  23. M.N. Wegman, J.L. Carter, New hash functions and their use in authentication and set equality. J. Comput. Syst. Sci. 22(3), 265–279 (1981)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK

    Graham Cormode

  2. School of Electrical and Computer Engineering, Technical University of Crete, University Campus—Kounoupidiana, Chania, 73100, Greece

    Minos Garofalakis

Authors
  1. Graham Cormode
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minos Garofalakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Graham Cormode .

Editor information

Editors and Affiliations

  1. University Campus - Kounoupidiana, School of ECE, Techn. Univ. of Crete University Campus - Kounoupidiana, Chania, Greece

    Minos Garofalakis

  2. Microsoft Corporation, Redmond, Washington, USA

    Johannes Gehrke

  3. Amazon India , Bangalore, India

    Rajeev Rastogi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Cormode, G., Garofalakis, M. (2016). Join Sizes, Frequency Moments, and Applications. In: Garofalakis, M., Gehrke, J., Rastogi, R. (eds) Data Stream Management. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28608-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-28608-0_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-28607-3

  • Online ISBN: 978-3-540-28608-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation