Abstract
We study the problem of generating a large sample from a data stream of elements (i,v), where the sample consists of pairs (i,C i ) for C i = ∑ (i,v) ∈ stream v. We consider strict turnstile streams and general non-strict turnstile streams, in which C i may be negative. Our sample is useful for approximating both forward and inverse distribution statistics, within an additive error ε and provable success probability 1 − δ.
Our sampling method improves by an order of magnitude the known processing time of each stream element, a crucial factor in data stream applications, thereby providing a feasible solution to the problem. For example, for a sample of size O(ε − 2 log(1/δ)) in non-strict streams, our solution requires O((loglog(1/ε))2 + (loglog(1/δ)) 2) operations per stream element, whereas the best previous solution requires O(ε − 2 log2(1/δ)) evaluations of a fully independent hash function per element.
We achieve this improvement by constructing an efficient K-elements recovery structure from which K elements can be extracted with probability 1 − δ. Our structure enables our sampling algorithm to run on distributed systems and extract statistics on the difference between streams.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cohen, E., Cormode, G., Duffield, N.G.: Don’t let the negatives bring you down: sampling from streams of signed updates. In: SIGMETRICS, pp. 343–354 (2012)
Cormode, G., Datar, M., Indyk, P., Muthukrishnan, S.: Comparing data streams using hamming norms (how to zero in). IEEE Trans. Knowl. Data Eng. 15(3), 529–540 (2003)
Cormode, G., Muthukrishnan, S., Rozenbaum, I.: Summarizing and mining inverse distributions on data streams via dynamic inverse sampling. In: VLDB, pp. 25–36 (2005)
Datar, M., Muthukrishnan, S.M.: Estimating rarity and similarity over data stream windows. In: Möhring, R.H., Raman, R. (eds.) ESA 2002. LNCS, vol. 2461, pp. 323–334. Springer, Heidelberg (2002)
Dubhashi, D.P., Ranjan, D.: Balls and bins: A study in negative dependence. Random Struct. Algorithms 13(2), 99–124 (1998)
Frahling, G., Indyk, P., Sohler, C.: Sampling in dynamic data streams and applications. In: Symposium on Computational Geometry, pp. 142–149 (2005)
Frahling, G., Indyk, P., Sohler, C.: Sampling in dynamic data streams and applications. Int. J. Comput. Geometry Appl. 18(1/2), 3–28 (2008)
Ganguly, S.: Counting distinct items over update streams. Theor. Comput. Sci. 378(3), 211–222 (2007)
von zur Gathen, J., Gerhard, J.: Modern computer algebra. Cambridge University Press, New York (1999)
Gemulla, R., Lehner, W., Haas, P.J.: A dip in the reservoir: Maintaining sample synopses of evolving datasets. In: VLDB, pp. 595–606 (2006)
Gemulla, R., Lehner, W., Haas, P.J.: Maintaining bernoulli samples over evolving multisets. In: PODS, pp. 93–102 (2007)
Gibbons, P.B.: Distinct sampling for highly-accurate answers to distinct values queries and event reports. In: VLDB, pp. 541–550 (2001)
Gilbert, A.C., Li, Y., Porat, E., Strauss, M.J.: Approximate sparse recovery: optimizing time and measurements. In: STOC, pp. 475–484 (2010)
Gohberg, I., Olshevsky, V.: Fast algorithms with preprocessing for matrix-vector multiplication problems. J. Complexity 10(4), 411–427 (1994)
Goodrich, M.T., Mitzenmacher, M.: Invertible bloom lookup tables. In: 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 792–799. IEEE (2011)
Indyk, P.: A small approximately min-wise independent family of hash functions. J. Algorithms 38(1), 84–90 (2001)
Jowhari, H., Saglam, M., Tardos, G.: Tight bounds for lp samplers, finding duplicates in streams, and related problems. In: PODS, pp. 49–58 (2011)
Kane, D.M., Nelson, J., Woodruff, D.P.: An optimal algorithm for the distinct elements problem. In: PODS, pp. 41–52 (2010)
Karamcheti, V., Geiger, D., Kedem, Z.M., Muthukrishnan, S.: Detecting malicious network traffic using inverse distributions of packet contents. In: MineNet, pp. 165–170 (2005)
Micciancio, D.: Oblivious data structures: Applications to cryptography. In: STOC, pp. 456–464 (1997)
Monemizadeh, M., Woodruff, D.P.: 1-pass relative-error lp-sampling with applications. In: SODA, pp. 1143–1160 (2010)
Muthukrishnan, S.: Data streams: Algorithms and applications. Foundations and Trends in Theoretical Computer Science 1(2) (2005)
Naor, M., Teague, V.: Anti-persistence: History independent data structures. IACR Cryptology ePrint Archive 2001, 36 (2001)
Pagh, A., Pagh, R.: Uniform hashing in constant time and optimal space. SIAM J. Comput. 38(1), 85–96 (2008)
Porat, E., Lipsky, O.: Improved sketching of hamming distance with error correcting. In: Ma, B., Zhang, K. (eds.) CPM 2007. LNCS, vol. 4580, pp. 173–182. Springer, Heidelberg (2007)
Reed, I., Golomb, S.: Polynomial codes over certain finite fields. Joint Society of Industrial and Applied Mathematics Journal 8(2), 300–304 (1960)
Tao, Y., Lian, X., Papadias, D., Hadjieleftheriou, M.: Random sampling for continuous streams with arbitrary updates. IEEE Trans. Knowl. Data Eng. 19(1), 96–110 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Barkay, N., Porat, E., Shalem, B. (2013). Efficient Sampling of Non-strict Turnstile Data Streams. In: Gąsieniec, L., Wolter, F. (eds) Fundamentals of Computation Theory. FCT 2013. Lecture Notes in Computer Science, vol 8070. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40164-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-40164-0_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40163-3
Online ISBN: 978-3-642-40164-0
eBook Packages: Computer ScienceComputer Science (R0)