Skip to main content
SpringerLink
Log in

XOR-Satisfiability Set Membership Filters

  • Conference paper
  • First Online:
Theory and Applications of Satisfiability Testing – SAT 2018 (SAT 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10929))

Abstract

Set membership filters are used as a primary test for whether large sets contain given elements. The most common such filter is the Bloom filter [6]. Most pertinent to this article is the recently introduced Satisfiability (SAT) filter [31]. This article proposes the XOR-Satisfiability filter, a variant of the SAT filter based on random k-XORSAT. Experimental results show that this new filter can be more than \(99\%\) efficient (i.e., achieve the information-theoretic limit) while also having a query speed comparable to the standard Bloom filter, making it practical for use with very large data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The intuition for this idea came from Bryan Jacobs’ work on isomorphic k-SAT filters and work by Heule and van Maaren on parallelizing SAT solvers using bitwise operators [19].

  2. 2.

    As long as s is not greater than the native register size of the machine on which the solver is running.

  3. 3.

    Adding an extra r bits of metadata means that the filter now has r more solutions.

  4. 4.

    A constrained model is one where every variable appears in at least two equations.

References

  1. Achlioptas, D.: Random satisfiability. In: Biere et al. [5], pp. 245–270

    Google Scholar 

  2. Albrecht, M., Bard, G.: The M4RI library (2018). https://m4ri.sagemath.org/

  3. Azinović, M., Herr, D., Heim, B., Brown, E., Troyer, M.: Assessment of quantum annealing for the construction of satisfiability filters. SciPost Phys. 2, 013 (2017). https://doi.org/10.21468/SciPostPhys.2.2.013

    Article  Google Scholar 

  4. Bard, G.V.: The method of four Russians. In: Algebraic Cryptanalysis. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-88757-9_9

    Chapter  Google Scholar 

  5. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)

    MATH  Google Scholar 

  6. Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)

    Article  Google Scholar 

  7. Brodnik, A., Munro, J.I.: Membership in constant time and almost-minimum space. SIAM J. Comput. 28(5), 1627–1640 (1999)

    Article  MathSciNet  Google Scholar 

  8. Chazelle, B., Kilian, J., Rubinfeld, R., Tal, A.: The Bloomier filter: an efficient data structure for static support lookup tables. In: Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 30–39. Society for Industrial and Applied Mathematics (2004)

    Google Scholar 

  9. Cohen, S., Matias, Y.: Spectral Bloom filters. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp. 241–252. ACM (2003)

    Google Scholar 

  10. Collet, Y.: xxHash: extremely fast hash algorithm (2017)

    Google Scholar 

  11. Daudé, H., Ravelomanana, V.: Random 2-XORSAT at the satisfiability threshold. In: Laber, E.S., Bornstein, C., Nogueira, L.T., Faria, L. (eds.) LATIN 2008. LNCS, vol. 4957, pp. 12–23. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78773-0_2

    Chapter  Google Scholar 

  12. Dietzfelbinger, M., Goerdt, A., Mitzenmacher, M., Montanari, A., Pagh, R., Rink, M.: Tight thresholds for cuckoo hashing via XORSAT. In: Abramsky, S., Gavoille, C., Kirchner, C., Meyer auf der Heide, F., Spirakis, P.G. (eds.) ICALP 2010. LNCS, vol. 6198, pp. 213–225. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14165-2_19

    Chapter  Google Scholar 

  13. Dietzfelbinger, M., Pagh, R.: Succinct data structures for retrieval and approximate membership (Extended Abstract). In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008. LNCS, vol. 5125, pp. 385–396. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70575-8_32

    Chapter  MATH  Google Scholar 

  14. Douglass, A., King, A.D., Raymond, J.: Constructing SAT filters with a quantum annealer. In: Heule, M., Weaver, S. (eds.) SAT 2015. LNCS, vol. 9340, pp. 104–120. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24318-4_9

    Chapter  MATH  Google Scholar 

  15. Erdős, P., Renyi, A.: On a classical problem of probability theory (1961)

    Google Scholar 

  16. Fan, B., Andersen, D.G., Kaminsky, M., Mitzenmacher, M.D.: Cuckoo filter: practically better than Bloom. In: Proceedings of the 10th ACM International Conference on emerging Networking Experiments and Technologies, pp. 75–88. ACM (2014)

    Google Scholar 

  17. Fang, C., Zhu, Z., Katzgraber, H.G.: NAE-SAT-based probabilistic membership filters. preprint arXiv:1801.06232 (2018)

  18. Goh, E.J., et al.: Secure indexes. IACR Cryptology ePrint Archive 2004, 216 (2004)

    Google Scholar 

  19. Heule, M.J., van Maaren, H.: Parallel SAT solving using bit-level operations. J. Satisf. Boolean Model. Comput. 4, 99–116 (2008)

    MATH  Google Scholar 

  20. Ibrahimi, M., Kanoria, Y., Kraning, M., Montanari, A.: The set of solutions of random XORSAT formulae. In: Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 760–779. SIAM (2012)

    Chapter  Google Scholar 

  21. Kader, A.A., Dorojevets, M.: Novel integration of Dimetheus and WalkSAT solvers for k-SAT filter construction. In: Systems, Applications and Technology Conference (LISAT 2017), pp. 1–5. IEEE (2017)

    Google Scholar 

  22. Krimer, E., Erez, M.: The power of \(1 + \alpha \) for memory-efficient Bloom filters. Internet Math. 7(1), 28–44 (2011)

    Article  MathSciNet  Google Scholar 

  23. Mitchell, D., Selman, B., Levesque, H.: Hard and easy distributions of SAT problems. In: AAAI 1992, pp. 459–465 (1992)

    Google Scholar 

  24. Mitzenmacher, M.D.: Compressed Bloom filters. IEEE/ACM Trans. Netw. (TON) 10(5), 604–612 (2002)

    Article  Google Scholar 

  25. Pittel, B., Sorkin, G.B.: The satisfiability threshold for k-XORSAT. Comb. Probab. Comput. 25(02), 236–268 (2016)

    Article  MathSciNet  Google Scholar 

  26. Porat, E.: An optimal Bloom filter replacement based on matrix solving. In: Frid, A., Morozov, A., Rybalchenko, A., Wagner, K.W. (eds.) CSR 2009. LNCS, vol. 5675, pp. 263–273. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03351-3_25

    Chapter  Google Scholar 

  27. Pouliot, D., Wright, C.V.: The shadow nemesis: inference attacks on efficiently deployable, efficiently searchable encryption. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1341–1352. ACM (2016)

    Google Scholar 

  28. Putze, F., Sanders, P., Singler, J.: Cache-, hash-, and space-efficient Bloom filters. J. Exp. Algorithmics 14, 4 (2009)

    Article  MathSciNet  Google Scholar 

  29. Arlazarov, V.L., Dinitz, Y.A., Kronrod, M.A., Faradzev, I.A.: On economical construction of transitive closure of an oriented graph. Dokl. Akad. Nauk SSSR 194(3), 487 (1970)

    MathSciNet  Google Scholar 

  30. Walker, A.: Filters. Master’s thesis, Haverford College (2007). http://math.uchicago.edu/~akwalker/filtersFinal.pdf

  31. Weaver, S.A., Ray, K.J., Marek, V.W., Mayer, A.J., Walker, A.K.: Satisfiability-based set membership filters. J. Satisf. Boolean Model. Comput. 8, 129–148 (2014)

    MathSciNet  MATH  Google Scholar 

  32. Zhang, Y., Katz, J., Papamanthou, C.: All your queries are belong to us: the power of file-injection attacks on searchable encryption. In: USENIX Security Symposium, pp. 707–720 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Assurance Research Group, U.S. National Security Agency, 9800 Savage Rd., Suite 6845, Ft. George G. Meade, MD, 20755, USA

    Sean A. Weaver, Hannah J. Roberts & Michael J. Smith

Authors
  1. Sean A. Weaver
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hannah J. Roberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael J. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sean A. Weaver .

Editor information

Editors and Affiliations

  1. Friedrich Schiller University Jena, Jena, Germany

    Olaf Beyersdorff

  2. Microsoft Research Ltd., Cambridge, United Kingdom

    Christoph M. Wintersteiger

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Weaver, S.A., Roberts, H.J., Smith, M.J. (2018). XOR-Satisfiability Set Membership Filters. In: Beyersdorff, O., Wintersteiger, C. (eds) Theory and Applications of Satisfiability Testing – SAT 2018. SAT 2018. Lecture Notes in Computer Science(), vol 10929. Springer, Cham. https://doi.org/10.1007/978-3-319-94144-8_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94144-8_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94143-1

  • Online ISBN: 978-3-319-94144-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation