# Dual Hashing-Based Algorithms for Discrete Integration

• Alexis de Colnet
• Kuldeep S. Meel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11802)

## Abstract

Given a boolean formula F and a weight function $$\rho$$, the problem of discrete integration seeks to compute the weight of F, defined as the sum of the weights of satisfying assignments. Discrete integration, also known as weighted model counting, is a fundamental problem in computer science with wide variety of applications ranging from machine learning and statistics to physics and infrastructure reliability. Given the intractability of the exact variant, the problem of approximate weighted model counting has been subject to intense theoretical and practical investigations over the years.

The primary contribution of this paper is to investigate development of algorithmic approaches for discrete integration. Our framework allows us to derive two different algorithms: WISH, which was already discovered by Ermon et al. [8], and a new algorithm: SWITCH. We argue that these algorithms can be seen as dual to each other, in the sense that their complexities differ only by a permutation of certain parameters. Indeed we show that, for F defined over n variables, a weight function $$\rho$$ that can be represented using p bits, and a confidence parameter $$\delta$$, there is a function f and an NP oracle such that WISH makes $$\mathcal {O} \left( f(n,p,\delta )\right)$$ calls to NP oracle while SWITCH makes $$\mathcal {O}\left( f(p,n,\delta )\right)$$ calls. We find $$f(x,y,\delta )$$ polynomial in x, y and $$1/\delta$$, more specifically $$f(x,y,\delta ) = x\log (y)\log (x/\delta )$$. We first focus on striking similarities of both the design process and structure of the two algorithms but then show that despite this quasi-symmetry, the analysis yields time complexities dual to each other. Another contribution of this paper is the use of 3-wise property independence of XOR based hash functions in the analysis of WISH and SWITCH. To the best of our knowledge, this is the first usage of 3-wise independence in deriving stronger concentration bounds and we hope our usage can be generalized to other applications.

## Notes

### Acknowledgements

This research has been supported in part by the National Research Foundation Singapore under its AI Singapore Programme [Award Number: AISG-RP-2018-005] and the NUS ODPRT Grant [R-252-000-685-133].

## References

1. 1.
Bellare, M., Petrank, E.: Making zero-knowledge provers efficient. In: Proceedings of the 24th Annual Symposium on the Theory of Computing. ACM Citeseer (1992)Google Scholar
2. 2.
Brooks, S., Gelman, A., Jones, G., Meng, X.L.: Handbook of Markov Chain Monte Carlo. Chapman & Hall/CRC, Hoboken (2011)
3. 3.
Carter, J.L., Wegman, M.N.: Universal classes of hash functions. J. Comput. Syst. Sci. 18, 143–154 (1977)
4. 4.
Chakraborty, S., Fremont, D.J., Meel, K.S., Seshia, S.A., Vardi, M.Y.: Distribution-aware sampling and weighted model counting for sat. In: Proceedings of AAAI, pp. 1722–1730 (2014)Google Scholar
5. 5.
Chakraborty, S., Meel, K.S., Vardi, M.Y.: A scalable approximate model counter. In: Schulte, C. (ed.) CP 2013. LNCS, vol. 8124, pp. 200–216. Springer, Heidelberg (2013).
6. 6.
Chakraborty, S., Meel, K.S., Vardi, M.Y.: Algorithmic improvements in approximate counting for probabilistic inference: from linear to logarithmic SAT calls. In: Proceedings of IJCAI (2016)Google Scholar
7. 7.
Ermon, S., Gomes, C., Sabharwal, A., Selman, B.: Embed and project: discrete sampling with universal hashing. In: Proceedings of NIPS, pp. 2085–2093 (2013)Google Scholar
8. 8.
Ermon, S., Gomes, C., Sabharwal, A., Selman, B.: Taming the curse of dimensionality: discrete integration by hashing and optimization. In: Proceedings of ICML, pp. 334–342 (2013)Google Scholar
9. 9.
Gogate, V., Dechter, R.: Approximate counting by sampling the backtrack-free search space. In: Proceedings of the AAAI, vol. 22, p. 198 (2007)Google Scholar
10. 10.
Gomes, C., Sabharwal, A., Selman, B.: Near-uniform sampling of combinatorial spaces using XOR constraints. In: Proceedings of NIPS, pp. 481–488 (2006)Google Scholar
11. 11.
Jerrum, M.R., Sinclair, A.: The markov chain monte carlo method: an approach to approximate counting and integration. In: Approximation Algorithms for NP-Hard Problems, pp. 482–520 (1996)Google Scholar
12. 12.
Kitchen, N., Kuehlmann, A.: Stimulus generation for constrained random simulation. In: Proceedings of ICCAD, pp. 258–265 (2007)Google Scholar
13. 13.
Paredes, R., Duenas-Osorio, L., Meel, K.S., Vardi, M.Y.: Network reliability estimation in theory and practice. In: Reliability Engineering and System Safety (2018)Google Scholar
14. 14.
Roth, D.: On the hardness of approximate reasoning. Artif. Intell. 82(1–2), 273–302 (1996)
15. 15.
Soos, M., Meel, K.S.: Bird: Engineering an efficient CNF-XOR sat solver and its applications to approximate model counting. In: Proceedings of AAAI Conference on Artificial Intelligence (AAAI 2019) (2019)
16. 16.
Stockmeyer, L.: The complexity of approximate counting. In: Proceedings of STOC, pp. 118–126 (1983)Google Scholar
17. 17.
Tzikas, D.G., Likas, A.C., Galatsanos, N.P.: The variational approximation for Bayesian inference. IEEE Sig. Process. Mag. 25(6), 131–146 (2008)
18. 18.
Valiant, L.G.: The complexity of computing the permanent. Theoret. Comput. Sci. 8, 189–201 (1977)
19. 19.
Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1–2), 1–305 (2008)