Bernoulli Ballot Polling: A Manifest Improvement for Risk-Limiting Audits

  • Kellie Ottoboni
  • Matthew Bernhard
  • J. Alex Halderman
  • Ronald L. Rivest
  • Philip B. StarkEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11599)


We present a method and software for ballot-polling risk-limiting audits (RLAs) based on Bernoulli sampling: ballots are included in the sample with probability p, independently. Bernoulli sampling has several advantages: (1) it does not require a ballot manifest; (2) it can be conducted independently at different locations, rather than requiring a central authority to select the sample from the whole population of cast ballots or requiring stratified sampling; (3) it can start in polling places on election night, before margins are known. If the reported margins for the 2016 U.S. Presidential election are correct, a Bernoulli ballot-polling audit with a risk limit of 5% and a sampling rate of \(p_0=1\%\) would have had at least a 99% probability of confirming the outcome in 42 states. (The other states were more likely to have needed to examine additional ballots). Logistical and security advantages that auditing in the polling place affords may outweigh the cost of examining more ballots than some other methods might require.


  1. Aslam, J.A., Popa, R.A., Rivest, R.L.: On auditing elections when precincts have different sizes. In: 2008 USENIX/ACCURATE Electronic Voting Technology Workshop, San Jose, CA, 28–29 July 2008Google Scholar
  2. Blaze, M., Braun, J., Hursti, H., Lorenzo Hall, J., MacAlpine, M., Moss, J.: DEFCON 25 Voting Village Report, September 2017.
  3. Blaze, M., Braun, J., Hursti, H., Jefferson, D., MacAlpine, M., Moss, J.: DEFCON 26 Voting Village Report, September 2018.
  4. Bowen, D.: Top-to-bottom review of voting machines certified for use in California. Technical report, California Secretary of State (2007).
  5. Cochran, W.G.: Sampling Techniques, 3rd edn. Wiley, Hoboken (1977)zbMATHGoogle Scholar
  6. Lindeman, M., Stark, P.B.: A gentle introduction to risk-limiting audits. IEEE Secur. Priv. 10, 42–49 (2012)CrossRefGoogle Scholar
  7. Lindeman, M., Stark, P.B., Yates, V.: BRAVO: ballot-polling risk-limiting audits to verify outcomes. In: 2011 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE 2012). USENIX (2012)Google Scholar
  8. Lindeman, M., McBurnett, N., Ottoboni, K., Stark, P.B.: Next steps for the colorado risk-limiting audit (CORLA) program, March 2018.
  9. Lohr, S.: Sampling: Design and Analysis. Nelson Education, Toronto (2009)zbMATHGoogle Scholar
  10. McDaniel, P., Blaze, M., Vigna, G.: EVEREST: evaluation and validation of election-related equipment, standards and testing. Technical report, Ohio Secretary of State (2007).
  11. Ottoboni, K., Stark, P.B., Lindeman, M., McBurnett, N.: Risk-limiting audits by stratified union-intersection tests of elections (SUITE). In: Krimmer, R., et al. (eds.) E-Vote-ID 2018. LNCS, vol. 11143, pp. 174–188. Springer, Cham (2018). Scholar
  12. Panel on Nonstandard Mixtures of Distributions: Statistical Models and Analysis in Auditing: A Study of Statistical Models and Methods for Analyzing Nonstandard Mixtures of Distributions in Auditing. National Academy Press, Washington, D.C. (1988)Google Scholar
  13. Rivest, R.L.: ClipAudit: a simple risk-limiting post-election audit (2017).
  14. Särndal, C.-E., Swensson, B., Wretman, J.: Model Assisted Survey Sampling. Springer Series in Statistics. Springer, New York (2003)zbMATHGoogle Scholar
  15. Stark, P.B.: Conservative statistical post-election audits. Ann. Appl. Stat. 2(2), 550–581 (2008) MathSciNetCrossRefGoogle Scholar
  16. Stark, P.B., Teague, V.: Verifiable European elections: risk-limiting audits for D’Hondt and its relatives. JETS USENIX J. Election Technol. Syst. 3(1), 18–39 (2014)Google Scholar
  17. Thompson, M.: Theory of Sample Surveys, Monographs on Statistics and Applied Probability, vol. 74. Chapman & Hall, London (1997)Google Scholar
  18. U.S. Senate Select Committee on Intelligence: Russian targeting of election infrastructure during the 2016 election: Summary of initial findings and recommendations, May 2018.,Recs2.pdf
  19. Wald, A.: Sequential tests of statistical hypotheses. Ann. Math. Stat. 16, 117–186 (1945)MathSciNetCrossRefGoogle Scholar

Copyright information

© International Financial Cryptography Association 2020

Authors and Affiliations

  • Kellie Ottoboni
    • 1
  • Matthew Bernhard
    • 2
  • J. Alex Halderman
    • 2
  • Ronald L. Rivest
    • 3
  • Philip B. Stark
    • 1
    Email author
  1. 1.Department of StatisticsUniversity of CaliforniaBerkeleyUSA
  2. 2.Department of Computer Science and EngineeringUniversity of MichiganAnn ArborUSA
  3. 3.CSAIL, Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations