Comparing Password Ranking Algorithms on Real-World Password Datasets

  • Weining Yang
  • Ninghui LiEmail author
  • Ian M. Molloy
  • Youngja Park
  • Suresh N. Chari
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9878)


Password-based authentication is the most widely used authentication mechanism. One major weakness of password-based authentication is that users generally choose predictable and weak passwords. In this paper, we address the question: How to best check weak passwords? We model different password strength checking methods as Password Ranking Algorithms (PRAs), and introduce two methods for comparing different PRAs: the \(\beta \)-Residual Strength Graph (\(\beta \)-RSG) and the Normalized \(\beta \)-Residual Strength Graph (\(\beta \)-NRSG). In our experiments, we find some password datasets that have been widely used in password research contain many problematic passwords that are not naturally created. We develop techniques to cleanse password datasets by removing these problematic accounts. We then apply the two metrics on cleansed datasets and show that several PRAs, including the dictionary-based PRA, the Markov Models with and without backoff, have similar performances. If the size of PRAs are limited in order to be able to be transmitted over the internet, a hybrid method combining a small dictionary of weak passwords and a Markov model with backoff with a limited size can provide the most accurate strength measurement.


Markov Model Training Dataset Testing Dataset Residual Strength Markov Chain Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This paper is based upon work supported in part by an IBM OCR grant from IBM Research.

Supplementary material


  1. 1.
  2. 2.
    CSDN cleartext passwords (2011).
  3. 3.
    John the ripper password cracker (2014).
  4. 4.
    Adams, A., Sasse, M.A.: Users are not the enemy. Commun. ACM 42(12), 40–46 (1999)CrossRefGoogle Scholar
  5. 5.
    Bergadano, F., Crispo, B., Ruffo, G.: Proactive password checking with decision trees. In: Proceedings of the 4th ACM Conference on Computer and Communications Security, pp. 67–77 (1997)Google Scholar
  6. 6.
    Bonneau, J.: The science of guessing: analyzing an anonymized corpus of 70 million passwords. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 538–552 (2012)Google Scholar
  7. 7.
    Bonneau, J., Herley, C., van Oorschot, P.C., Stajano, F.: Passwords and the evolution of imperfect authentication. Commun. ACM 58(7), 78–87 (2015)CrossRefGoogle Scholar
  8. 8.
    Boztas, S.: Entropies, guessing, and cryptography. Technical report 6, Department of Mathematics, Royal Melbourne Institute of Technology (1999)Google Scholar
  9. 9.
    Brostoff, S., Sasse, M.A.: “Ten strikes and you’re out”: increasing the number of login attempts can improve password usability. In: Proceedings of the Human-computer Interaction Security Workshop (2003)Google Scholar
  10. 10.
    Burnett, M.: Today I am releasing ten million passwords (2015).
  11. 11.
    Burr, W.E., Dodson, D.F., Polk, W.T.: Electronic authentication guideline. US Department of Commerce, Technology Administration, National Institute of Standards and Technology (2004)Google Scholar
  12. 12.
    Castelluccia, C., Chaabane, A., Dürmuth, M., Perito, D.: When privacy meets security: Leveraging personal information for password cracking. arXiv preprint arXiv:1304.6584 (2013)
  13. 13.
    Castelluccia, C., Dürmuth, M., Perito, D.: Adaptive password-strength meters from Markov models. In: Proceedings of the Network and Distributed System Security Symposium (2012)Google Scholar
  14. 14.
    de Carné de Carnavalet, X., Mannan, M.: From very weak to very strong: analyzing password-strength meters. In: Proceedings of the Network and Distributed System Security Symposium (2014)Google Scholar
  15. 15.
    Dell’Amico, M., Filippone, M.: Monte carlo strength evaluation: fast and reliable password checking. In: Proceedings of the 22nd ACM Conference on Computer and Communications Security, pp. 158–169 (2015)Google Scholar
  16. 16.
    Egelman, S., Sotirakopoulos, A., Muslukhov, I., Beznosov, K., Herley, C.: Does my password go up to eleven?: the impact of password meters on password selection. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2379–2388 (2013)Google Scholar
  17. 17.
    Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd ACM Conference on Knowledge Discovery and Data Mining, vol. 96, pp. 226–231 (1996)Google Scholar
  18. 18.
    Florêncio, D., Herley, C.: A large-scale study of web password habits. In: Proceedings of the 16th International Conference on World Wide Web, pp. 657–666 (2007)Google Scholar
  19. 19.
    Forget, A., Chiasson, S., van Oorschot, P.C., Biddle, R.: Improving text passwords through persuasion. In: Proceedings of the 4th Symposium on Usable Privacy and Security, pp. 1–12 (2008)Google Scholar
  20. 20.
    Grampp, F.T., Morris, R.H.: The unix system: unix operating system security. AT&T Bell Laboratories Tech. J. 63(8), 1649–1672 (1984)CrossRefGoogle Scholar
  21. 21.
    Herley, C., van Oorschot, P.C.: A research agenda acknowledging the persistence of passwords. IEEE Secur. Priv. 10(1), 28–36 (2012)CrossRefGoogle Scholar
  22. 22.
    Kelley, P.G., Komanduri, S., Mazurek, M.L., Shay, R., Vidas, T., Bauer, L., Christin, N., Cranor, L.F., Lopez, J.: Guess again (and again and again): measuring password strength by simulating password-cracking algorithms. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 523–537(2012)Google Scholar
  23. 23.
    Klein, D.V.: Foiling the cracker: a survey of, and improvements to, password security. In: Proceedings of the 2nd USENIX Security Workshop, pp. 5–14 (1990)Google Scholar
  24. 24.
    Komanduri, S., Shay, R., Cranor, L.F., Herley, C., Schechter, S.: Telepathwords: preventing weak passwords by reading users’ minds. In: Proceedings of the 23rd USENIX Security Symposium, pp. 591–606 (2014)Google Scholar
  25. 25.
    Komanduri, S., Shay, R., Kelley, P.G., Mazurek, M.L., Bauer, L., Christin, N., Cranor, L.F., Egelman, S.: Of passwords and people: measuring the effect of password-composition policies. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2595–2604 (2011)Google Scholar
  26. 26.
    Ma, J., Yang, W., Luo, M., Li, N.: A study of probabilistic password models. In: IEEE Symposium on Security and Privacy (SP), pp. 689–704. IEEE (2014)Google Scholar
  27. 27.
    Manber, U., Wu, S.: An algorithm for approximate membership checking with application to password security. Inf. Process. Lett. 50(4), 191–197 (1994)CrossRefzbMATHGoogle Scholar
  28. 28.
    Morris, R., Thompson, K.: Password security: a case history. Commun. ACM 22(11), 594–597 (1979)CrossRefGoogle Scholar
  29. 29.
    Narayanan, A., Shmatikov, V.: Fast dictionary attacks on passwords using time-space tradeoff. In Proceedings of the 12th ACM Conference on Computer and Communications Security, pp. 364–372 (2005)Google Scholar
  30. 30.
    Riley, S.: Password security: What users know and what they actually do. In: Chaparro, B.S. (ed.) Usability News, vol. 8 of 1, Software Usability Research Laboratory (SURL) at Wichita State University (2006)Google Scholar
  31. 31.
    Schechter, S., Herley, C., Mitzenmacher, M.: Popularity is everything: a new approach to protecting passwords from statistical-guessing attacks. In: Proceedings of the 5th USENIX Conference on Hot Topics in Security, pp. 1–8 (2010)Google Scholar
  32. 32.
    Spafford, E.H.: OPUS: preventing weak password choices. Comput. Secur. 11(3), 273–278 (1992)CrossRefGoogle Scholar
  33. 33.
    Ur, B., Kelley, P.G., Komanduri, S., Lee, J., Maass, M., Mazurek, M., Passaro, T., Shay, R., Vidas, T., Bauer, L., et al.: How does your password measure up? The effect of strength meters on password creation. In: Proceedings of the 21st USENIX Security Symposium, pp. 65–80 (2012)Google Scholar
  34. 34.
    Ur, B., Segreti, S.M., Bauer, L., Christin, N., Cranor, L.F., Komanduri, S., Kurilova, D., Mazurek, M.L., Melicher, W., Shay, R.: Measuring real-world accuracies and biases in modeling password guessability. In: Proceeding of the 24th USENIX Security Symposium, pp. 463–481 (2015)Google Scholar
  35. 35.
    Veras, R., Collins, C., Thorpe, J.: On the semantic patterns of passwords and their security impact. In: Proceedings of the Network and Distributed System Security Symposium (2014)Google Scholar
  36. 36.
    Weir, M., Aggarwal, S., Collins, M., Stern, H.: Testing metrics for password creation policies by attacking large sets of revealed passwords. In: Proceedings of the 17th ACM Conference on Computer and Communications Security, pp. 162–175 (2010)Google Scholar
  37. 37.
    Weir, M., Aggarwal, S., de Medeiros, B., Glodek, B.: Password cracking using probabilistic context-free grammars. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 391–405 (2009)Google Scholar
  38. 38.
    Wheeler, D.: zxcvbn: realistic password strength estimation. Dropbox blog article (2012)Google Scholar
  39. 39.
    Yan, J.J.: A note on proactive password checking. In: Proceedings of the 2001 Workshop on New Security Paradigms, pp. 127–135 (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Weining Yang
    • 1
  • Ninghui Li
    • 1
    Email author
  • Ian M. Molloy
    • 2
  • Youngja Park
    • 2
  • Suresh N. Chari
    • 2
  1. 1.Purdue UniversityWest LafayetteUSA
  2. 2.IBM T. J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations