Password Guessing via Neural Language Modeling

  • Hang Li
  • Mengqi Chen
  • Shengbo Yan
  • Chunfu JiaEmail author
  • Zhaohui Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11806)


Passwords are the major part of authentication in current social networks. The state-of-the-art password guessing approaches, such as Markov model and probabilistic context-free grammars (PCFG) model, assign a probability value to each password by a statistic approach without any parameters. These methods require large datasets to accurately estimate probability due to the law of large number. The neural network, approximating target probability distribution through iteratively training its parameters, was used to model passwords by some researches. However, since the network architectures they used are simple and straightforward, there are many ways to improve it.

In this paper, we view password guessing as a language modeling task and introduce a deeper, more robust, and faster-converged model with several useful techniques to model passwords. This model shows great ability in modeling passwords while significantly outperforms state-of-the-art approaches. Inspired by the most advanced sequential model named Transformer, we use it to model passwords with bidirectional masked language model which is powerful but unlikely to provide normalized probability estimation. Then we distill Transformer model’s knowledge into our proposed model to further boost its performance. Comparing with the PCFG, Markov and previous neural network models, our models show remarkable improvement in both one-site tests and cross-site tests. Moreover, our models are robust to the password policy by controlling the entropy of output distribution.


Authentication Password guessing Neural network 



The authors are grateful to the anonymous reviewers for their constructive comments. This work was supported in part by the National Natural Science Foundation of China under Grant 61702399 and Grant 61772291 and Grant 61972215 in part by the Natural Science Foundation of Tianjin, China, under Grant 17JCZDJC30500.


  1. 1.
    John the ripper (1996).
  2. 2.
    Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)
  3. 3.
    Castelluccia, C., Dürmuth, M., Perito, D.: Adaptive password-strength meters from Markov models. In: NDSS (2012)Google Scholar
  4. 4.
    Dell’Amico, M., Filippone, M.: Monte carlo strength evaluation: fast and reliable password checking. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 158–169. ACM (2015)Google Scholar
  5. 5.
    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  6. 6.
    Dürmuth, M., Angelstorf, F., Castelluccia, C., Perito, D., Chaabane, A.: OMEN: faster password guessing using an ordered Markov enumerator. In: Piessens, F., Caballero, J., Bielova, N. (eds.) ESSoS 2015. LNCS, vol. 8978, pp. 119–132. Springer, Cham (2015). Scholar
  7. 7.
    Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 1019–1027 (2016)Google Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  9. 9.
    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
  10. 10.
    Hitaj, B., Gasti, P., Ateniese, G., Perez-Cruz, F.: PassGAN: a deep learning approach for password guessing. In: Deng, R.H., Gauthier-Umaña, V., Ochoa, M., Yung, M. (eds.) ACNS 2019. LNCS, vol. 11464, pp. 217–237. Springer, Cham (2019). Scholar
  11. 11.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1996)CrossRefGoogle Scholar
  12. 12.
    Houshmand, S., Aggarwal, S., Flood, R.: Next gen PCFG password cracking. IEEE Trans. Inf. Forensics Secur. 10(8), 1776–1791 (2015)CrossRefGoogle Scholar
  13. 13.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
  14. 14.
  15. 15.
    Kelley, P.G., et al.: Guess again (and again and again): measuring password strength by simulating password-cracking algorithms. In: 2012 IEEE Symposium on Security and Privacy (SP), pp. 523–537. IEEE (2012)Google Scholar
  16. 16.
    Krause, B., Kahembwe, E., Murray, I., Renals, S.: Dynamic evaluation of neural sequence models. arXiv preprint arXiv:1709.07432 (2017)
  17. 17.
    Li, Z., Han, W., Xu, W.: A large-scale empirical analysis of chinese web passwords. In: USENIX Security Symposium, pp. 559–574 (2014)Google Scholar
  18. 18.
    Liu, Y., et al.: GENPass: a general deep learning model for password guessing with PCFG rules and adversarial generation. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2018)Google Scholar
  19. 19.
    Ma, J., Yang, W., Luo, M., Li, N.: A study of probabilistic password models. In: 2014 IEEE Symposium on Security and Privacy (SP), pp. 689–704. IEEE (2014)Google Scholar
  20. 20.
    Melicher, W., et al.: Fast, lean, and accurate: modeling password guessability using neural networks. In: USENIX Security Symposium, pp. 175–191 (2016)Google Scholar
  21. 21.
    Merity, S., Keskar, N.S., Socher, R.: Regularizing and optimizing LSTM language models. arXiv preprint arXiv:1708.02182 (2017)
  22. 22.
  23. 23.
    Narayanan, A., Shmatikov, V.: Fast dictionary attacks on passwords using time-space tradeoff. In: Proceedings of the 12th ACM Conference on Computer and Communications Security, pp. 364–372. ACM (2005)Google Scholar
  24. 24.
  25. 25.
  26. 26.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Tang, Z., Wang, D., Zhang, Z.: Recurrent neural network training with dark knowledge transfer. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5900–5904. IEEE (2016)Google Scholar
  28. 28.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)Google Scholar
  29. 29.
    Weir, M., Aggarwal, S., De Medeiros, B., Glodek, B.: Password cracking using probabilistic context-free grammars. In: 2009 30th IEEE Symposium on Security and Privacy, pp. 391–405. IEEE (2009)Google Scholar
  30. 30.
    Xu, L., et al.: Password guessing based on LSTM recurrent neural networks. In: 2017 IEEE International Conference on Computational Science and Engineering (CSE) and Embedded and Ubiquitous Computing (EUC), vol. 1, pp. 785–788. IEEE (2017)Google Scholar
  31. 31.
  32. 32.
    Yang, Z., Dai, Z., Salakhutdinov, R., Cohen, W.W.: Breaking the softmax bottleneck: a high-rank RNN language model. arXiv preprint arXiv:1711.03953 (2017)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Hang Li
    • 1
    • 3
  • Mengqi Chen
    • 2
    • 3
  • Shengbo Yan
    • 2
    • 3
  • Chunfu Jia
    • 2
    • 3
    Email author
  • Zhaohui Li
    • 2
    • 3
  1. 1.College of Artificial IntelligenceNankai UniversityTianjinChina
  2. 2.College of Cyber ScienceNankai UniversityTianjinChina
  3. 3.Tianjin Key Laboratory of Network and Data SecurityNankai UniversityTianjinChina

Personalised recommendations