Neural Network Based Min-entropy Estimation for Random Number Generators

  • Jing Yang
  • Shuangyi Zhu
  • Tianyu ChenEmail author
  • Yuan Ma
  • Na Lv
  • Jingqiang Lin
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 255)


Random Number Generators (RNGs) are essential for cryptographic systems and communication security. A cryptographic application is prone to have a serious security risk if the entropy source that generates the random number cannot provide sufficient randomness (unpredictability) as expected. The min-entropy is usually employed to evaluate the unpredictability, which measures the difficulty of guessing the most likely output of RNGs. Recently, predictors for min-entropy estimation are proposed in the NIST 800-90B (90B), which attempt to predict the next sample in a sequence based on all previous samples. However, these predictors have shortfalls in evaluating random number with long dependence and multivariate due to huge time complexity (i.e., high-order polynomial time complexity). From the concept of predictors, we provide several suitable and efficient predictors based on neural networks for min-entropy estimation. The neural networks apply to approximating the Probability Distribution Function (PDF) and have a linear complexity of the sample space. Compared to the 90B’s predictors, the experimental results on various simulated source demonstrate that our proposed predictors have a comparable accuracy, and the execution efficiency has a significant improvement. Furthermore, when the sample space is over \(2^2\) and sample size is over \(10^8\), the 90B’s predictors cannot give the estimated result. Instead, our proposed predictors still can provide an accurate result.


Random number Neural network Entropy estimation Predictor 



This work was partially supported by National Natural Science Foundation of China (No. 61602476 and No. 61772518), and Cryptography Development Foundation of China (No. MMJJ20170205).


  1. 1.
    Amaki, T., Hashimoto, M., Mitsuyama, Y., Onoye, T.: A worst-case-aware design methodology for noise-tolerant oscillator-based true random number generator with stochastic behavior modeling. IEEE Trans. Inf. Forensics Secur. 8(8), 1331–1342 (2013)CrossRefGoogle Scholar
  2. 2.
    Aras, S., Kocakoç, I.D.: A new model selection strategy in time series forecasting with artificial neural networks: IHTS. Neurocomputing 174, 974–987 (2016). Scholar
  3. 3.
    Baudet, M., Lubicz, D., Micolod, J., Tassiaux, A.: On the security of oscillator-based random number generators. J. Cryptol. 24(2), 398–425 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Cai, X., Zhang, N., Venayagamoorthy, G.K., Wunsch II, D.C.: Time series prediction with recurrent neural networks trained by a hybrid PSO-EA algorithm. Neurocomputing 70(13–15), 2342–2353 (2007). Scholar
  5. 5.
    Donate, J.P., Li, X., Sánchez, G.G., de Miguel, A.S.: Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm. Neural Comput. Appl. 22(1), 11–20 (2013). Scholar
  6. 6.
    Dorrendorf, L., Gutterman, Z., Pinkas, B.: Cryptanalysis of the random number generator of the windows operating system. ACM Trans. Inf. Syst. Secur. 13(1), 10:1–10:32 (2009)CrossRefGoogle Scholar
  7. 7.
    Golic, J.D.: New methods for digital generation and postprocessing of random data. IEEE Trans. Comput. 55(10), 1217–1229 (2006)CrossRefGoogle Scholar
  8. 8.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.orgzbMATHGoogle Scholar
  9. 9.
    de Groot, C., Würtz, D.: Analysis of univariate time series with connectionist nets: a case study of two classical examples. Neurocomputing 3(4), 177–192 (1991). Scholar
  10. 10.
    Gutterman, Z., Pinkas, B., Reinman, T.: Analysis of the linux random number generator. In: 2006 IEEE Symposium on Security and Privacy (S&P 2006), 21–24 May 2006, Berkeley, California, USA, pp. 371–385 (2006)Google Scholar
  11. 11.
  12. 12.
    ISO/IEC JTC 1/SC 27, Berlin, Germany: ISO/IEC 18031: Information technology - Security techniques - Random bit generation (2011)Google Scholar
  13. 13.
    Jain, A., Kumar, A.M.: Hybrid neural network models for hydrologic time series forecasting. Appl. Soft Comput. 7(2), 585–592 (2007). Scholar
  14. 14.
    Menezes Jr., J.M.P., Barreto, G.A.: Long-term time series prediction with the NARX network: an empirical evaluation. Neurocomputing 71(16–18), 3335–3343 (2008). Scholar
  15. 15.
    Kelsey, J., McKay, K.A., Sönmez Turan, M.: Predictive models for min-entropy estimation. In: Güneysu, T., Handschuh, H. (eds.) CHES 2015. LNCS, vol. 9293, pp. 373–392. Springer, Heidelberg (2015). Scholar
  16. 16.
    Killmann, W., Schindler, W.: AIS 31: Functionality Classes and Evaluation Methodology for True (Physical) Random Number Generators. Version 3.1. T-Systems GEI GmbH and Bundesamt fr Sicherheit in der Informationstechnik (BSI), Bonn, Germany (2001)Google Scholar
  17. 17.
    Killmann, W., Schindler, W.: A design for a physical RNG with robust entropy estimators. In: Oswald, E., Rohatgi, P. (eds.) CHES 2008. LNCS, vol. 5154, pp. 146–163. Springer, Heidelberg (2008). Scholar
  18. 18.
    Luna-Sanchez, J.C., Gómez-Ramírez, E., Najim, K., Ikonen, E.: Forecasting time series with a logarithmic model for the polynomial artificial neural networks. In: The 2011 International Joint Conference on Neural Networks, IJCNN 2011, San Jose, California, USA, 31 July–5 August 2011, pp. 2725–2732 (2011).
  19. 19.
    Ma, Y., Lin, J., Chen, T., Xu, C., Liu, Z., Jing, J.: Entropy evaluation for oscillator-based true random number generators. In: Batina, L., Robshaw, M. (eds.) CHES 2014. LNCS, vol. 8731, pp. 544–561. Springer, Heidelberg (2014). Scholar
  20. 20.
    Menezes, A., van Oorschot, P.C., Vanstone, S.A.: Handbook of Applied Cryptography. CRC Press, Boca Raton (1996)CrossRefGoogle Scholar
  21. 21.
    NIST: The NIST Statistical Test Suite (2010).
  22. 22.
    Turan, M.S., Barker, E., Kelsey, J., McKay, K., Baish, M., Boyle, M.: (Second Draft) NIST special publication 800-90B: recommendation for the entropy sources used for random bit generation, January 2016.
  23. 23.
    Vanhoef, M., Piessens, F.: Predicting, decrypting, and abusing WPA2/802.11 group keys. In: 25th USENIX Security Symposium, USENIX Security 16, Austin, TX, USA, 10–12 August 2016, pp. 673–688 (2016)Google Scholar
  24. 24.
    Wieczorek, P.Z., Golofit, K.: Dual-metastability time-competitive true random number generator. IEEE Trans. Circuits Syst. 61–I(1), 134–145 (2014). Scholar
  25. 25.
    Yang, J., Ma, Y., Chen, T., Lin, J., Jing, J.: Extracting more entropy for TRNGs based on coherent sampling. In: Deng, R., Weng, J., Ren, K., Yegneswaran, V. (eds.) SecureComm 2016. LNICSSITE, vol. 198, pp. 694–709. Springer, Cham (2017). Scholar
  26. 26.
    Zhu, S., Ma, Y., Chen, T., Lin, J., Jing, J.: Analysis and improvement of entropy estimators in NIST SP 800-90B for Non-IID entropy sources. IACR Trans. Symmetric Cryptol. 2017(3), 151–168 (2017)Google Scholar
  27. 27.
    Zhu, S., Ma, Y., Lin, J., Zhuang, J., Jing, J.: More powerful and reliable second-level statistical randomness tests for NIST SP 800-22. In: Proceedings Advances in Cryptology - ASIACRYPT 2016 - 22nd International Conference on the Theory and Application of Cryptology and Information Security, Hanoi, Vietnam, 4–8 December 2016, Part I, pp. 307–329 (2016)CrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018

Authors and Affiliations

  • Jing Yang
    • 1
    • 2
    • 3
  • Shuangyi Zhu
    • 1
    • 2
    • 3
  • Tianyu Chen
    • 1
    • 2
    Email author
  • Yuan Ma
    • 1
    • 2
  • Na Lv
    • 1
    • 2
    • 3
  • Jingqiang Lin
    • 1
    • 2
  1. 1.Data Assurance and Communication Security Research CenterChinese Academy of SciencesBeijingChina
  2. 2.State Key Laboratory of Information Security, Institute of Information EngineeringChinese Academy of SciencesBeijingChina
  3. 3.School of Cyber SecurityUniversity of Chinese Academy of SciencesBeijingChina

Personalised recommendations