Skip to main content
SpringerLink
Log in

Study on upper limit of sample size for a two-level test in NIST SP800-22

  • Original Paper
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

NIST SP800-22 is one of the most widely used statistical testing tools for pseudorandom number generators (PRNGs). This tool consists of 15 tests (one-level tests) and two additional tests (two-level tests). Each one-level test provides one or more p-values. The two-level tests measure the uniformity of the obtained p-values for a fixed one-level test. One of the two-level tests categorizes the p-values into ten intervals of equal length, and apply a chi-squared goodness-of-fit test. This two-level test is often more powerful than one-level tests, but sometimes it rejects even good PRNGs when the sample size at the second level is too large, since it detects approximation errors in the computation of p-values. In this paper, we propose a practical upper limit of the sample size in this two-level test, for each of six tests appeared in SP800-22. These upper limits are derived by the chi-squared discrepancy between the distribution of the approximated p-values and the uniform distribution U(0, 1). We also computed a “risky” sample size at the second level for each one-level test. Our experiments show that the two-level test with the proposed upper limit gives appropriate results, while using the risky size often rejects even good PRNGs. We also propose another improvement: to use the exact probability for the ten categories in the computation of goodness-of-fit at the two-level test. This allows us to increase the sample size at the second level, and would make the test more sensitive than the NIST’s recommending usage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bassham III, L.E., et al.: SP 800-22 Rev. 1a. A statistical test suite for random and pseudorandom number generators for cryptographic applications. Technical report, National Institute of Standards & Technology, Gaithersburg, MD, United States. https://csrc.nist.gov/projects/random-bit-generation/documentation-and-software (2010)

  2. Doty-Humphrey, C.: Practically random: specific tests in PractRand. http://pracrand.sourceforge.net/Tests_engines.txt

  3. Galassi, M., et al.: GNU scientific library reference manual. https://www.gnu.org/software/gsl/ (2018)

  4. Haramoto, H.: Automation of statistical tests on randomness to obtain clearer conclusion. In: Ecuyer, P.L., Owen, A.B. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2008, pp. 411–421. Springer, Berlin (2009)

    Chapter  Google Scholar 

  5. Haramoto, H., Matsumoto, M.: A method to compute an appropriate sample size of a two-level test for the NIST test suite. In: Owen, A.B., Glynn, P.W. (eds.) Monte Carlo and Quasi-Monte Carlo Methods, pp. 283–294. Springer International Publishing, Cham (2018)

    Chapter  Google Scholar 

  6. Haramoto, H., Matsumoto, M.: Checking the quality of approximation of p-values in statistical tests for random number generators by using a three-level test. Mathematics and computers in simulation 161, 66–75 (2019). https://doi.org/10.1016/j.matcom.2018.08.005. http://www.sciencedirect.com/science/article/pii/S0378475418302039(Special issue on the Eleventh International Conference on Monte Carlo Methods and Applications (MCM 2017), held in Montreal, Canada, July 03–07, 2017)

  7. Kim, S.J., Umeno, K., Hasegawa, A.: Corrections of the NIST statistical test suite for randomness. Cryptology ePrint Archive, Report 2004/018. https://eprint.iacr.org/2004/018 (2004)

  8. Knuth, D.E.: The Art of Computer Programming, Volume 2 (3rd ed.): Seminumerical Algorithms. Addison-Wesley, Boston (1997)

    Google Scholar 

  9. L’Ecuyer, P.: Testing random number generators. In: Proceedings of the 1992 Winter Simulation Conference, pp. 305–313. IEEE Press (1992)

  10. L’Ecuyer, P.: Uniform random number generation. Ann. Oper. Res. 53, 77–120 (1994)

    Article  MathSciNet  Google Scholar 

  11. L’Ecuyer, P.: Tests based on sum-functions of spacings for uniform random numbers. J. Stat. Comput. Simul. 59(3), 251–269 (1997). https://doi.org/10.1080/00949659708811859

    Article  MATH  Google Scholar 

  12. L’Ecuyer, P., Simard, R.: TestU01: a C library for empirical testing of random number generators. ACM Trans. Math. Softw. 33(4), 22:1–22:40 (2007). https://doi.org/10.1145/1268776.1268777

  13. Matsumoto, M., Nishimura, T.: Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8(1), 3–30 (1998)

    Article  Google Scholar 

  14. Matsumoto, M., Nishimura, T.: A Nonempirical test on the weight of pseudorandom number generators. Monte Carlo and quasi-Monte Carlo methods. 2000 (Hong Kong), pp. 381–395. Springer, Berlin (2002)

  15. Panneton, F., L’Ecuyer, P., Matsumoto, M.: Improved long-period generators based on linear recurrences modulo 2. ACM Trans. Math. Softw. 32(1), 1–16 (2006). https://doi.org/10.1145/1132973.1132974

    Article  MathSciNet  MATH  Google Scholar 

  16. Pareschi, F., Rovatti, R., Setti, G.: Second-level NIST randomness tests for improving test reliability. In: 2007 IEEE International Symposium on Circuits and Systems, pp. 1437–1440 (2007). https://doi.org/10.1109/ISCAS.2007.378572

  17. Pareschi, F., Rovatti, R., Setti, G.: On statistical tests for randomness included in the NIST SP800-22 test suite and based on the binomial distribution. IEEE Trans. Inf. Forensics Secur. 7(2), 491–505 (2012). https://doi.org/10.1109/TIFS.2012.2185227

    Article  Google Scholar 

  18. Shiryaev, A.N.: Probability, 2nd edn. Springer, Berlin (1995)

    MATH  Google Scholar 

  19. Tiku, M.: Noncentral chi-square distribution. In: Kotz, S., Johnson, N. (eds.) Encyclopedia of Statistical science, vol. 6, pp. 276–280. Wiley, New York (1981)

    Google Scholar 

  20. Yamaguchi, A., Saito, A.: On the statistical test of randomness based on the uniformity of \(p\)-values used in NIST statistical test suite. In: Proceedings of the 2015 JSIAM Annual Meeting, Applied Chaos, pp. 1–2 (2015)

Download references

Acknowledgements

The author is indebted to Professor Makoto Matsumoto for constant help and encouragements. The author is thankful to the anonymous referees for many valuable comments. This study was carried out under the ISM Cooperative Research Program (2018-ISMCRP-10 and 2019-ISMCRP-05).

Author information

Authors and Affiliations

  1. Faculty of Education, Ehime University, 3 Bunkyocho, Matsuyama, Ehime, 790-8577, Japan

    Hiroshi Haramoto

Authors
  1. Hiroshi Haramoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroshi Haramoto.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The author is supported by Grant-in-aid for Science Research, nos. 16K13750, 17K14234, and 18K03213.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Haramoto, H. Study on upper limit of sample size for a two-level test in NIST SP800-22. Japan J. Indust. Appl. Math. 38, 193–209 (2021). https://doi.org/10.1007/s13160-020-00434-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-020-00434-y

Keywords

Mathematics Subject Classification

Search

Navigation