NIST SP800-22 is one of the most widely used statistical testing tools for pseudorandom number generators (PRNGs). This tool consists of 15 tests (one-level tests) and two additional tests (two-level tests). Each one-level test provides one or more p-values. The two-level tests measure the uniformity of the obtained p-values for a fixed one-level test. One of the two-level tests categorizes the p-values into ten intervals of equal length, and apply a chi-squared goodness-of-fit test. This two-level test is often more powerful than one-level tests, but sometimes it rejects even good PRNGs when the sample size at the second level is too large, since it detects approximation errors in the computation of p-values. In this paper, we propose a practical upper limit of the sample size in this two-level test, for each of six tests appeared in SP800-22. These upper limits are derived by the chi-squared discrepancy between the distribution of the approximated p-values and the uniform distribution U(0, 1). We also computed a “risky” sample size at the second level for each one-level test. Our experiments show that the two-level test with the proposed upper limit gives appropriate results, while using the risky size often rejects even good PRNGs. We also propose another improvement: to use the exact probability for the ten categories in the computation of goodness-of-fit at the two-level test. This allows us to increase the sample size at the second level, and would make the test more sensitive than the NIST’s recommending usage.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Bassham III, L.E., et al.: SP 800-22 Rev. 1a. A statistical test suite for random and pseudorandom number generators for cryptographic applications. Technical report, National Institute of Standards & Technology, Gaithersburg, MD, United States. https://csrc.nist.gov/projects/random-bit-generation/documentation-and-software (2010)
Doty-Humphrey, C.: Practically random: specific tests in PractRand. http://pracrand.sourceforge.net/Tests_engines.txt
Galassi, M., et al.: GNU scientific library reference manual. https://www.gnu.org/software/gsl/ (2018)
Haramoto, H.: Automation of statistical tests on randomness to obtain clearer conclusion. In: Ecuyer, P.L., Owen, A.B. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2008, pp. 411–421. Springer, Berlin (2009)
Haramoto, H., Matsumoto, M.: A method to compute an appropriate sample size of a two-level test for the NIST test suite. In: Owen, A.B., Glynn, P.W. (eds.) Monte Carlo and Quasi-Monte Carlo Methods, pp. 283–294. Springer International Publishing, Cham (2018)
Haramoto, H., Matsumoto, M.: Checking the quality of approximation of p-values in statistical tests for random number generators by using a three-level test. Mathematics and computers in simulation 161, 66–75 (2019). https://doi.org/10.1016/j.matcom.2018.08.005. http://www.sciencedirect.com/science/article/pii/S0378475418302039(Special issue on the Eleventh International Conference on Monte Carlo Methods and Applications (MCM 2017), held in Montreal, Canada, July 03–07, 2017)
Kim, S.J., Umeno, K., Hasegawa, A.: Corrections of the NIST statistical test suite for randomness. Cryptology ePrint Archive, Report 2004/018. https://eprint.iacr.org/2004/018 (2004)
Knuth, D.E.: The Art of Computer Programming, Volume 2 (3rd ed.): Seminumerical Algorithms. Addison-Wesley, Boston (1997)
L’Ecuyer, P.: Testing random number generators. In: Proceedings of the 1992 Winter Simulation Conference, pp. 305–313. IEEE Press (1992)
L’Ecuyer, P.: Uniform random number generation. Ann. Oper. Res. 53, 77–120 (1994)
L’Ecuyer, P.: Tests based on sum-functions of spacings for uniform random numbers. J. Stat. Comput. Simul. 59(3), 251–269 (1997). https://doi.org/10.1080/00949659708811859
L’Ecuyer, P., Simard, R.: TestU01: a C library for empirical testing of random number generators. ACM Trans. Math. Softw. 33(4), 22:1–22:40 (2007). https://doi.org/10.1145/1268776.1268777
Matsumoto, M., Nishimura, T.: Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8(1), 3–30 (1998)
Matsumoto, M., Nishimura, T.: A Nonempirical test on the weight of pseudorandom number generators. Monte Carlo and quasi-Monte Carlo methods. 2000 (Hong Kong), pp. 381–395. Springer, Berlin (2002)
Panneton, F., L’Ecuyer, P., Matsumoto, M.: Improved long-period generators based on linear recurrences modulo 2. ACM Trans. Math. Softw. 32(1), 1–16 (2006). https://doi.org/10.1145/1132973.1132974
Pareschi, F., Rovatti, R., Setti, G.: Second-level NIST randomness tests for improving test reliability. In: 2007 IEEE International Symposium on Circuits and Systems, pp. 1437–1440 (2007). https://doi.org/10.1109/ISCAS.2007.378572
Pareschi, F., Rovatti, R., Setti, G.: On statistical tests for randomness included in the NIST SP800-22 test suite and based on the binomial distribution. IEEE Trans. Inf. Forensics Secur. 7(2), 491–505 (2012). https://doi.org/10.1109/TIFS.2012.2185227
Shiryaev, A.N.: Probability, 2nd edn. Springer, Berlin (1995)
Tiku, M.: Noncentral chi-square distribution. In: Kotz, S., Johnson, N. (eds.) Encyclopedia of Statistical science, vol. 6, pp. 276–280. Wiley, New York (1981)
Yamaguchi, A., Saito, A.: On the statistical test of randomness based on the uniformity of \(p\)-values used in NIST statistical test suite. In: Proceedings of the 2015 JSIAM Annual Meeting, Applied Chaos, pp. 1–2 (2015)
The author is indebted to Professor Makoto Matsumoto for constant help and encouragements. The author is thankful to the anonymous referees for many valuable comments. This study was carried out under the ISM Cooperative Research Program (2018-ISMCRP-10 and 2019-ISMCRP-05).
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The author is supported by Grant-in-aid for Science Research, nos. 16K13750, 17K14234, and 18K03213.
About this article
Cite this article
Haramoto, H. Study on upper limit of sample size for a two-level test in NIST SP800-22. Japan J. Indust. Appl. Math. 38, 193–209 (2021). https://doi.org/10.1007/s13160-020-00434-y
- Statistical testing
- Pseudorandom number generators
- NIST SP800-22
- Two-level test
Mathematics Subject Classification