Accept–Reject Methods

  • Luca Martino
  • David Luengo
  • Joaquín Míguez
Chapter
Part of the Statistics and Computing book series (SCO)

Abstract

The accept/reject method, also known as rejection sampling (RS), was suggested by John von Neumann in 1951. It is a classical Monte Carlo technique for universal sampling that can be used to generate samples virtually from any target density p o (x) by drawing from a simpler proposal density π(x). The sample is either accepted or rejected by an adequate test of the ratio of the two pdfs, and it can be proved that accepted samples are actually distributed according to the target distribution. Specifically, the RS algorithm can be viewed as choosing a subsequence of i.i.d. realizations from the proposal density π(x) in such a way that the elements of the subsequence have density p o (x).

In this chapter, we present the basic theory of RS as well as different variants found in the literature. Computational cost issues and the range of applications are analyzed in depth. Several combinations with other Monte Carlo techniques are also described.

References

  1. 1.
    J.H. Ahrens, Sampling from general distributions by suboptimal division of domains. Grazer Math. Berichte 319, 20 (1993)MATHGoogle Scholar
  2. 2.
    J.H. Ahrens, A one-table method for sampling from continuous and discrete distributions. Computing 54(2), 127–146 (1995)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    N.C. Beaulieu, C. Cheng, Efficient Nakagami-m fading channel simulation. IEEE Trans. Veh. Technol. 54(2), 413–424 (2005)CrossRefGoogle Scholar
  4. 4.
    A. Bignami, A. De Matteis, A note on sampling from combinations of distributions. J. Inst. Math. Appl. 8(1), 80–81 (1971)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    C. Botts, W. Hörmann, J. Leydold, Transformed density rejection with inflection points. Stat. Comput. 23, 251–260 (2013)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    J.C. Butcher, Random sampling from the normal distribution. Comput. J. 3, 251–253 (1960)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    B.S. Caffo, J.G. Booth, A.C. Davison, Empirical supremum rejection sampling. Biometrika 89(4), 745–754 (2002)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    G. Casella, C.P. Robert, Post-processing accept-reject samples: recycling and rescaling. J. Comput. Graph. Stat. 7(2), 139–157 (1998)MathSciNetGoogle Scholar
  9. 9.
    R. Chen, Another look at rejection sampling through importance sampling. Stat. Probab. Lett. 72, 277–283 (2005)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    R.C.H. Cheng, The generation of gamma variables with non-integral shape parameter. J. R. Stat. Soc. Ser. C Appl. Stat. 26, 71–75 (1977)Google Scholar
  11. 11.
    J. Dagpunar, Principles of Random Variate Generation (Clarendon Press, Oxford/New York, 1988)MATHGoogle Scholar
  12. 12.
    L. Devroye, Random variate generation for unimodal and monotone densities. Computing 32, 43–68 (1984)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    L. Devroye, Non-Uniform Random Variate Generation (Springer, New York, 1986)CrossRefMATHGoogle Scholar
  14. 14.
    W.R. Gilks, N.G. Best, K.K.C. Tan, Adaptive rejection Metropolis sampling within Gibbs sampling. Appl. Stat. 44(4), 455–472 (1995)CrossRefMATHGoogle Scholar
  15. 15.
    J.A. Greenwood, Moments of time to generate random variables by rejection. Ann. Inst. Stat. Math. 28, 399–401 (1976)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    W. Hörmann, The transformed rejection method for generating Poisson random variables. Insur. Math. Econ. 12(1), 39–45 (1993)CrossRefGoogle Scholar
  17. 17.
    W. Hörmann, A note on the performance of the Ahrens algorithm. Computing 69, 83–89 (2002)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    W. Hörmann, G. Derflinger, The transformed rejection method for generating random variables, an alternative to the ratio of uniforms method. Commun. Stat. Simul. Comput. 23, 847–860 (1994)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    W. Hörmann, J. Leydold, G. Derflinger, Automatic Nonuniform Random Variate Generation (Springer, New York, 2003)MATHGoogle Scholar
  20. 20.
    R.A. Kronmal, A.V. Peterson, A variant of the acceptance-rejection method for computer generation of random variables. J. Am. Stat. Assoc. 76(374), 446–451 (1981)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    R.A. Kronmal, A.V. Peterson, An acceptance-complement analogue of the mixture-plus-acceptance-rejection method for generating random variables. ACM Trans. Math. Softw. 10(3), 271–281 (1984)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    P.K. Kythe, M.R. Schaferkotter, Handbook of Computational Methods for Integration (Chapman and Hall/CRC, Boca Raton, 2004)CrossRefMATHGoogle Scholar
  23. 23.
    J. Leydold, J. Janka, W. Hörmann, Variants of transformed density rejection and correlation induction, in Monte Carlo and Quasi-Monte Carlo Methods 2000 (Springer, Heidelberg, 2002), pp. 345–356Google Scholar
  24. 24.
    J.S. Liu, Monte Carlo Strategies in Scientific Computing (Springer, New York, 2004)CrossRefGoogle Scholar
  25. 25.
    J.S. Liu, R. Chen, W.H. Wong, Rejection control and sequential importance sampling. J. Am. Stat. Assoc. 93(443), 1022–1031 (1998)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    D. Luengo, L. Martino, Almost rejectionless sampling from Nakagami-m distributions (m ≥ 1). IET Electron. Lett. 48(24), 1559–1561 (2012)Google Scholar
  27. 27.
    I. Lux, Another special method to sample some probability density functions. Computing 21, 359–364 (1979)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    G. Marsaglia, The exact-approximation method for generating random variables in a computer. Am. Stat. Assoc. 79(385), 218–221 (1984)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    G. Marsaglia, W.W. Tsang, The ziggurat method for generating random variables. J. Stat. Softw. 8(5), 1–7 (2000)Google Scholar
  30. 30.
    L. Martino, J. Míguez, A novel rejection sampling scheme for posterior probability distributions, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2009)Google Scholar
  31. 31.
    L. Martino, J. Read, D. Luengo, Independent doubly adaptive rejection metropolis sampling within Gibbs sampling. IEEE Trans. Signal Process. 63(12), 3123–3138 (2015)MathSciNetCrossRefGoogle Scholar
  32. 32.
    L. Martino, H. Yang, D. Luengo, J. Kanniainen, J. Corander, A fast universal self-tuned sampler within Gibbs sampling. Dig. Signal Process. 47, 68–83 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    L. Martino, D. Luengo, Extremely efficient acceptance-rejection method for simulating uncorrelated Nakagami fading channels. Commun. Stat. Simul. Comput. (2018, to appear)Google Scholar
  34. 34.
    W.K. Pang, Z.H. Yang, S.H. Hou, P.K. Leung, Non-uniform random variate generation by the vertical strip method. Eur. J. Oper. Res. 142, 595–609 (2002)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    W.H. Payne, Normal random numbers: using machine analysis to choose the best algorithm. ACM Trans. Math. Softw. 4, 346–358 (1977)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    J.G. Proakis, Digital Communications, 4th edn. (McGraw-Hill, Singapore, 2000)MATHGoogle Scholar
  37. 37.
    C.P. Robert, G. Casella, Monte Carlo Statistical Methods (Springer, New York, 2004)CrossRefMATHGoogle Scholar
  38. 38.
    M. Sibuya, Further consideration on normal random variable generator. Ann. Inst. Stat. Math. 14, 159–165 (1962)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    L. Tierney, Exploring posterior distributions using Markov Chains, in Computer Science and Statistics: Proceedings of IEEE 23rd Symposium on the Interface (1991), pp. 563–570Google Scholar
  40. 40.
    L. Tierney, Markov chains for exploring posterior distributions. Ann. Stat. 22(4), 1701–1728 (1994)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    M.D. Troutt, W.K. Pang, S.H. Hou, Vertical Density Representation and Its Applications (World Scientific, Singapore, 2004)CrossRefMATHGoogle Scholar
  42. 42.
    I. Vaduva, On computer generation of gamma random variables by rejection and composition procedures. Math. Oper. Stat. Ser. Stat. 8, 545–576 (1977)MathSciNetMATHGoogle Scholar
  43. 43.
    J. von Neumann, Various techniques in connection with random digits, in Monte Carlo Methods, ed. by A.S. Householder, G.E. Forsythe, H.H. Germond. National Bureau of Standards Applied Mathematics Series (U.S. Government Printing Office, Washington, DC, 1951), pp. 36–38Google Scholar
  44. 44.
    C.S. Wallace, Transformed rejection generators for gamma and normal pseudo-random variables. Aust. Comput. J. 8, 103–105 (1976)MathSciNetMATHGoogle Scholar
  45. 45.
    Q.M. Zhu, X.Y. Dang, D.Z. Xu, X.M. Chen, Highly efficient rejection method for generating Nakagami-m sequences. IET Electron. Lett. 47(19), 1100–1101 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Luca Martino
    • 1
  • David Luengo
    • 2
  • Joaquín Míguez
    • 1
  1. 1.Department of Signal Theory and CommunicationsCarlos III University of MadridMadridSpain
  2. 2.Department of Signal Theory and CommunicationsTechnical University of MadridMadridSpain

Personalised recommendations