Approximate Bayesian computational methods

Abstract

Approximate Bayesian Computation (ABC) methods, also known as likelihood-free techniques, have appeared in the past ten years as the most satisfactory approach to intractable likelihood problems, first in genetics then in a broader spectrum of applications. However, these methods suffer to some degree from calibration difficulties that make them rather volatile in their implementation and thus render them suspicious to the users of more traditional Monte Carlo methods. In this survey, we study the various improvements and extensions brought on the original ABC algorithm in recent years.

This is a preview of subscription content, access via your institution.

References

  1. Bartolucci, F., Scaccia, L., Mira, A.: Efficient Bayes factor estimation from the reversible jump output. Biometrika 93(1), 41–52 (2006)

    MathSciNet  MATH  Article  Google Scholar 

  2. Beaumont, M., Zhang, W., Balding, D.: Approximate Bayesian computation in population genetics. Genetics 162(4), 2025–2035 (2002)

    Google Scholar 

  3. Beaumont, M., Cornuet, J.-M., Marin, J.-M., Robert, C.: Adaptive approximate Bayesian computation. Biometrika 96(4), 983–990 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  4. Beaumont, M., Nielsen, R., Robert, C., Hey, J., Gaggiotti, O., Knowles, L., Estoup, A., Mahesh, P., Coranders, J., Hickerson, M., Sisson, S., Fagundes, N., Chikhi, L., Beerli, P., Vitalis, R., Cornuet, J.-M., Huelsenbeck, J., Foll, M., Yang, Z., Rousset, F., Balding, D., Excoffier, L.: In defense of model-based inference in phylogeography. Mol. Ecol. 19(3), 436–446 (2010)

    Article  Google Scholar 

  5. Berger, J., Fienberg, S., Raftery, A., Robert, C.: Incoherent phylogeographic inference. Proc. Natl. Acad. Sci. 107(41), E57 (2010)

    Article  Google Scholar 

  6. Blum, M.: Approximate Bayesian computation: a non-parametric perspective. J. Am. Stat. Assoc. 105(491), 1178–1187 (2010)

    Article  Google Scholar 

  7. Blum, M., François, O.: Non-linear regression models for approximate Bayesian computation. Stat. Comput. 20(1), 63–73 (2010)

    MathSciNet  Article  Google Scholar 

  8. Calvet, L., Czellar, V.: State-observation sampling and the econometrics of learning models. Technical Report (2011). arXiv:1105.4519

  9. Campillo, F., Rossi, V.: Convolution particle filter for parameter estimation in general state-space models. IEEE Trans. Aerosp. Electron. Syst. 45(3), 1063–1072 (2009)

    Article  Google Scholar 

  10. Cornuet, J.-M., Santos, F., Beaumont, M.A., Robert, C.P., Marin, J.-M., Balding, D.J., Guillemaud, T., Estoup, A.: Inferring population history with DIYABC: a user-friendly approach to approximate Bayesian computation. Bioinformatics 24(23), 2713–2719 (2008)

    Article  Google Scholar 

  11. Csillèry, K., Blum, M., Gaggiotti, O., François, O.: Approximate Bayesian computation (ABC) in practice. Trends Ecol. Evol. 25(7), 410–418 (2010a)

    Article  Google Scholar 

  12. Csillèry, K., Blum, M., Gaggiotti, O., François, O.: Invalid arguments against ABC: a reply to A.R. Templeton. Trends Ecol. Evol. 25(7), 490–491 (2010b)

    Article  Google Scholar 

  13. Cucala, L., Marin, J.-M., Robert, C., Titterington, D.: Bayesian inference in k-nearest-neighbour classification models. J. Am. Stat. Assoc. 104(485), 263–273 (2009)

    MathSciNet  Article  Google Scholar 

  14. Dean, T.A., Singh, S.S., Jasra, A., Peters, G.W.: Parameter estimation for hidden Markov models with intractable likelihoods. Technical Report (2011). arXiv:1103.5399

  15. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 68(3), 411–436 (2006)

    MATH  Article  Google Scholar 

  16. Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. (2011, to appear)

  17. Didelot, X., Everitt, R., Johansen, A., Lawson, D.: Likelihood-free estimation of model evidence. Bayesian Anal. 6(1), 48–76 (2011)

    MathSciNet  Article  Google Scholar 

  18. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Convergence of adaptive mixtures of importance sampling schemes. Ann. Stat. 35(1), 420–448 (2007)

    MathSciNet  MATH  Article  Google Scholar 

  19. Drovandi, C., Pettitt, A.: Estimation of parameters for macroparasite population evolution using approximate Bayesian computation. Biometrics 67(1), 225–233 (2010)

    MathSciNet  Article  Google Scholar 

  20. Fearnhead, P., Prangle, D.: Semi-automatic approximate Bayesian computation. Technical Report (2010). arXiv:1004.1112

  21. Friel, N., Pettitt, A.: Marginal likelihood estimation via power posteriors. J. R. Stat. Soc. B 70(3), 589–607 (2008)

    MathSciNet  MATH  Article  Google Scholar 

  22. Gauchi, J.-P., Vila, J.-P.: Nonparametric filtering approaches for identification and inference in nonlinear dynamic systems. Technical report. Personal communication (2011)

  23. Gelfand, A., Smith, A.: Sampling based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85(410), 398–409 (1990)

    MathSciNet  MATH  Article  Google Scholar 

  24. Gouriéroux, C., Monfort, A., Renault, E.: Indirect inference. J. Appl. Econom. 8, 85–118 (1993)

    Article  Google Scholar 

  25. Grelaud, A., Marin, J.-M., Robert, C., Rodolphe, F., Tally, F.: Likelihood-free methods for model choice in Gibbs random fields. Bayesian Anal. 3(2), 427–442 (2009)

    Google Scholar 

  26. Jaakkola, T., Jordan, M.: Bayesian parameter estimation via variational methods. Stat. Comput. 10(1), 25–37 (2000)

    Article  Google Scholar 

  27. Jasra, A., Singh, S.S., Martin, J.S., McCoy, E.: Filtering via approximate Bayesian computation. Stat. Comput. (2011, to appear)

  28. Joyce, P., Marjoram, P.: Approximately sufficient statistics and Bayesian computation. Stat. Appl. Genet. Mol. Biol. 7(1), 26 (2008)

    MathSciNet  Google Scholar 

  29. Leuenberger, C., Wegmann, D., Excoffier, L.: Bayesian computation and model selection in population genetics. Genetics 184(1), 243–252 (2010)

    Article  Google Scholar 

  30. Marin, J.-M., Robert, C.: Bayesian Core. Springer, New York (2007)

    MATH  Google Scholar 

  31. Marin, J.-M., Robert, C.: Importance sampling methods for Bayesian discrimination between embedded models. In: Chen, M.-H., Dey, D., Müller, P., Sun, D., Ye, K. (eds.) Frontiers of Statistical Decision Making and Bayesian Analysis, pp. 513–527. Springer, New York (2010)

    Google Scholar 

  32. Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. 100(26), 15324–15328 (2003)

    Article  Google Scholar 

  33. McKinley, T., Cook, A., Deardon, R.: Inference in epidemic models without likelihoods. Int. J. Biostat. 5(1), 24 (2009)

    MathSciNet  Google Scholar 

  34. Møller, J., Pettitt, A., Reeves, R., Berthelsen, K.: An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants. Biometrika 93(2), 451–458 (2006)

    MathSciNet  Article  Google Scholar 

  35. Pritchard, J., Seielstad, M., Perez-Lezaun, A., Feldman, M.: Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Mol. Biol. Evol. 16(12), 1791–1798 (1999)

    Article  Google Scholar 

  36. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2006)

    Google Scholar 

  37. Ratmann, O.: ABC under model uncertainty. PhD thesis, Imperial College, London (2009)

  38. Ratmann, O., Andrieu, C., Wiujf, C., Richardson, S.: Model criticism based on likelihood-free inference, with an application to protein network evolution. Proc. Natl. Acad. Sci. 106(26), 1–6 (2009)

    Google Scholar 

  39. Ratmann, O., Andrieu, C., Wiuf, C., Richardson, S.: Reply to Robert et al.: Model criticism informs model choice and model comparison. Proc. Natl. Acad. Sci. 107(3), E6 (2010)

    Article  Google Scholar 

  40. Robert, C.: The Bayesian Choice, 2nd edn. Springer, New York (2001)

    MATH  Google Scholar 

  41. Robert, C., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004)

    MATH  Google Scholar 

  42. Robert, C.P., Mengersen, K., Chen, C.: Model choice versus model criticism. Proc. Natl. Acad. Sci. 107(3), E5 (2010)

    Article  Google Scholar 

  43. Robert, C.P., Cornuet, J.-M., Marin, J.-M., Pillai, N.: Lack of confidence in ABC model choice. Proc. Natl. Acad. Sci. 108(37), 15112–15117 (2011)

    Article  Google Scholar 

  44. Rubin, D.: Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12(4), 1151–1172 (1984)

    MATH  Article  Google Scholar 

  45. Rue, H., Held, L.: Gaussian Markov Random Fields: Theory and Applications. Monographs on Statistics and Applied Probability, vol. 104. Chapman & Hall, London (2005)

    MATH  Book  Google Scholar 

  46. Rue, H., Martino, S., Chopin, N.: Approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations. J. R. Stat. Soc. B 71(2), 319–392 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  47. Sisson, S.A., Fan, Y., Tanaka, M.: Sequential Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. 104(6), 1760–1765 (2007)

    MathSciNet  MATH  Article  Google Scholar 

  48. Sisson, S.A., Fan, Y., Tanaka, M.: Sequential Monte Carlo without likelihoods: Errata. Proc. Natl. Acad. Sci. 106(39), 16889 (2009)

    Google Scholar 

  49. Tavaré, S., Balding, D., Griffith, R., Donnelly, P.: Inferring coalescence times from DNA sequence data. Genetics 145(2), 505–518 (1997)

    Google Scholar 

  50. Templeton, A.: Statistical hypothesis testing in intraspecific phylogeography: nested clade phylogeographical analysis vs. approximate Bayesian computation. Mol. Ecol. 18(2), 319–331 (2008)

    MathSciNet  Article  Google Scholar 

  51. Templeton, A.: Coherent and incoherent inference in phylogeography and human evolution. Proc. Natl. Acad. Sci. 107(14), 6376–6381 (2010)

    Article  Google Scholar 

  52. Tierney, L., Kadane, J.: Accurate approximations for posterior moments and marginal densities. J. Am. Stat. Assoc. 81(393), 82–86 (1986)

    MathSciNet  MATH  Article  Google Scholar 

  53. Toni, T., Stumpf, M.: Simulation-based model selection for dynamical systems in systems and population biology. Bioinformatics 26(1), 104–110 (2010)

    Article  Google Scholar 

  54. Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6(31), 187–202 (2009)

    Article  Google Scholar 

  55. Wasserman, L.: All of Nonparametric Statistics. Springer, New York (2007)

    Google Scholar 

  56. Wilkinson, R.D.: Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Technical Report (2008). arXiv:0811.3355

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jean-Michel Marin.

Additional information

This research was financially supported by the French Agence Nationale de la Recherche grant ‘EMILE’ ANR-09-BLAN-0145-01, as well as by the Fondation des Sciences Mathématiques de Paris and a GIS scholarship for the fourth author.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Marin, JM., Pudlo, P., Robert, C.P. et al. Approximate Bayesian computational methods. Stat Comput 22, 1167–1180 (2012). https://doi.org/10.1007/s11222-011-9288-2

Download citation

Keywords

  • Likelihood-free methods
  • Bayesian statistics
  • ABC methodology
  • DIYABC
  • Bayesian model choice