Behavior Research Methods

, Volume 51, Issue 6, pp 2777–2799 | Cite as

Parallel probability density approximation

  • Yi-Shin LinEmail author
  • Andrew Heathcote
  • William R. Holmes


Probability density approximation (PDA) is a nonparametric method of calculating probability densities. When integrated into Bayesian estimation, it allows researchers to fit psychological processes for which analytic probability functions are unavailable, significantly expanding the scope of theories that can be quantitatively tested. PDA is, however, computationally intensive, requiring large numbers of Monte Carlo simulations in order to attain good precision. We introduce Parallel PDA (pPDA), a highly efficient implementation of this method utilizing the Armadillo C++ and CUDA C libraries to conduct millions of model simulations simultaneously in graphics processing units (GPUs). This approach provides a practical solution for rapidly approximating probability densities with high precision. In addition to demonstrating this method, we fit a piecewise linear ballistic accumulator model (Holmes, Trueblood, & Heathcote, 2016) to empirical data. Finally, we conducted simulation studies to investigate various issues associated with PDA and provide guidelines for pPDA applications to other complex cognitive models.


C++ CUDA GPU Kernel density estimate Markov chain Monte Carlo Bayesian modeling Probability density approximation 


Author note

W.R.H. was supported by National Science Foundation (USA) Grant SES-1530760. A.H. is supported by Australian Research Council Discovery Project DP160101891


  1. Ball, K., & Sekuler, R. (1982). A specific and enduring improvement in visual motion discrimination. Science, 218, 697–698. CrossRefPubMedGoogle Scholar
  2. Beaumont, M. A. (2010). Approximate Bayesian computation in evolution and ecology. Annual Review of Ecology, Evolution, and Systematics, 41, 379–406.CrossRefGoogle Scholar
  3. Brooks, S., Gelman, A., Jones, G., & Meng, X. L. (2011). Handbook of Markov chain Monte Carlo. New York, NY: CRC Press.CrossRefGoogle Scholar
  4. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. CrossRefPubMedGoogle Scholar
  5. Cheng, J., Grossman, M., & McKercher, T. (2014). Professional CUDA C programming. Indianapolis, IN: Wiley.Google Scholar
  6. Chiu, S.-T. (1991). Bandwidth selection for kernel density estimation. Annals of Statistics, 19, 1883–1905. Retrieved from CrossRefGoogle Scholar
  7. Cisek, P., Puskas, G. A., & El-Murr, S. (2009). Decisions in changing conditions: the urgency-gating model. Journal of Neuroscience, 29, 11560–11571CrossRefGoogle Scholar
  8. Dawson, M. R. (1988). Fitting the ex-Gaussian equation to reaction time distributions. Behavior Research Methods, Instruments, & Computers, 20, 54–57.CrossRefGoogle Scholar
  9. Dutilh, G., Annis, J., Brown, S. D., Cassey, P., Evans, N. J., Grasman, R. P. P. P., . . . Donkin, C. (2018). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review. Advance online publication. CrossRefGoogle Scholar
  10. Gelman, A. (2014). Bayesian data analysis. Boca Raton, FL: CRC Press.Google Scholar
  11. Goldenshluger, A., & Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Annals of Statistics, 39, 1608–1632. CrossRefGoogle Scholar
  12. Gureckis, T. M., & Love, B. C. (2009). Learning in noise: Dynamic decision-making in a variable environment. Journal of Mathematical Psychology, 53, 180–193.CrossRefGoogle Scholar
  13. Harris, M. (2007). Optimizing parallel reduction in CUDA. Retrieved from
  14. Heathcote, A. (2004). Fitting Wald and ex-Wald distributions to response time data: An example using functions for the S-PLUS package. Behavior Research Methods, Instruments, & Computers, 36, 678–694. CrossRefGoogle Scholar
  15. Heathcote, A., Brown, S. D., & Wagenmakers, E.-J. (2015). An introduction to good practices in cognitive modeling. In B. U. Forstmann & E.-J. Wagenmakers (Eds.), An introduction to model-based cognitive neuroscience (pp. 25–48). New York, NY, US: Springer Science + Business Media.Google Scholar
  16. Heathcote, A., Lin, Y.-S., Reynolds, A., Strickland, L., Gretton, M., & Matzke, D. (2018). Dynamic models of choice. Behavior Research Methods. Advance online publication. CrossRefGoogle Scholar
  17. Hoffman, M. D., & Gelman, A. (2014). The no-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1593–1623.Google Scholar
  18. Hohle, R. H. (1965). Inferred components of reaction times as functions of foreperiod duration. Journal of Experimental Psychology, 69, 382–386. CrossRefPubMedGoogle Scholar
  19. Holmes, W. R. (2015). A practical guide to the Probability Density Approximation (PDA) with improved implementation and error characterization. Journal of Mathematical Psychology, 68–69, 13–24. CrossRefGoogle Scholar
  20. Holmes, W. R., & Trueblood J. S. (2018). Bayesian analysis of the piecewise diffusion decision model. Behavior Research Methods, 50, 730–743. CrossRefPubMedGoogle Scholar
  21. Holmes, W. R., Trueblood, J. S., & Heathcote, A. (2016). A new framework for modeling decisions about changing information: The piecewise linear ballistic accumulator model. Cognitive Psychology, 85, 1–29.CrossRefGoogle Scholar
  22. Hu, B., & Tsui, K.-W. (2005). Distributed evolutionary Monte Carlo with applications to Bayesian analysis (Working paper). Madison, WI: University of Wisconsin, Department of Statistics.Google Scholar
  23. Hu, B., & Tsui, K.-W. (2010). Distributed evolutionary Monte Carlo for Bayesian computing. Computional Statistics and Data Analysis, 54, 688–697. CrossRefGoogle Scholar
  24. Luce, R. D (1986). Response times. New York, NY: Oxford University Press.Google Scholar
  25. Matzke, D., & Wagenmakers, E.-J. (2009). Psychological interpretation of the ex-Gaussian and shifted Wald parameters: A diffusion model analysis. Psychonomic Bulletin & Review, 16, 798–817. CrossRefGoogle Scholar
  26. McClelland, J. L. (1979). On the time relations of mental processes: An examination of systems of processes in cascade. Psychological Review, 86, 287–330. CrossRefGoogle Scholar
  27. Miletić, S., Turner, B. M., Forstmann, B. U., & van Maanen, L. (2017). Parameter recovery for the Leaky Competing Accumulator model. Journal of Mathematical Psychology, 76, 25–50. CrossRefGoogle Scholar
  28. Neal, R. M. (1994). An improved acceptance procedure for the hybrid Monte Carlo algorithm. Journal of Computational Physics, 111, 194–203.CrossRefGoogle Scholar
  29. Nvidia. (2018). CUDA C programming guide PG-02829-001_v9.1 | March 2018. Retrieved 20 Apr 2018 from
  30. Palestro, J. J., Sederberg, P. B., Osth, A. F., van Zandt, T., & Turner, B. M. (2018). Likelihood-free methods for cognitive science. Cham, Switzerland: Springer.CrossRefGoogle Scholar
  31. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076.CrossRefGoogle Scholar
  32. R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from Google Scholar
  33. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. CrossRefGoogle Scholar
  34. Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20, 873–922. CrossRefPubMedPubMedCentralGoogle Scholar
  35. Robert, C. P. (1995). Simulation of truncated normal variables. Statistics and Computing, 5, 121–125. CrossRefGoogle Scholar
  36. Roberts, G. O., & Rosenthal, J. S. (2001). Optimal scaling for various Metropolis–Hastings algorithms. Statistical Science, 16, 351–367.CrossRefGoogle Scholar
  37. Sanderson, C., & Curtin, R. (2016). Armadillo: A template-based C++ library for linear algebra. Journal of Open Source Software, 1(2), 26. CrossRefGoogle Scholar
  38. Schwarz, W. (2002). On the convolution of inverse Gaussian and exponential random variables. Communications in Statistics Theory and Methods, 31, 2113–2121.CrossRefGoogle Scholar
  39. Silverman, B. W. (1982). Algorithm AS 176: Kernel density estimation using the fast Fourier transform. Journal of the Royal Statistical Society: Series C, 31, 93–99. CrossRefGoogle Scholar
  40. Silverman, B. W. (1986). Density estimation for statistics and data analysis. London, UK: Chapman & Hall.CrossRefGoogle Scholar
  41. Sisson, S. A., & Fan, Y. (2010). Likelihood-free Markov chain Monte Carlo. arXiv preprint. arXiv:1001.2058.Google Scholar
  42. Smith, P. L. (2016). Diffusion theory of decision making in continuous report. Psychological Review, 123, 425–451. CrossRefPubMedGoogle Scholar
  43. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B, 64, 583–639.CrossRefGoogle Scholar
  44. Ter Braak, C. J. F. (2006). A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: Easy Bayesian computing for real parameter spaces. Statistics and Computing, 16, 239–249.CrossRefGoogle Scholar
  45. Thura, D., Beauregard-Racine, J., Fradet, C.-W., & Cisek, P. (2012). Decision-making by urgency gating: Theory and experimental support. Journal of Neurophysiology, 108, 2912–2930.CrossRefGoogle Scholar
  46. Trueblood, J. S., Holmes, W. R., Seegmiller, A. C., Douds, J., Compton, M., Szentirmai, E., . . . Eichbaum, Q. (2018). The impact of speed and bias on the cognitive processes of experts and novices in medical image decision-making. Cognitive Research: Principles and Implications, 3, 28.Google Scholar
  47. Tsetsos, K., Usher, M., & McClelland, J. L. (2011). Testing multi-alternative decision models with non-stationary evidence. Frontiers in Neuroscience, 5, 63. CrossRefPubMedPubMedCentralGoogle Scholar
  48. Turner, B. M., & Sederberg, P. B. (2012). Approximate Bayesian computation with differential evolution. Journal of Mathematical Psychology, 56, 375–385.CrossRefGoogle Scholar
  49. Turner, B. M., & Sederberg, P. B. (2014). A generalized, likelihood-free method for posterior estimation. Psychonomic Bulletin & Review, 21, 227–250. CrossRefGoogle Scholar
  50. Turner, B. M., Sederberg, P. B., Brown, S. D., & Steyvers, M. (2013). A method for efficiently sampling from distributions with correlated dimensions. Psychological Methods, 18, 368–384. CrossRefPubMedPubMedCentralGoogle Scholar
  51. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592. CrossRefGoogle Scholar
  52. Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin & Review, 7, 424–465. CrossRefGoogle Scholar
  53. Verdonck, S., Meers, K., & Tuerlinckx, F. (2016). Efficient simulation of diffusion-based choice RT models on CPU and GPU. Behavior Research Methods, 48, 13–27. CrossRefPubMedGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  • Yi-Shin Lin
    • 1
    Email author
  • Andrew Heathcote
    • 1
  • William R. Holmes
    • 2
  1. 1.Division of Psychology, School of MedicineUniversity of TasmaniaHobartAustralia
  2. 2.Department of Physics and Astronomy & Department of Mathematics, Quantitative Systems Biology CenterVanderbilt UniversityNashvilleUSA

Personalised recommendations