The relative merit of empirical priors in non-identifiable and sloppy models: Applications to models of learning and decision-making
- 226 Downloads
Formal modeling approaches to cognition provide a principled characterization of observed responses in terms of a set of postulated processes, specifically in terms of parameters that modulate the latter. These model-based characterizations are useful to the extent that there is a clear, one-to-one mapping between parameters and model expectations (identifiability) and that parameters can be recovered from reasonably sized data using a typical experimental design (recoverability). These properties are sometimes not met for certain combinations of model classes and data. One suggestion to improve parameter identifiability and recoverability involves the use of “empirical priors”, which constrain parameters according to a previously observed distribution of values. We assessed the efficacy of this proposal using a combination of real and artificial data. Our results showed that a point-estimate variant of the empirical-prior method could not improve parameter recovery systematically. We identified the source of poor parameter recovery in the low information content of the data. As a follow-up step, we developed a fully Bayesian variant of the empirical-prior method and assessed its performance. We find that even such a method that takes the covariance structure of the parameter distributions into account cannot reliably improve parameter recovery. We conclude that researchers should invest additional efforts in improving the informativeness of their experimental designs, as many of the problems associated to impoverished designs cannot be alleviated by modern statistical methods alone.
KeywordsIdentifiability Empirical priors Reinforcement learning Prospect theory
- Ahn, W.-Y., Vasilev, G., Lee, S.-H., Busemeyer, J. R., Kruschke, J. K., Bechara, A., & Vassileva, J. (2014). Decision-making in stimulant and opiate addicts in protracted abstinence: Evidence from computational modeling with pure users. Frontiers in Psychology, 5, 1–15. https://doi.org/10.3389/fpsyg.2014.00849 CrossRefGoogle Scholar
- Canessa, N., Crespi, C., Motterlini, M., Baud-Bovy, G., Chierchia, G., Pantaleo, G., & Cappa, S. F. (2013). The functional and structural neural basis of individual differences in loss aversion. Journal of Neuroscience, 33, 14307–14317. https://doi.org/10.1523/JNEUROSCI.0497-13.2013 CrossRefPubMedGoogle Scholar
- Chase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015). Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis. Cognitive, Affective, & Behavioral Neuroscience. https://doi.org/10.3758/s13415-015-0338-7.
- Cousineau, D., & Hélie, S. (2013). Improving maximum likelihood estimation using prior probabilities: A tutorial on maximum a posteriori estimation and an examination of the Weibull distribution. Tutorials in Quantitative Methods for Psychology, 9, 61–71. https://doi.org/10.20982/tqmp.09.2.p061 CrossRefGoogle Scholar
- Frank, M. J., Gagne, C., Nyhus, E., Masters, S., Wiecki, T. V., Cavanagh, J. F., & Badre, D. (2015). fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning. Journal of Neuroscience, 35, 485–494. https://doi.org/10.1523/JNEUROSCI.2036-14.2015.CrossRefPubMedPubMedCentralGoogle Scholar
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) Bayesian data analysis, (3rd edn.) Boca Raton: CRC Press.Google Scholar
- Hartig, F., Minunno, F., & Paul, S. (2017). BayesianTools: General-purpose MCMC and SMC samplers and tools for Bayesian statistics. R package version 0.1.3. Retrieved from https://github.com/florianhartig/bayesiantools.
- Hulme, C., Roodenrys, S., Schweickert, R., Brown, G. D. A., et al., (1997). Word-frequency effects on short-term memory tasks: Evidence for a redintegration process in immediate serial recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 1217–1232. https://doi.org/10.1037//0278-73220.127.116.117 PubMedGoogle Scholar
- Lee, D., Seo, H., & Jung, M. W. (2012). Neural basis of reinforcement learning and decision making. Annual Review of Neuroscience, 35, 287–308. https://doi.org/10.1146/annurev-neuro-062111-150512 CrossRefPubMedPubMedCentralGoogle Scholar
- Lee, M. D. (forthcoming). Bayesian methods in cognitive modeling. In J.T. Wixted (Ed.) The Stevens’ handbook of experimental psychology and cognitive neuroscience (4th edition, volume 5: Methodology). New York: Wiley.Google Scholar
- Lee, M. D., & Vanpaemel, W. (2017). Determining informative priors for cognitive models. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-017-1238-3.
- Lewandowsky, S., & Farrell, S. (2010) Computational modeling in cognition: Principles and practice. Thousand Oaks: Sage Publications Inc.Google Scholar
- Niv, Y., Daniel, R., Geana, A., Gershman, S. J., Leong, Y. C., Radulescu, A., & Wilson, R. C. (2015). Reinforcement learning in multidimensional environments relies on attention mechanisms. Journal of Neuroscience, 35, 8145–8157. https://doi.org/10.1523/JNEUROSCI.2978-14.2015 CrossRefPubMedPubMedCentralGoogle Scholar
- R Development Core Team (2008). R: A language and environment for statistical computing. R Foundation for Statistical computing, Vienna, Austria. Retrieved from http://www.r-project.org.
- Stan Development Team (2016a). PyStan: The Python interface to Stan. Retrieved from http://mc-stan.org.
- Stan Development Team (2016b). RStan: The R interface to Stan. Retrieved from http://mc-stan.org.
- Sutton, R. S., & Barto, A. G. (1998) Reinforcement learning: An introduction. Cambridge: MIT Press.Google Scholar
- White, C. N., Servant, M., & Logan, G. D. (2017). Testing the validity of conflict drift-diffusion models for use in estimating cognitive processes: a parameter-recovery study. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-017-1271-2.