# Response-time data provide critical constraints on dynamic models of multi-alternative, multi-attribute choice

- 23 Downloads

## Abstract

Understanding the cognitive processes involved in multi-alternative, multi-attribute choice is of interest to a wide range of fields including psychology, neuroscience, and economics. Prior investigations in this domain have relied primarily on choice data to compare different theories. Despite numerous such studies, results have largely been inconclusive. Our study uses state-of-the-art response-time modeling and data from 12 different experiments appearing in six different published studies to compare four previously proposed theories/models of these effects: multi-alternative decision field theory (MDFT), the leaky-competing accumulator (LCA), the multi-attribute linear ballistic accumulator (MLBA), and the associative accumulation model (AAM). All four models are, by design, dynamic process models and thus a comprehensive evaluation of their theoretical properties requires quantitative evaluation with both choice and response-time data. Our results show that response-time data is critical at distinguishing among these models and that using choice data alone can lead to inconclusive results for some datasets. In conclusion, we encourage future research to include response-time data in the evaluation of these models.

## Keywords

Decision-making Multi-attribute choice Context effects Bayesian methods## Notes

### Acknowledgements

The authors would like to thank Audrey Parrish, Michael Beran, and George Farmer for sharing their data. The authors would also like to thank Jerome Busemeyer for his comments on extending MDFT to SDE formalism. All authors were supported by NSF grant SES-1556415. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the funding agency.

## Supplementary material

## References

- Berkowitsch, N. A., Scheibehenne, B., & Rieskamp, J. (2014). Rigorously testing multialternative decision field theory against random utility models.
*Journal of Experimental Psychology: General*,*143*(3), 1331.Google Scholar - Berkowitsch, N. A., Scheibehenne, B., Rieskamp, J., & Matthäus, M. (2015). A generalized distance function for preferential choices.
*British Journal of Mathematical and Statistical Psychology*,*68*(2), 310–325.PubMedGoogle Scholar - Bhatia, S. (2013). Associations and the accumulation of preference.
*Psychological Review*,*120*(3), 522.PubMedGoogle Scholar - Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation.
*Cognitive Psychology*,*57*, 153–178.PubMedGoogle Scholar - Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory.
*Mathematical Social Sciences*,*43*(3), 345–370.Google Scholar - Busemeyer, J. R., & Townsend, J. T. (1992). Fundamental derivations from decision field theory.
*Mathematical Social Sciences*,*23*(3), 255–282.Google Scholar - Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision-making in an uncertain environment.
*Psychological Review*,*100*(3), 432.PubMedGoogle Scholar - Busemeyer, J.R., & Wang, Y.-M. (2000). Model comparisons and model selections based on generalization criterion methodology.
*Journal of Mathematical Psychology*,*44*(1), 171–189.PubMedGoogle Scholar - Cataldo, A.M., & Cohen, A.L. (2018). Reversing the similarity effect: The effect of presentation format.
*Cognition*,*175*, 141–156.PubMedGoogle Scholar - Cohen, A. L., Kang, N., & Leise, T.L. (2017). Multi-attribute, multi-alternative models of choice: Choice, reaction time, and process tracing.
*Cognitive Psychology*,*98*, 45–72.PubMedGoogle Scholar - Donkin, C., Brown, S., Heathcote, A. J., & Wagenmakers, E. -J. (2011). Diffusion versus linear ballistic accumulation: Different models for response time, same conclusions about psychological mechanisms?
*Psychonomic Bulletin & Review*,*55*, 140–151.Google Scholar - Dutilh, G., Annis, J., Brown, S.D., Cassey, P., Evans, N.J., Grasman, R.P.P.P., & Donkin, C. (2018). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models.
*Psychonomic Bulletin & Review*. https://doi.org/10.3758/s13423-017-1417-2 - Estes, W. K. (1956). The problem of inference from curves based on group data.
*Psychological Bulletin*,*53*(2), 134.PubMedGoogle Scholar - Evans, N.J., & Brown, S.D. (2017). People adopt optimal policies in simple decision-making, after practice and guidance.
*Psychonomic Bulletin & Review*,*24*(2), 597–606.Google Scholar - Evans, N.J., & Brown, S.D. (2018). Bayes factors for the linear ballistic accumulator model of decision-making.
*Behavior Research Methods*,*50*(2), 589–603.PubMedGoogle Scholar - Evans, N. J., Hawkins, G. E., Boehm, U., Wagenmakers, E. -J., & Brown, S. D. (2017a). The computations that support simple decision-making: A comparison between the diffusion and urgency-gating models.
*Scientific Reports*,*7*, 16433.Google Scholar - Evans, N. J., Howard, Z. L., Heathcote, A., & Brown, S. D. (2017b). Model flexibility analysis does not measure the persuasiveness of a fit.
*Psychological Review*,*124*(3), 339.Google Scholar - Evans, N.J., Rae, B., Bushmakin, M., Rubin, M., & Brown, S.D. (2017c). Need for closure is associated with urgency in perceptual decision-making.
*Memory & Cognition*,*45*(7), 1193–1205.Google Scholar - Evans, N. J., Brown, S. D., Mewhort, D. J., & Heathcote, A. (2018). Refining the law of practice.
*Psychological Review*,*125*(4), 592.PubMedGoogle Scholar - Evans, N. J., Steyvers, M., & Brown, S.D (2018). Modeling the covariance structure of complex datasets using cognitive models: An application to individual differences and the heritability of cognitive ability.
*Cognitive Science*.Google Scholar - Farmer, G. D., Warren, P. A., El-Deredy, W., & Howes, A. (2016). The effect of expected value on attraction effect preference reversals.
*Journal of Behavioral Decision Making*.Google Scholar - Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007). Universally sloppy parameter sensitivities in systems biology models.
*PLoS Computational Biology*,*3*(10), e189.PubMedCentralGoogle Scholar - Heathcote, A., Brown, S., & Mewhort, D.J. (2000). The power law repealed: The case for an exponential law of practice.
*Psychonomic Bulletin & Review*,*7*(2), 185–207.Google Scholar - Ho, T. C., Yang, G., Wu, J., Cassey, P., Brown, S.D., Hoang, N., & Yang, T. T. (2014). Functional connectivity of negative emotional processing in adolescent depression.
*Journal of Affective Disorders*,*155*, 65–74. https://doi.org/10.1016/j.jad.2013.10.025 PubMedGoogle Scholar - Holmes, W.R. (2015). A practical guide to the probability density approximation (PDA) with improved implementation and error characterization.
*Journal of Mathematical Psychology*,*68*, 13–24.Google Scholar - Holmes, W.R., & Trueblood, J.S. (2018). Bayesian analysis of the piecewise diffusion decision model.
*Behavior Research Methods*,*50*(2), 730–743.PubMedGoogle Scholar - Holmes, W. R., Trueblood, J.S., & Heathcote, A. (2016). A new framework for modeling decisions about changing information: The piecewise linear ballistic accumulator model.
*Cognitive Psychology*,*85*, 1–29.PubMedPubMedCentralGoogle Scholar - Hotaling, J. M., Busemeyer, J. R., & Li, J. (2010). Theoretical developments in decision field theory: A comment on K. Tsetsos, N. Chater, and M. Usher.
*Psychological Review*,*117*, 1294– 1298.PubMedGoogle Scholar - Howes, A., Warren, P. A., Farmer, G., El-Deredy, W., & Lewis, R. L. (2016). Why contextual preference reversals maximize expected value.
*Psychological Review*,*123*(4), 368.PubMedPubMedCentralGoogle Scholar - Huang, K., Sen, S., & Szidarovszky, F. (2012). Connections among decision field theory models of cognition.
*Journal of Mathematical Psychology*,*56*(5), 287–296.Google Scholar - Huber, J., Payne, J. W., & Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis.
*Journal of Consumer Research*,*9*, 90–98.Google Scholar - Lerche, V., Voss, A., & Nagler, M. (2017). How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria.
*Behavior Research Methods*,*49*(2), 513–537.PubMedGoogle Scholar - Liew, S.X., Howe, P.D., & Little, D.R. (2016). The appropriacy of averaging in the study of context effects.
*Psychonomic Bulletin & Review*,*23*(5), 1639–1646.Google Scholar - Miletic̀, S., Turner, B. M., Forstmann, B. U., & van Maanen, L. (2017). Parameter recovery for the leaky competing accumulator model.
*Journal of Mathematical Psychology*,*76*, 25–50.Google Scholar - Myung, I.J. (2000). The importance of complexity in model selection.
*Journal of Mathematical Psychology*,*44*(1), 190–204.PubMedGoogle Scholar - Myung, I.J., & Pitt, M.A. (1997). Applying Occam’s razor in modeling cognition: A Bayesian approach.
*Psychonomic Bulletin & Review*,*4*(1), 79–95.Google Scholar - Nosofsky, R.M., & Palmeri, T.J. (2015). An exemplar-based random-walk model of categorization and recognition. In
*The Oxford handbook of computational and mathematical psychology*(p. 142). Oxford University Press, USA.Google Scholar - Parrish, A.E., Evans, T.A., & Beran, M.J. (2015). Rhesus macaques (
*Macaca mulatta*) exhibit the decoy effect in a perceptual discrimination task.*Attention, Perception, & Psychophysics*,*77*(5), 1715–1725.Google Scholar - Pettibone, J. C. (2012). Testing the effect of time pressure on asymmetric dominance and compromise decoys in choice.
*Judgment and Decision Making*,*7*(4), 513.Google Scholar - Ratcliff, R., Smith, P. L., Brown, S.D., & McKoon, G. (2016). Diffusion decision model: Current issues and history.
*Trends in Cognitive Sciences*,*20*(4), 260–281.PubMedPubMedCentralGoogle Scholar - Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision making.
*Psychological Review*,*108*, 370–392.PubMedGoogle Scholar - Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects.
*Journal of Consumer Research*,*16*, 158–174.Google Scholar - Soltani, A., De Martino, B., & Camerer, C. (2012). A range-normalization model of context-dependent choice: A new model and evidence.
*PLoS Computational Biology*,*8*(7), 1–15.Google Scholar - Spiegelhalter, D.J., Best, N.G., Carlin, B.P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit.
*Journal of the Royal Statistical Society: Series B (Statistical Methodology)*,*64*(4), 583–639.Google Scholar - Ter Braak, C.J. (2006). A Markov chain Monte Carlo version of the genetic algorithm differential evolution: Easy Bayesian computing for real parameter spaces.
*Statistics and Computing*,*16*(3), 239–249.Google Scholar - Trueblood, J. S. (2012). Multi-alternative context effects obtained using an inference task.
*Psychonomic Bulletin & Review*,*19*(5), 962–968.Google Scholar - Trueblood, J. S., Brown, S. D., & Heathcote, A. (2014). The multiattribute linear ballistic accumulator model of context effects in multialternative choice.
*Psychological Review*,*121*(2), 179.PubMedGoogle Scholar - Trueblood, J.S., Brown, S.D., & Heathcote, A. (2015). The fragile nature of contextual preference reversals: Reply to Tsetsos, Chater, and Usher (2015).
*Psychological Review*,*122*(4), 848–853.PubMedGoogle Scholar - Trueblood, J. S., Brown, S. D., Heathcote, A., & Busemeyer, J. R. (2013). Not just for consumers: Context effects are fundamental to decision-making.
*Psychological Science*,*24*, 901–908.PubMedGoogle Scholar - Trueblood, J. S., Holmes, W. R., Seegmiller, A. C., Douds, J., Compton, M., Szentirmai, E., & Eichbaum, Q. (2018). The impact of speed and bias on the cognitive processes of experts and novices in medical image decision-making.
*Cognitive Research: Principles and Implications*,*3*(1), 28.Google Scholar - Trueblood, J. S., & Pettibone, J. C. (2017). The phantom decoy effect in perceptual decision making.
*Journal of Behavioral Decision Making*,*30*(2), 157–167.Google Scholar - Tsetsos, K., Chater, N., & Usher, M. (2015). Examining the mechanisms underlying contextual preference reversal: Comment on Trueblood, Brown, and Heathcote (2014).
*Psychological Review*,*122*(4), 838–847.PubMedGoogle Scholar - Tsetsos, K., Usher, M., & Chater, N. (2010). Preference reversal in multi-attribute choice.
*Psychological Review*,*117*, 1275–1291.PubMedGoogle Scholar - Turner, B. M., Schley, D. R., Muller, C., & Tsetsos, K. (2018). Competing models of multi-attribute, multi-alternative preferential choice.
*Psychological Review*,*125*, 329–362.PubMedGoogle Scholar - Turner, B. M., & Sederberg, P. B. (2014). A generalized, likelihood-free method for posterior estimation.
*Psychonomic Bulletin & Review*,*21*(2), 227–250.Google Scholar - Turner, B. M., Sederberg, P. B., Brown, S. D., & Steyvers, M. (2013). A method for efficiently sampling from distributions with correlated dimensions.
*Psychological Methods*,*18*(3), 368.PubMedPubMedCentralGoogle Scholar - Tversky, A. (1972). Elimination by aspects: A theory of choice.
*Psychological Review*,*79*, 281–299.Google Scholar - Usher, M., Elhalal, A., & McClelland, J. L. (2008). The neurodynamics of choice, value-based decisions, and preference reversal. In N. Chater, & M. Oaksford (Eds.)
*The probabilistic mind: Prospects for Bayesian cognitive science*(pp. 277–300). Oxford: Oxford University Press.Google Scholar - Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model.
*Psychological Review*,*108*(3), 550.PubMedGoogle Scholar - Usher, M., & McClelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice.
*Psychological Review*,*111*, 757–769.PubMedGoogle Scholar - van Ravenzwaaij, D., Dutilh, G., & Wagenmakers, E.-J. (2012). A diffusion model decomposition of the effects of alcohol on perceptual decision making.
*Psychopharmacology*,*219*(4), 1017–1025. https://doi.org/10.1007/s00213-011-2435-9 PubMedGoogle Scholar - Wollschlager, L. M., & Diederich, A. (2012). The 2n-ary choice tree model for n-alternative preferential choice.
*Frontiers in Cognitive Science*,*3*, 1–11.Google Scholar