# Decision-making under uncertainty: biases and Bayesians

- 1.6k Downloads
- 44 Citations

## Abstract

Animals (including humans) often face circumstances in which the best choice of action is not certain. Environmental cues may be ambiguous, and choices may be risky. This paper reviews the theoretical side of decision-making under uncertainty, particularly with regard to unknown risk (ambiguity). We use simple models to show that, irrespective of pay-offs, whether it is optimal to bias probability estimates depends upon how those estimates have been generated. In particular, if estimates have been calculated in a Bayesian framework with a sensible prior, it is best to use unbiased estimates. We review the extent of evidence for and against viewing animals (including humans) as Bayesian decision-makers. We pay particular attention to the Ellsberg Paradox, a classic result from experimental economics, in which human subjects appear to deviate from optimal decision-making by demonstrating an apparent aversion to ambiguity in a choice between two options with equal expected rewards. The paradox initially seems to be an example where decision-making estimates are biased relative to the Bayesian optimum. We discuss the extent to which the Bayesian paradigm might be applied to the evolution of decision-makers and how the Ellsberg Paradox may, with a deeper understanding, be resolved.

## Keywords

Ambiguity Animal decisions Cognitive bias Ellsberg Paradox Risk Uncertainty## Notes

### Acknowledgments

PCT was supported by ERC grant 250209 Evomech.

## References

- Abrams PA (1994) Should prey overestimate the risk of predation? Am Nat 144:317–328CrossRefGoogle Scholar
- Al-Najjar N, Weinstein J (2009) The ambiguity aversion literature: a critical assessment. Econ Philos 25:249–284CrossRefGoogle Scholar
- Bach DR, Seymour B, Dolan RJ (2009) Neural activity associated with the passive prediction of ambiguity and risk for aversive events. J Neurosci 29:1648–1656PubMedCrossRefGoogle Scholar
- Basile BM, Hampton RR, Suomi SJ, Murray EA (2009) An assessment of memory awareness in tufted capuchin monkeys (
*Cebus apella*). Anim Cogn 12:169–180PubMedCrossRefGoogle Scholar - Bednekoff PA, Houston AI (1994) Dynamic models of mass-dependent predation, risk-sensitive foraging, and premigratory fattening in birds. Ecology 75:1131–1140CrossRefGoogle Scholar
- Binmore K (2009) Making decisions in large worlds. http://www.else.econ.ucl.ac.uk/papers/uploaded/266.pdf
- Blanchette I, Richards A (2003) Anxiety and the interpretation of ambiguous information: beyond the emotion-congruent effect. J Exp Psychol Gen 132:294–309PubMedCrossRefGoogle Scholar
- Bouskila A, Blumstein DT (1992) Rules of thumb for predation hazard assessment - predictions from a dynamic model. Am Nat 139:161–176CrossRefGoogle Scholar
- Burman OHP, Parker R, Paul ES, Mendl M (2008) A spatial judgement task to determine background emotional state in laboratory rats,
*Rattus norvegicus*. Anim Behav 76:801–809CrossRefGoogle Scholar - Call J (2010) Do apes know that they could be wrong? Anim Cogn (in press). doi: 10.1007/s10071-010-0317-x
- Camerer C, Weber M (1992) Recent developments in modeling preferences—uncertainty and ambiguity. J Risk Uncertainty 5:325–370CrossRefGoogle Scholar
- Carlin BP, Louis TA (1996) Bayes and empirical Bayes methods for data analysis. Monographs on statistics and applied probability 69. Chapman & Hall, London, pp 50–52Google Scholar
- Christopoulos GI, Tobler PN, Bossaerts P, Dolan RJ, Schultz W (2009) Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J Neurosci 29:12574–12583PubMedCrossRefGoogle Scholar
- Dayan P, Daw ND (2008) Decision theory, reinforcement learning, and the brain. Cognitive Affective & Behavioral Neuroscience 8(4):429–453CrossRefGoogle Scholar
- Dobbs IM (1991) A Bayesian approach to decision-making under ambiguity. Economica, New Series 58(232):417–440Google Scholar
- Ellsberg D (1961) Risk, ambiguity and the Savage axioms. Q J Econ 75(4):643–669CrossRefGoogle Scholar
- Erichsen JT, Krebs JR, Houston AI (1980) Optimal foraging and cryptic prey. J Anim Ecol 49:271–276CrossRefGoogle Scholar
- Gigerenzer G, Hoffrage U (1995) How to improve Bayesian reasoning without instruction: frequency formats. Psychol Rev 102(4):684–704CrossRefGoogle Scholar
- Gigerenzer G, Todd PM (1999) Simple heuristics that make us smart. Oxford University Press, OxfordGoogle Scholar
- Halevy Y, Feltkamp V (2005) A Bayesian approach to uncertainty aversion. Rev Econ Stud 72(2):449–466CrossRefGoogle Scholar
- Hampton RR, Zivin A, Murray EA (2004) Rhesus monkeys (
*Macaca mulatta*) discriminate between knowing and not knowing and collect information as needed before acting. Anim Cogn 7:239–246PubMedCrossRefGoogle Scholar - Harding EJ, Paul ES, Mendl M (2004) Animal behavior–cognitive bias and affective state. Nature 427:312PubMedCrossRefGoogle Scholar
- Hayden BY, Platt ML (2009) Gambling for Gatorade: risk-sensitive decision making for fluid rewards in humans. Anim Cogn 12:201–207PubMedCrossRefGoogle Scholar
- Hayden BY, Heilbronner SR, Platt ML (2010) Ambiguity aversion in rhesus macaques. Frontiers in Neuroscience 4:1–7CrossRefGoogle Scholar
- Hazen GB (1992) Decisions versus policy: an expected utility resolution of the Ellsberg Paradox. In: Geweke J (ed) Decision making under risk and uncertainty: new models and empirical findings. Kluwer, Dordrecht, pp 25–36Google Scholar
- Hertwig R, Barron G, Weber EU, Erev I (2004) Decisions from experience and the effect of rare events in risky choice. Psychol Sci 15:534–539PubMedCrossRefGoogle Scholar
- Houston AI (2009) Flying in the face of nature. Behav Process 80:295–305CrossRefGoogle Scholar
- Houston AI, McNamara JM (1982) A sequential approach to risk-taking. Anim Behav 30:1260–1261CrossRefGoogle Scholar
- Houston AI, McNamara JM (1985) The choice of two prey types that minimises the probability of starvation. Behav Ecol Sociobiol 17(2):135–141Google Scholar
- Houston AI, McNamara JM (1989) The value of food: effects of open and closed economies. Anim Behav 37:546–562CrossRefGoogle Scholar
- Houston AI, McNamara JM (1999) Models of adaptive behaviour. Cambridge University Press, CambridgeGoogle Scholar
- Houston AI, Krebs JR, Erichsen JT (1980) Optimal prey choice and discrimination time in the great tit
*Parus major*. Behav Ecol Sociobiol 6:169–175CrossRefGoogle Scholar - Houston AI, Kacelnik A, McNamara JM (1982) Some learning rules for acquiring information. In: McFarland DJ (ed) Functional Ontogeny. Pitman, London, pp 140–191Google Scholar
- Houston AI, McNamara JM, Steer MD (2007) Violations of transitivity under fitness maximization. Biol Lett 3:365–367PubMedCrossRefGoogle Scholar
- Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF (2005) Neural systems responding to degrees of uncertainty in human decision-making. Science 310(5754):1680–1683PubMedCrossRefGoogle Scholar
- Jackson RR, Pollard SD, Li D, Fijn N (2002) Interpolation variation in the risk-related decisions of
*Porta labiata*, an araneophagic jumping spider (Araneae, Salticidae), during predatory sequences with spitting spiders. Anim Cogn 5:215–223PubMedCrossRefGoogle Scholar - Kacelnik A, Bateson M (1996) Risky theories - the effects of variance on foraging decisions. Am Zool 36:402–434Google Scholar
- Kahneman D, Tversky A (1972) Subjective probability: a judgement of representativeness. Cogn Psychol 3:430–454CrossRefGoogle Scholar
- Kawamori A, Matsushima T (2010) Subjective value of risky foods for individual domestic chicks: a hierarchical Bayesian model. Anim Cogn 13:431–441PubMedCrossRefGoogle Scholar
- Klaassen RHG, Nolet BA, van Gils JA, Bauer S (2006) Optimal movement between patches under incomplete information about the spatial distribution of food items. Theor Popul Biol 70:452–463PubMedCrossRefGoogle Scholar
- Knight FH (1921) Risk. uncertainty and profit. Houghton Mifflin Company, New York, pp 43–46Google Scholar
- Krebs JR, Kacelnik A, Taylor P (1978) Test of optimal sampling by foraging great tits. Nature 275:27–31CrossRefGoogle Scholar
- Lagorio CH, Hackenberg TD (2010) Risky choice in pigeons and humans: a cross-species comparison. J Exp Anal Behav 93:27–44PubMedCrossRefGoogle Scholar
- Lange A, Dukas R (2009) Bayesian approximations and extensions: optimal decisions for small brains and possibly big ones too. J Theor Biol 259(3):503–516PubMedCrossRefGoogle Scholar
- Langen TA (1999) How western scrub-jays (
*Alphelocoma californica*) select a nut: effects of the number of options, variation in nut size, and social competition among foragers. Anim Cogn 2:223–233CrossRefGoogle Scholar - Loewenstein G, Lerner JS (2003) The role of affect in decision making. In: Davidson RJ, Scherer KR, Goldsmith HH (eds) Handbook of Affective Sciences. Oxford University Press, Oxford, pp 619–642Google Scholar
- Loewenstein GF, Weber EU, Hsee CK, Welch N (2001) Risk as feelings. Psychol Bull 127(2):267–286PubMedCrossRefGoogle Scholar
- McNamara JM, Dall SRX (2010) Information is a fitness enhancing resource. Oikos 119:231–236CrossRefGoogle Scholar
- McNamara JM, Houston AI (1980) The application of statistical decision theory to animal behaviour. J Theor Biol 85:673–690PubMedCrossRefGoogle Scholar
- McNamara JM, Houston AI (1985) A simple model of information use in the exploitation of patchily distributed food. Anim Behav 33:553–560CrossRefGoogle Scholar
- McNamara JM, Houston AI (1987) A general framework for understanding the effects of variability and interruptions on foraging behaviour. Acta Biotheor 36:3–22PubMedCrossRefGoogle Scholar
- McNamara JM, Houston AI (1992) Risk-sensitive foraging—a review of the theory. Bull Math Biol 54:355–378Google Scholar
- McNamara JM, Houston AI (1997) Currencies for foraging based on energetic gain. Am Nat 150:603–617PubMedCrossRefGoogle Scholar
- McNamara JM, Houston AI (2009) Integrating function and mechanism. Trends Ecol Evol 24(12):670–675PubMedCrossRefGoogle Scholar
- McNamara JM, Merad S, Houston AI (1991) A model of risk-sensitive foraging for a reproducing animal. Anim Behav 41:787–792CrossRefGoogle Scholar
- McNamara JM, Green RF, Olsson O (2006) Bayes’ theorem and its applications in animal behaviour. Oikos 112:243–251CrossRefGoogle Scholar
- Mendl M, Burman OHP, Parker RMA, Paul ES (2009) Cognitive bias as an indicator of animal emotion and welfare: emerging evidence and underlying mechanisms. Appl Anim Behav Sci 118:161–181CrossRefGoogle Scholar
- Mendl M, Burman OHP, Paul ES (2010) An integrative and functional framework for the study of animal emotion and mood. Proc R Soc B 277:2895–2904PubMedCrossRefGoogle Scholar
- Mogg K, Bradbury KE, Bradley BP (2006) Interpretation of ambiguous information in clinical depression. Behav Res Ther 44:1411–1419PubMedCrossRefGoogle Scholar
- Naug D, Arathi HS (2007) Sampling and decision rules used by honey bees in a foraging arena. Anim Cogn 10:117–124PubMedCrossRefGoogle Scholar
- Nettle D (2004) Adaptive illusions: optimism, control and human rationality. In: Evans D, Cruse P (eds) Emotion, evolution and rationality. Oxford University Press, Oxford, pp 193–208Google Scholar
- Oaksford M, Chater N (2009) Precis of Bayesian rationality: the probabilistic approach to human reasoning. Behav Brain Sci 32:69–120PubMedCrossRefGoogle Scholar
- Olsson O, Holmgren NMA (1998) The survival-rate-maximizing policy for Bayesian foragers: wait for good news. Behav Ecol 9:345–353CrossRefGoogle Scholar
- Olsson O, Wiktander U, Holmgren NMA, Nilsson S (1999) Gaining ecological information about Bayesian foragers through their behaviour. II. A field test with woodpeckers. Oikos 87:264–276Google Scholar
- Ozdenoren E, Peck J (2008) Ambiguity aversion, games against nature, and dynamic consistency. Game Econ Behav 62:106–115CrossRefGoogle Scholar
- Parker GA, Maynard Smith J (1990) Optimality theory in evolutionary biology. Nature 348:27–33CrossRefGoogle Scholar
- Paul ES, Harding EJ, Mendl M (2005) Measuring emotional processes in animals: the utility of a cognitive approach. Neurosci Biobehav 29:469–491CrossRefGoogle Scholar
- Platt ML, Huettel SA (2008) Risky business: the neuroeconomics of decision making under uncertainty. Nature Neurosci 11:398–403PubMedCrossRefGoogle Scholar
- Pulford BD (2009) Is luck on my side? Optimism, pessimism, and ambiguity aversion. Q J Exp Psychol 62:1079–1087CrossRefGoogle Scholar
- Raiffa H (1961) Risk, ambiguity, and the Savage axioms: comment. Q J Econ 75:690–694CrossRefGoogle Scholar
- Raiffa H, Schlaifer R (1961) Applied statistical decision theory. MIT Press, Cambridge, MassachusettsGoogle Scholar
- Real L, Caraco T (1986) Risk and foraging in stochastic environments. Annu Rev Ecol Syst 17:371–390CrossRefGoogle Scholar
- Rilling JK, Sanfey AG, Aronson JA, Nystrom LE, Cohen JD (2004) The neural correlates of theory of mind within interpersonal interactions. NeuroImage 22:1694–1703PubMedCrossRefGoogle Scholar
- Roberts HV (1963) Risk, ambiguity, and the Savage axioms: comment. Q J Econ 77(2):327–336CrossRefGoogle Scholar
- Rode C, Cosmides L, Hell W, Tooby J (1999) When and why do people avoid unknown probabilities in decisions under uncertainty? Testing some predictions from optimal foraging theory. Cognition 72(3):269–304PubMedCrossRefGoogle Scholar
- Rosati AG, Hare B (2011) Chimpanzees and bonobos distinguish between risk and ambiguity. Biol Lett 7(1):15–18PubMedCrossRefGoogle Scholar
- Schultz W, Preuschoff K, Camerer C, Hsu M, Fiorillo CD, Tobler PN, Bossaerts P (2008) Explicit neural signals reflecting reward uncertainty. Phil Trans R Soc B 363:3801–3811PubMedCrossRefGoogle Scholar
- Smith JD, Redford JS, Beran MJ, Washburn DA (2010) Rhesus monkeys (
*Macaca mulatta*) adaptively monitor uncertainty while multi-tasking. Anim Cogn 13:93–101PubMedCrossRefGoogle Scholar - Stephens DW, Krebs JR (1986) Foraging theory. Princeton University Press, PrincetonGoogle Scholar
- Trautmann ST, Vieder FM, Wakker PP (2008) Causes of ambiguity aversion: known versus unknown preferences. J Risk Uncertainty 36:225–243CrossRefGoogle Scholar
- Trimmer PC, Houston AI, Marshall JAR, Bogacz R, Paul ES, Mendl MT, McNamara JM (2008) Mammalian choices: combining fast-but-inaccurate and slow-but-accurate decision-making systems. Proc R Soc B 275:2353–2361PubMedCrossRefGoogle Scholar
- Uehara T, Yokomizo H, Iwasa Y (2005) Mate-choice copying as Bayesian decision making. Am Nat 165(3):403–410PubMedCrossRefGoogle Scholar
- Valone TJ (2006) Are animals capable of Bayesian updating? An empirical review. Oikos 112:252–259CrossRefGoogle Scholar
- van Gils JA, Schenk IW, Bos O, Piersma T (2003) Incompletely informed shorebirds that face a digestive constraint maximize net energy gain when exploiting patches. Am Nat 161:777–793PubMedCrossRefGoogle Scholar
- Viscusi WK, Magat WA (1992) Bayesian decisions with ambiguous belief aversion. J Risk Uncertainty 5(4):371–387CrossRefGoogle Scholar
- Waite TA (2008) Preference for oddity: uniqueness heuristic or hierarchical choice process? Anim Cogn 11:707–713PubMedCrossRefGoogle Scholar
- Weber EU, Shafir S, Blais AR (2004) Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol Rev 111:430–445PubMedCrossRefGoogle Scholar
- Welton NJ, McNamara JM, Houston AI (2003) Assessing predation risk: optimal behaviour and rules of thumb. Theor Popul Biol 64:417–430PubMedCrossRefGoogle Scholar
- Zhu L, Gigerenzer G (2006) Children can solve Bayesian problems: the role of representation in mental computation. Cognition 98:287–308PubMedCrossRefGoogle Scholar