Advertisement

Psychonomic Bulletin & Review

, Volume 25, Issue 6, pp 2024–2036 | Cite as

Uncertainty processing in bees exposed to free choices: Lessons from vertebrates

  • Patrick AnselmeEmail author
Theoretical Review

Abstract

Bees tend to avoid or to show indifference to uncertain (“risky”) relative to certain (“safe”) food rewards, whether in nectar volume or in nectar concentration. The unattractiveness of uncertain food rewards is also sometimes independent of the energy budget of bees. This pattern of responses seems to differ from that observed in mammals and birds, which may exhibit a strong preference for the uncertainty over the certainty of food delivery on a given trial in dual-choice tasks. Upon analysis of the conditions that determine preference and aversion for uncertain food rewards in “higher” vertebrates, I attempt to demonstrate that bees react to uncertainty in a similar way. It is argued that, because of their social organization and of the type of resources they seek, bees are essentially exposed to situations in which “higher” vertebrates find reward uncertainty unattractive as well. The nature of their representation of food distribution is discussed, and it is suggested that scout bees may differ from recruits with respect to uncertainty processing.

Keywords

Bee Preference Food seeking Risk sensitivity 

Sensitivity to reward uncertainty (often called “risk”) has been studied in a variety of animal species in vertebrates (especially rats, pigeons, starlings, and humans) and in invertebrates (bees and wasps). Typically, animals are given a choice between an option in which a conditioned stimulus (CS) predicts constant conditions (certainty) of reward delivery from trial to trial and an option in which the CS predicts variable conditions (uncertainty) of reward delivery from trial to trial (for a review, see Kacelnik & Bateson, 1996). In vertebrates, this instrumental procedure often induces a preference for the uncertain option when uncertainty is about the number of responses to produce (ratio schedules) or about the time between response and reward (delay schedules). A strong preference for uncertainty is also shown with probabilistic schedules, in which one option is occasionally rewarded and the other consistently rewarded. In contrast, a preference for the certain option or indifference between the two options is frequently observed when uncertainty is about the amount of reward (amount schedules), where uncertainty consists of receiving more or less reward than with the certain option. Here, the word uncertainty refers to the inability to predict what an outcome will be (reward or nonreward), irrespective of whether the animal knows the probability of reward or not—in doing this, I put together the concepts of risk and uncertainty such as initially defined by Knight (1921; for details, see Anselme, 2015a).

In invertebrates, the behaviour of bees and wasps has been examined in similar experimental conditions and seems to display a more homogenous pattern than in the case of vertebrates: those insects often develop indifference to or come to avoid reward uncertainty. The present article reviews the scientific literature on the sensitivity to reward uncertainty in bees and is an attempt to explain bees’ behaviour in the light of what is known about mammals and birds. Most behavioural similarities existing between these two zoological groups are likely to result from convergent evolution (homoplasy) rather than from phenotypic traits present in a common ancestor (homology), which ‘lived about half billion years ago and must hardly have had any brain at all’ (Bitterman, 1996, p. 1900). But bees have demonstrated such high-level capabilities (with respect to navigation and search, visual recognition, communication, etc.) compared to many vertebrate species that it is useful to analyse their behavioural strategies by means of the same theoretical tools. For example, agitated honeybees show a ‘pessimistic’ bias when they are exposed to an ambiguous CS (odour) that predicts both reward and punishment (Bateson, Desire, Gartside, & Wright, 2011), a phenomenon possibly related to the ‘optimistic’ bias exhibited by rats, pigs, and starlings reared in an enriched, stress-free environment (Brydges, Leach, Nicol, Wright, & Bateson, 2011; Douglas, Bateson, Walsh, Bédué, & Edwards, 2012; Matheson, Asher, & Bateson, 2008; for an evolutionary approach to cognitive biases, see D. D. P. Johnson & Fowler, 2011; Marshall, Trimmer, Houston, & McNamara, 2013; von Hippel & Trivers, 2011). Also, because some brain structures and neurotransmitters appear to control the same processes across phyla (e.g., Adamo & Baker, 2011; Alcaro, Panksepp, & Huber, 2011; Barron, Søvik, & Cornish, 2010; Perry & Barron, 2013; Strausfeld & Hirth, 2013), invertebrates can exhibit comparable reactions—such as fear and stress responses—in many situations (Elwood, Barr, & Patterson, 2009; Ottaviani & Franceschi, 1996; Stefano et al. 2002). In particular, Strausfeld and Hirth (2013) found homology between central complex (insects) and basal ganglia, which are known to play a determining role in reward and uncertainty processing in vertebrates (e.g., Berridge, 2007; Fiorillo, Tobler, & Schultz, 2003).

The existence of highly conserved behavioural reactions in different taxa—because all animals have to deal with similar constraints, such as finding food and avoiding competitors—suggests that the basic processes controlling decision-making under uncertainty in bees may be comparable to those found in birds and mammals. In this article, it is argued that, given the type of resources they seek and the way they seek it, bees rarely experience a delay-based or a ratio-based variability; they are almost exclusively exposed to uncertain amounts of food. In other words, they essentially experience situations in which mammals and birds find reward uncertainty unattractive as well. It is also suggested that evolution may have shaped the brain of recruits with a foraging strategy that differs from that of scouts relative to their sensitivity to uncertain delays and ratios of responses.

The effects of uncertain rewards in pollinator insects

In this section, I present and discuss a number of experimental situations in which bees and wasps were given a choice between a certain/constant and an uncertain/variable food source. In addition to details given in the text, a summary of the main results is reported in Table 1.
Table 1

Choice responses of bees to reward uncertainty

Species

Procedure

Response to uncertainty

Reference

B. sandersoni

100% (2 μl) vs. 33% (6 μl)

Aversion

Real (1981)

B. edwardsii

100% (0.1 μl) vs. 10% (1 μl)

Aversion

Waddington et al. (1981)

B. pennsylvanicus

100% (2 μl) ≥ Variable vol.

100% (2 μl) < Variable vol.

Aversion

Attraction

Real et al. (1982)

A. mellifera

100% (20 μl) vs. 33% (20 μl)

Indifference

Wells & Wells (1983)

B. occidentalis

100% (2 μl) vs. 50% (1 or 5 μl)

Attraction

Cartar & Dill (1990)

B. melanopygus

B. mixtus

B. sitkensis

0.07 ± 0.08 μl vs. 0.42 ± 1.06 μl (natural conditions)

Attraction

Cartar (1991)

A. mellifera

100% (1 μl, 20% sucrose) vs.

50% (1 μl, 10% sucrose) or

50% (1 μl, 30% sucrose)

Aversion (45%)

Attraction (37.5%)

Indifference (17.5%)

Banschbach & Waddington (1994)

B. fervidus

100% (1 μl, 20% sucrose) vs.

50% (1 μl, 10% sucrose) or

50% (1 μl, 30% sucrose)

Indifference

Waddington (1995)

X. micans

100% (2 μl) vs. 50% (1 or 3 μl)

Indifference

Perez & Waddington (1996)

A. m. ligustica

A. m. caucasica

A. m. scutellata

100% (5 μl) vs. 33% (5 μl)

Aversion

Cakmak et al. (1999)

A. mellifera

75% (0.4 μl) vs. 25% (0 or 1.6 μl)

75% (0.4 μl) vs. 25% (0 or 2.4 μl)

75% (0.6 μl) vs. 25% (0.2 or 1.8 μl)

Aversion

Aversion

Indifference

Shafir et al. (1999)

A. mellifera

Fixed vs. variable time visits

Indifference

Fülöp & Menzel (2000)

A. mellifera

100% (5 μl) vs. 16.7% (30 μl)

100% (5 μl) vs. 33.3% (30 μl)

100% (10 μl, 15% sucrose) vs.

50% (10 μl, 60% sucrose)

Aversion

Aversion

Attraction

Shapiro et al. (2001)

B. flavifrons

100% (3 μl) vs. 50% (6 μl)

Aversion

Biernaskie et al. (2002)

A. mellifera

* workers

* drones

67% (0.4 μl) vs. 33% (1.2 μl)

Aversion

Indifference

Shafir et al. (2005)

A. mellifera

Several probabilities of low

and high variable rewards

Aversion

Indifference

Drezner-Levy & Shafir (2007)

A. m. caucasica

A. m. carnica

A. m. syriaca

100% (2 μl) vs. 33% (6 μl)

Indifference

Cakmak et al. (2010)

A. mellifera

100% (0.2 μl) vs. 50% (0.4 μl)

Aversion (if increasing budget)

Attraction (if decreasing budget)

Indifference (if constant budget)

Mayack & Naug (2011)

Note. Most studies report aversion or indifference to variability as opposed to constancy, although a preference for variability is shown under specific conditions. The studies are reported by chronological order. More details in the text

Main findings reporting uncertainty avoidance or indifference

Perhaps the first experiments on this topic were conducted by Waddington, Allen, and Heinrich (1981). They provided a constant volume of 0.1 μl of nectar in one type of artificial flowers and 1 μl of nectar with a 10% probability in another type, and showed that bumblebees (Bombus edwardsii) strongly chose the constant type. It is reasonable to think that the lack of uncertainty that a 10% probability represents (i.e., the bees were almost certain of finding no food in those flowers) may explain their avoidance of this option. It is also possible that the difference between the constant and the uncertain gains was too small for the two options to be distinguished by bees, although honeybees (Apis mellifera) can discriminate between 0 and 0.4 μl of a sucrose solution (Shafir, Menda, & Smith, 2005). However, bumblebees (B. sandersoni) were given artificial flowers containing 2 μl of nectar and others containing 6 μl of nectar with a 33% probability, and they also showed a strong preference for the certain rather than uncertain flowers (Real, 1981). Diluted honey was used in that study. Since bees can detect odours from a distance, it is, however, possible that they avoided the empty flowers without attempting to assess variance (Fülöp & Menzel, 2000). Some subsequent experimental works suppressed possible shortcomings but failed to reverse the preference of bees (Cakmak et al., 2010; Drezner-Levy & Shafir, 2007; Fischer, Couvillon, & Bitterman, 1993; Fülöp & Menzel, 2000; Perez & Waddington, 1996; Shafir et al., 2005; Shafir, Wiegmann, Smith, & Real, 1999; Shapiro, Couvillon, & Bitterman, 2001; Waddington, 1995; Wells & Wells, 1983). In nature, nectar volume per flower is more variable than nectar concentration (Rathcke, 1992), but these studies show that pollinators dislike variability whether in volume or in concentration.

Profitability of a food source is important for bees. When the colour of two flower types was equally attractive (here, blue and white), the Africanized honeybee subspecies, A. m. scutellata, favoured the flower type that offered a reward more frequently, irrespective of whether its harvest rate was equivalent or higher than that of the other flower type (Cakmak et al., 1999). In contrast, when a colour preference existed between two flower types (here, blue or white was preferred to yellow), a change in flower type only occurred when the higher reward frequency was linked to a higher harvest rate. Thus, these bees are attracted by CSs that reliably predict reward delivery. But this pattern of responses may be different in other subspecies, since A. m. ligustica and A. m. caucasica were indifferent to reward frequency. (Avoidance and indifference are distinct reactions and should logically be treated separately. However, in this article, I am specifically interested in the conditions that may generate a clear-cut preference for uncertainty. Thus, avoidance and indifference are considered together.)

Most studies of the sensitivity to reward uncertainty were conducted using eusocial bees, not solitary individuals. In order to eliminate complications related to sociality (colony size, food sharing, etc.), Perez and Waddington (1996) tested a solitary bee (Xylocopa micans)—which does not store nectar and, hence, runs the risk of energy shortfall every day. Their results were similar to those found with eusocial bees: X. micans was indifferent to uncertainty, regardless of whether variability was in nectar volume or in nectar concentration. Also, the bee’s energy budget did not affect foraging behaviour. Of course, it is not possible to generalise how solitary bees respond to uncertainty from this single study. Solitary bees differ from eusocial bees in many respects, especially because they forage for themselves and are reproductive individuals. Also, solitary bees do not seem to be able to cover long foraging distances (Gathmann & Tscharntke, 2002; Zurbuchen et al., 2010), suggesting that they have to settle for the local availability of resources, irrespective of their abundance. Other differences may exist, such as a lower learning ability in the solitary bee X. virginica compared to the social bee B. bimaculatus (Dukas & Real, 1991). But it is unknown whether such differences may influence their reactivity to uncertain food.

According to Fülöp and Menzel (2000), the lack of sensitivity to variance in bees results from ‘an adaptation to cope with the unreliability of scattered food sources such as flowers’ (p. 665). They pointed out that flowers have differential productivity and are often exploited by other foraging animals so that it is difficult for bees to predict the amount of nectar a flower may contain at a given time. On account of this, bees should only consider average profitability and exhibit indifference to variability (see also Perez & Waddington, 1996). We shall see further that this general argument does not explain why, in other situations of equal profitability, mammals and birds may show a strong preference for the uncertain option.

Main findings reporting uncertainty attraction

A few studies seem to demonstrate uncertainty attraction under specific conditions, although their results were not always replicable. Real, Ott, and Silverfine (1982) showed that B. pennsylvanicus accept a larger variance only if nectar volume is increased in the uncertain flowers relative to the constant flowers. Shapiro et al. (2001) failed to reproduce this result with honeybees. However, they obtained uncertainty attraction by increasing the concentration of nectar in the uncertain flowers. These two results indicate that bees can positively respond to uncertainty in volume and concentration, provided that the uncertain option offers more food than the constant one, on average.

Mayack and Naug (2011) provided evidence that honeybees do not react to an absolute energy budget but shift from uncertainty aversion to uncertainty attraction when their energy budget is decreased, and vice versa. Although these authors may have ‘measured survivorship (i.e. a correlate of fitness) rather than behavioural preferences’ (Kacelnik & Mouden, 2013, p. 1119), this result suggests that it is worth expecting a higher reward rate from variable flowers if constant ones fail to provide enough energy for individual bees. Perhaps in the same vein, Cartar and Dill (1990) found that attraction for uncertain nectar volumes could exist at the colony rather than the individual level. When the colony reserves in nectar are high, B. occidentalis avoid uncertain flowers, but when the colony reserves are low, they prefer to forage on uncertain flowers (for the same effect in natural conditions and with other species, see Cartar, 1991). Banschbach and Waddington (1994) failed to replicate this result, but they tested the bees by means of another method: honeybees had to choose between a constant 20% sucrose solution and an uncertain 10%–30% sucrose solution. Both options were selected with the same frequency, whether the colony honey stores were increased or decreased. All the bees but one were consistent in their preference between the foraging bouts. Interestingly, 37.5% of them exhibited uncertainty attraction (45% expressed aversion and 17.5% were indifferent). Such a diversity of responses among bees may simply reflect individual variations in responding, as found in vertebrates exposed to a similar task. Waddington (1995) obtained a great majority of indifferent bumblebees (B. fervidus) with the same task, suggesting that the proportion of responses (attraction, aversion, and indifference) may be species specific. Compared to the two studies initially mentioned in this section (Real et al., 1982; Shapiro et al. 2001), the unattractiveness of uncertainty reported here (Banschbach & Waddington, 1994) could occur because, on average, uncertainty is not associated with more gains than the constant option.

Can bees detect variability?

Given the number of studies showing indifference to variability in bees, would it be possible that they do not detect it at all? This hypothesis is hardly plausible because, in a number of situations, bees respond to a change in reward contingency. Honeybees exhibit successive negative contrast, reducing their consummatory behaviour when they receive tap water (which is not rewarding) after training with large drops of a 30% sucrose solution (Couvillon & Bitterman, 1984). A solution whose sucrose concentration is reduced from 50% to 20% also transiently disrupts consumption in B. impatiens (Wiegmann, Wiegmann, & Waldron, 2003). This behavioural reaction to a sudden reduction in reward rate is also observed in mammals, but not in other vertebrates (fish, toads, turtles, and pigeons), and is typically interpreted as a frustration effect (Papini, 2006). Frustration occurs when less than the expected reward is obtained (Amsel, 1958). It is reasonable to think that convergent evolution produced similar psychological states in response to the absence of an expected reward in rats and bees. It is also important to note that honeybees trained under partial reinforcement exhibit a higher performance in acquisition (partial reinforcement acquisition effect, PRAE) and a greater resistance to extinction (partial reinforcement extinction effect, PREE) compared with individuals trained under continuous reinforcement (Ishida, Couvillon, & Bitterman, 1992). The PRAE and the PREE have commonly been found in mammals and birds (e.g., Anselme, Robinson, & Berridge, 2013; Gibbon, Farrell, Locurto, Duncan & Terrace, 1980; Gottlieb, 2004; Papini, Thomas, & McVicar, 2002; Pearce, Readhead, & Aydin, 1997; Rescorla, 1999), but not in “lower” vertebrates (Papini, 2006). Interestingly, a PREE does not occur in honeybees when rewarded trials are never preceded by nonrewarded trials during partial reinforcement (Ammon, Abramson, & Bitterman, 1986), a result also shown in the albino rat (Spivey & Hess, 1968). Taken together, these results strongly suggest that bees can distinguish rewarded from nonrewarded (or less rewarded) trials.

According to Shafir (2000), the sensitivity of nectarivores to uncertain amounts is proportional to the coefficient of variation of a food distribution (SD/mean), but is hardly influenced by variance alone (SD2). The use of variance as a criterion of variability in many studies would explain why they often report indifference to uncertainty in bees (for a review, see Kacelnik & Bateson, 1996). Indeed, a low coefficient of variation may mean that animals are unable to distinguish the uncertain option from the constant option because the relationship between the objective measures and their subjective perception is approximately linear (Waddington & Kirchner, 1992). In other words, if bees and wasps were tested with larger coefficients of variation, animals that appear indifferent might become sensitive to uncertainty. However, current data suggest that nectarivores develop stronger aversion rather than attraction when the coefficient of variation increases (Shafir, 2000). In bees under a positive energy budget, the aversion for uncertainty increases in proportion to the coefficient of variation and with the inclusion of a potential zero-reward option (Drezner-Levy & Shafir, 2007). In contrast, mammals and birds exposed to low coefficients of variation may show a preference for uncertainty, even under a positive energy budget (see Kacelnik & Bateson, 1996), indicating that bees and wasps are perhaps not prepared to accept it so easily. For example, pigeons may prefer a variable-ratio to a fixed-ratio of response for the same quantity of food, even if variability is disadvantageous in terms of energy cost (e.g. Ahearn, Hineline, & David, 1992; Field, Tonneau, Ahearn, & Hineline, 1996; Sherman & Thomas, 1968). In contrast, Waddington and Gottlieb (1990) provided some evidence that A. mellifera prefer flowers associated with lower costs when gains are equal.

Interestingly, flowering plants seem to exploit the aversion of pollinators (bees and hummingbirds) for uncertain nectar volumes. For plants with inflorescences of many flowers, the transfer of pollen to a stigma of a different flower on the same plant (geitonogamy) limits pollen export and leads to inbreeding depression due to selfing. It is therefore profitable for those plants that pollinators reduce the number of flower visits per inflorescence. Biernaskie, Cartar, and Hurly (2002) showed that B. flavifrons and the wild hummingbird, Selasphorus rufus, exhibit fewer flower visits on artificial inflorescences when nectar volume is variable than when it is constant. They concluded that flowering plants would have a variable production in nectar in order to generate fewer flower visits on inflorescences, and hence avoid the costs that geitonogamy may represent.

Formally, the behaviour of many bee species suggests that those insects evaluate reward-related parameters using a concave-down—increasing, but decelerating—function (Harder & Real, 1987; Shafir, Bechar, & Weber, 2003). This is to say that the subjective value associated with a constant distribution will be higher than the subjective perception of an uncertain alternative with an equivalent mean (a mathematical property called Jensen’s inequality), leading bees to be unattracted by the uncertain food source (see Fig. 1). Such a perceptual bias generates a greater expected rate of net energy intake with a constant reward, even though the average profitability of the uncertain reward is identical. Of note, a concave-down function will predict indifference when the mean reward is elevated since the relationship between the reward amounts and their associated subjective values becomes quasilinear. Thus, uncertainty aversion or indifference may depend on the amount of food provided. The problem of the representation of food distribution is crucial for the purpose of the present article and will be more extensively discussed further.
Fig. 1

Concave-down function. This function is typically used to show how mammals and birds represent food amounts. Bees seem to represent food distribution by means of the same function, which shows that a higher subjective value is attributed to fixed rather than variable amounts: V(a) > (V(a–n) + V(a+n))/2. Indifference may result from a gentler slope of the curve or from some changes in the values of each option

Functional explanations

Overall, the strongest evidence for uncertainty attraction in bees comes from studies in which the volume or the concentration of nectar was significantly increased in the uncertain flowers or from studies in which survival was imperilled. These results are compatible with optimal foraging theory (Pyke, 1984; Stephens & Krebs, 1986), which predicts that animals tend to maximize reward rate intake and to prefer variable environmental conditions when there is a risk of starvation (e.g., Barnard & Brown, 1985b; Caraco et al., 1990). Although interspecific differences are likely among bees, the available data indicate that bees are optimal foragers. For example, if B. pennsylvanicus experience moderately to highly rewarding patches, they tend to ignore variability, tracking the mean reward value only (Real et al., 1982).

A gold standard of optimal foraging theory is the energy budget rule, which predicts that organisms under a ‘positive’ (food nondeprived) budget should exhibit a preference for certainty, while organisms under a ‘negative’ (food deprived) budget should exhibit a preference for uncertainty. Although this rule may be difficult to test in individual bees, its application to the level of honey stores in a colony remains controversial. It is however noticeable that a reliable interaction between honey store levels and preference was found in four different species of bees (Cartar, 1991; Cartar & Dill, 1990). In vertebrates, a large proportion of studies fails to confirm the energy budget rule. According to Kacelnik and Mouden (2013), less than 24% of the studies published since 1980 found support for the energy budget rule, suggesting that the rule can only poorly predict behaviour. However, if only applied to reward amounts, the rule is relatively good to predict the influence of variable versus constant options on choice (Kacelnik & Bateson, 1996). Thus, given that bees under temperate climate forage during the good seasons and that their choices conform to the predictions of the energy budget rule, bees seem to forage as if they were predominantly exposed to variable food amounts. They consider variability only if it is associated with a higher profitability (in volume or in concentration) or with a potentially higher profitability (in the case of starved individuals and depleted honey store).

Comparing bees with mammals and birds

Artificial choice schedules

In vertebrates, a reward obtained after an uncertain delay whose mean is the value of a constant delay is preferred to—associated with a higher average subjective value than—the same reward obtained after the constant delay (Kacelnik & Bateson, 1996). The reason is that the variable option occasionally allows individuals to obtain food more quickly than the constant option, confirming the temporal discounting hypothesis (see Fig. 2): The subjective value of a reward decreases hyperbolically when the time between a response and reward delivery increases (Cardinal, 2006; Estle, Green, Myerson, & Holt, 2006; Mazur, 1987). Nevertheless, finding delay values for which variability is not advantageous, and hence is not preferred to constancy, remains possible (Anselme, Edeş, Tabrik, & Güntürkün, 2018). The same reasoning also applies to random-ratio schedules since the opportunity to receive a reward after a small number of responses is equivalent to its possible delivery after a short delay (e.g., Day, Jones, Wigthtman, & Carelli, 2010; P. S. Johnson, Madden, Brewer, Pinkston, J& Fowler, 2011). This interpretation is in accordance with the evolutionary explanation of this phenomenon (e.g., McNamara & Houston, 1987, 1992). To my knowledge, bees have never been tested in choices involving variable and constant delays, even though there is evidence that they have some perception of the time elapsed (Boisvert & Sherry, 2006). Such experiments would be of paramount importance for comparative psychology.
Fig. 2

Hyperbolic-decay function. This function is typically used to show the sensitivity of mammals and birds to delays for food. A higher subjective value is attributed to variable rather than fixed delays: V(d) < (V(d–t) + V(d+t))/2. Indifference may result from a gentler slope of the curve or from some changes in the values of each option

As shown earlier, bees are systematically tested in experiments involving probabilistic choices (e.g., 2 μl of sucrose with a 100% chance vs. 0 or 4 μl of sucrose with a 50% chance) and often find reward uncertainty unattractive. Do they react differently than vertebrates? At first sight, the answer to this question is yes, given the vast number of studies showing that mammals and birds may strongly prefer probabilistic over certain outcomes (Belke & Spetch 1994; Dunn & Spetch 1990; Gipson, Alessandri, Miller, & Zentall, 2009; Laude, Stagner, & Zentall, 2014; Mazur, 1991; Pattison, Laude, & Zentall, 2013; Spetch, Belke, Barnet, Dunn, & Pierce, 1990; Stagner & Zentall, 2010; Vasconcelos, Monteiro, & Kacelnik, 2015). But it must be realized that, in these experiments, a preference for uncertainty is obtained because the uncertain option is associated with reliable CSs (e.g., if red -> 0% chance of reward, if green -> 100% chance of reward) and the surer/certain option with unreliable CSs (e.g., if blue or yellow -> 100% chance of reward). In the reverse case, when uncertainty is predicted by unreliable CSs and certainty by reliable CSs, the preference becomes in favour of certainty (for an excellent review, see McDevitt, Dunn, Spetch, & Ludvig, 2016). This means that animals are tracking the reliability of CSs, not uncertainty or certainty in themselves (e.g., Smith & Zentall, 2016). Thus, it is important to consider how bees are exposed to uncertainty/certainty in psychology experiments: one flower colour unreliably predicts reward or no reward (uncertainty), while another flower colour reliably predicts reward (certainty). These experimental conditions reflect a situation in which pigeons would strongly prefer the certain over the uncertain option, like bees do; the experiments carried out with bees have simply been unable to reveal the exact conditions in which bees should, like “higher” vertebrates, exhibit massive preference for reward uncertainty.

In vertebrates, uncertain amounts—where all trials are immediately rewarded—are almost systematically avoided (Kacelnik & Bateson, 1996). Psychologically, this phenomenon is assumed to result from differential subjective valuation of gains and losses; a higher subjective value is assigned to the loss of a quantity than to the gain of the same quantity. In humans, it was demonstrated that the subjective compensation for a loss of x approximately requires a gain of 2.5x (Kahneman & Tversky, 1979). Consider the case of an individual which receives two food pellets for sure in the constant option and one or three pellets on a random basis (mean of two pellets) in the uncertain option, and assume that the delivery of two pellets is associated with a subjective value V. So, when the animal receives three pellets (gain of one pellet relative to the mean), the subjective value of that outcome is V + V/2 = 3V/2. But when the animal receives only one pellet (loss of one pellet relative to the mean), the subjective value of the outcome is V − 2.5V/2 = −0.5V/2. Thus, the average subjective value for the uncertain option is 3V/2 + (−0.5V/2)]/2 = 0.625V, which is lower than the value V for the constant option. A concave-down function is used to represent this result, which suggests that variability in amounts should not be preferred to a constant amount equal to their mean. The avoidance of uncertain amounts is quite sensitive to energy budget. After Kacelnik and Bateson (1996), if given a choice between a constant and an uncertain food amount, animals under a positive budget exhibit a preference for the uncertain option in only 14% of studies. In contrast, under a negative budget, 40% of studies report a shift in preference towards the uncertain option. As noted earlier, this relative dependence of uncertainty-related choices on energy budget also occurs in bees, and may lead to the conclusion that bees are mainly reacting to variable food amounts in their environment. In the next section, we shall describe some elements that might support this hypothesis.

Before that, think a little bit about the reason why losses carry more weight than gains. This phenomenon is not specific to vertebrates; bees also overestimate losses relative to gains. Waddington (1985) studied how honeybees perceive caloric costs and gains while foraging on artificial flowers. The rate of reversals in direction while performing a round dance was used to quantify the perception of bees. It appeared that caloric cost (induced by flight and handling) regression coefficients were larger in absolute magnitude than caloric gain (resulting from intake per floral visit) coefficients. This fact consolidates the suggestion that bees, like mammals and birds, represent amounts according to a concave-down function (see Harder & Real, 1987; Shafir et al., 2003). But it does not explain why animals (in general) use such a representation. If animals showed here a perceptual bias that obeyed the same logic as that shown in vertebrates with uncertain delays, a concave-up function (increasing and accelerating) should be expected—involving a preference for uncertain amounts owing to the occasional delivery of larger rewards. In contrast, a concave-down function means that occasional larger amounts are hardly more attractive, while occasional small amounts are strongly unattractive.

I suggest that a concave-down function is preferred to a concave-up function with respect to amount schedules, in order to satisfy, at best, the constraints at stake. If a concave-up function strongly promotes preference for larger amounts, it also requires that animals prefer variability to constancy—and hence that they accept to possibly receive less than the expected (mean) value. However, there is no advantage associated with this possibility—such as the opportunity to receive rewards sooner: contrary to delay and ratio schedules, the two options (certain and uncertain) of amount schedules provide the reward after the same interval of time. In other words, preferring variable-amount schedules would mean that animals accept to receive less without any kind of compensation. The strong aversion for variable-amount schedules is compatible with the evidence that receiving a smaller amount than expected/desired is particularly averse, as demonstrated using the successive negative contrast procedure (Couvillon & Bitterman, 1984; Flaherty, 1996). The only way of maintaining a preference for larger amounts while avoiding the smaller amounts is to represent the subjective value of reward amounts by means of a concave-down function. In a natural context, reasonable delays and ratios are, in themselves, unlikely to be a source of problems for animals. The main problem associated with those delays and ratios originates in the possible reduction in expected food amounts, caused by intraspecific and interspecific competition. In summary, the observation by Kahneman and Tversky (1979) that losses carry more weight than gains is here possibly explained. It is argued that the higher subjective value assigned to losses relative to gains is a rational property of decision processes; it is a direct consequence of the advantage (preferring larger amounts while avoiding smaller amounts) of concave-down functions to represent reward amounts.

The social dimension

Most bees belong to social species and often exhibit no preference for reward uncertainty. We saw that solitary bees also might exhibit the same behavioural propensity. Do social and solitary vertebrates behave in a similar way when exposed to reward uncertainty? Answering this question is difficult because vertebrates represent a much vaster diversity of animal species than bees, and those species have evolved under such different ecological conditions that this certainly has influenced the type of decisions they make (e.g., Heilbronner, Rosati, Stevens, Hare, & Hauser, 2008). It must also be noted that, in vertebrate-related studies, social foraging is synonymous with “foraging in the presence of competitors” because each individual is seeking food for itself (or for its offspring). This definition is not fully satisfactory with respect to social bees (and social insects in general), which mainly collect food for the colony rather than for their private consumption—although competition between colonies exists, nevertheless (e.g., Thomson, 2004). In mammals and birds, some studies report that individuals of the same species may use distinct foraging strategies depending on whether competitors are present or absent. Common shrews (Sorex araneus) with low energy reserves and foraging alone prefer unpredictable food rewards (Barnard & Brown, 1985b), while they become indifferent to variance when a competitor—not allowed to feed—is visible (Barnard & Brown, 1985a). Greenfinches (Carduelis chloris) with low energy reserves seek variable food sources and favour solitary foraging, but they exhibit the reverse strategies with higher energy reserves (Ekman & Hake, 1988). Interestingly, Brown (1988) showed that the proportion of prey capture successes increases with group size in cliff swallows (Hirundo pyrrhonota), and is consequently higher compared to that of solitary foragers. Accordingly, house sparrows (Passer domesticus) with low energy reserves scrounge more, decreasing variability in feeding rate and in patch finding time (Lendvai, Barta, Liker, & Bokony, 2004). As these authors pointed out, scrounging can be viewed as a risk-averse (uncertainty aversion) tactic. In conclusion, social foraging in vertebrates might induce direct competition that promotes uncertainty aversion, because exploiting the patches of successful individuals improves the chances of finding food, or because the presence of competitors generates a time pressure leading individuals to exploit the locations associated with more certainty. Social foraging in bees and wasps is less competitive, so the unattractiveness of reward uncertainty should essentially depend on parameters unrelated to the presence of other individuals on the same patch. But it is likely that, like with sparrows, foraging with many conspecifics contribute to reduce the uncertainty to find a food source, as we shall see further.

In summary, mammals and birds prefer reward uncertainty to reward certainty when choice is about a delay or a ratio of responses. Conversely, they tend to prefer reward certainty to reward uncertainty, or show indifference, when choice is about the amount or the probability of reward. But, when they choose the uncertain option, it is not because they find uncertainty attractive in itself. They only choose it when associated with some advantages, such as quicker food access or reliable CSs (i.e., properties increasing the chance of survival in the wild). Bees also do not seek uncertainty for itself; current evidence indicates that they react to uncertainty like mammals and birds exposed to similar situations do. Bees might decide under reward uncertainty by means of equivalent strategies, although crucial experiments need to be carried out to properly answer the remaining issues.

Hypothesis: Bees are mainly exposed to uncertain food amounts

The data available have been obtained in bees exposed to probabilistic choices, which indicate that bees are likely to track the reliability of CSs, and therefore prefer the constant/certain over the variable/uncertain option. However, it also appears that their preference may change according to their energy budget (or honey stores in the hive), like vertebrates exposed to uncertain amounts. For this reason, it is argued that bees are mainly exposed to uncertain food amounts during their lifetime because of the type of resources they seek and of the way they seek it. As a result, evolution may have favoured the concave-down function as a major foraging strategy to represent food distributions.

Typically, a hive is composed of a queen bee which never forages, of drone bees (up to a thousand) which fertilize the queen but do not forage, and of worker bees (30,000 to 80,000) whose activity varies across their lifetime. Initially devoted to tasks inside the hive (feeding the larvae, cleaning the hive, and defending the hive against intruders such as wasps and ants), worker bees perform their first learning flights before they can acquire the status of forager bees, in general 3–4 weeks after hatching. Forager bees approximately represent 20% of the bees within a hive. During a multitude of flower visits per day, they suck up nectar through their proboscis and collect pollen by means of the baskets located on their posterior legs, and then back to the hive where those resources are stored and used to feed larvae as well as nonforager individuals. Bees consume pollen in great quantities (20–30 kg per year on average for a colony) because it is their unique source of proteins. Pollen consumption is known to affect the bee’s size, longevity, and performance (Roulston & Crane, 2000). In order to be more attractive for pollinator insects, some flowers also produce nectar, a sugar-based liquid providing a source of energy. To find food, most forager bees (called recruits) follow the instructions (waggle and round dances) provided by scout bees, a second subcategory of foragers, which incessantly seek new sites for food in the vicinity of the hive (e.g., Dreller, 1998; Liang et al., 2012). We have already mentioned that bees forage during the good seasons, a strategy that reduces their exposure to reward uncertainty. This is not to say that they never experience failures. But several factors related to their social organization indicate that, when uncertainty is encountered, it is essentially about food amounts rather than delays or ratios.
  • Risk-taking is limited. Only a minority of individuals within a hive is concerned with foraging behaviour. Among the 20% of individuals attending to collect food resources, most forage with instructions, limiting the risk of failure. Inaccuracies in the execution of waggle dances may lead recruits to miss a food source, especially when it is near the hive (Weidenmüller & Seeley, 1999). But recruits end up finding out the target flowers, so that their flights are often successful when they decide to forage on a patch. Because they tend to be flower constant (Grüter and Ratnieks, 2011; Wells & Wells, 1983), they also minimise the risk of experiencing zero reward.

  • Using many recruits. A recruit that acts upon a waggle dance may require four to 12 energetically expensive trips before it can find the rewarding flowers (Anderson, 2001). But once the recruit has located the site, many profitable return trips to that site can be made and, in a sense, cancel the initial ‘feeling’ of failure. In addition, waggle dancers typically recruit not just one but many foragers, so that abundance is commonly met by bees. This picture contrasts with that of birds and mammals which individually look for small, sparse food items. Of course, some vertebrate species can also hoard food items (e.g., Hurly, 1992; Lucas, 1994), but their abundance is rarely encountered. In general, food distribution is relatively unpredictable (no patchy distribution).

  • Exploiting in-hive odours. Arenas, Fernández, and Farina (2007) showed that food odours learned within the hive can be used by bees during foraging flights. This odour-induced bias in food preference was maintained even 24 hours after the feeders were removed from the hive. Since the odours in a hive directly depend on the floral distributions around the nest site, this is likely to inform the inexperienced bees about the type of food they have to seek. It must be noted that when recruits leave the hive to find food, they often ignore the spatial information about a food source location provided by a scout, reaching a patch of flowers based on information learned independently—for example, recruits prefer scouts that carry food odours they knew from previous trips (Grüter, Balbuena, & Farina, 2008).

  • Identifying rewarding flowers. Both honeybees and bumblebees can mark rewarding flowers with specific scent that promotes probing by conspecifics (Ferguson & Free, 1979; Kato, 1988). Also, complex floral patterns (combining a specific scent with a specific shape) reduce bees’ uncertainty in identifying rewarding flowers; bees acquire visual information about shape more effectively in the presence of scent (Leonard, Dornhaus, & Papaj, 2011).

  • Identifying unrewarding flowers. Honeybees and bumblebees (B. terrestris and B. pascuorum) are also able to identify the flowers that have been recently visited by themselves, by conspecifics, or by other species of bees in order to avoid them for a while (e.g., Giurfa, 1993; Goulson, Hawson, & Stout, 1998). They seem to use a specific scent rather than vision for this selection process because they are unable to reject inflorescences from which the nectar had been removed artificially (Goulson et al., 1998).

  • Leaving too variable patches and low food densities. When bumblebees encounter unsatisfactory reward sources, they may sample alternative flowers and can fly considerable distances in search of landscapes where variation in floral resources is less (Jha & Kremen, 2013; Wiegmann et al., 2003). Also, when reward rate in nectar volume/concentration or in the number of flowers available is too low, bees can become less flower constant in order to increase their chance of obtaining more—but the differences in flower constancy among bee species remains unexplained (Grüter & Ratnieks, 2011).

  • Benefiting from hermaphrodism of many plants. The ‘feeling’ of failure in foragers may remain limited because a majority of plants are hermaphrodite, both males and females. Thus, most flowering plants contain pollen, which is a product of the male organ (or stamen). The availability in pollen and nectar varies according to plant species. Some flowers produce much pollen but no nectar (e.g., poppy, Papaver rhoeas) and, more rarely, much nectar without pollen (e.g., burdock, Arctium lappa). Although a majority of bees specialize on pollen or nectar, the two resources are available in many plants (including trees and bushes) involved in the production of honey by bees.

Taken together, these elements suggest that forager bees are mainly exposed to variability in amounts, where all trials are (more or less) rewarded, and this may explain in a large part why they are so often not attracted by reward uncertainty: bees represent food distribution as variable amounts rather than variable delays or variable ratios. According to Schmickl and Crailsheim (2004), the bee colony’s decentralized decision system (also called ‘swarm’ intelligence) allows a reduction in the potential costs associated with a loss of nectar income when environmental fluctuations generate a change in food quality. Swarm intelligence could be viewed as an effective strategy that protects the colony against the uncertainty of finding food. If food only varies in amounts for the colony (without risk of honey shortfall), this may explain why most individual bees as members of a colony are indifferent to or avoid variability—the individual bee represents food distribution as if it was the colony. Of course, this general rule does not preclude the existence of individual differences within a colony (e.g., Banschbach & Waddington, 1994). For example, the evidence that worker honeybees are more averse to uncertainty than drones (Shafir et al., 2005), irrespective of their energy budget, is an indication that selective pressures have acted against a preference for variability in foraging individuals, while those pressures have been absent in the case of drones.

Prediction: Scouts might differ from recruits

If recruits encounter uncertain amounts most of the time, the scout bees, which attempt to find new floral patches, should experience delay-type and ratio-type variability with a higher frequency—especially under average conditions of profitability. Given that mammals and birds prefer variable over constant delays/ratios, scouts could react differently from recruits. Scouts consist of a minority of individuals within a hive, and there is evidence that genetic predispositions are necessary to develop foraging/scouting behaviour. Humphries, Fondrk, and Page (2005) showed that honeybees selected to become foragers exhibit a greater walking activity and a stronger responsiveness to sucrose. Liang et al. (2012) demonstrated that scouts differ from recruits with respect to brain gene expression, including neurotransmitters such as dopamine, octopamine, glutamate, and γ-aminobutyric acid (GABA). Octopamine, the functional equivalent to noradrenaline in vertebrates (Roeder, 1999), is known to alter foraging preferences (Giray, Galindo-Cardona, & Oskay, 2007) and promotes scouting behaviour (Liang et al., 2012). These genetic differences may suggest that scouts are prepared to deal with the uncertainty (in delay and ratio) of food distributions. Several studies report that the number of honeybees finding their next foraging site through scouting is low when resources are abundant and high when resources are poorer (Anderson, 2001; Seeley, 1983; Waddington, Visscher, Herbert, & Richter, 1994), suggesting that scouts often experience delay/ratio-type uncertainty. Thus, if uncertainty can stimulate seeking behaviour (notably because the delay to reach food is unpredictable), it is more likely to obtain this effect with scout bees because they may seek food using strategies that mimic those used by “higher” vertebrates (Schultheiss, Cheng, & Reynolds, 2015).

Owing to the quasirandom distribution of resources in the environment, animals have to explore the surroundings, especially when the environment is unknown and some information is needed. Exploration is commonly met among vertebrates (e.g., Bean, Mason, & Bateson, 1999; Forkman, 1996) as well as invertebrates (e.g., Anselme, 2013, 2015b; Mailleux, Devigne, Deneubourg, & Detrain, 2010; Wilson & Fowler, 1976). Bees also explore their surroundings in the search of significant information (Jander, 1997). The bee’s exploratory activity does not aim to find out a new food source but rather to learn the location of orienting signals (landmarks) allowing the animal to facilitate the later return trips. Two forms of exploration flight are identified (Jander, 1997). First, during focal exploration, the bee faces the visited site and performs oscillatory movements (left-right) while backing away from the site, as if it attempted to form an accurate mental representation allowing the target and contextual cues to be retrieved in a subsequent visit. Second, peripheral exploration consists of a fly forward, with more and more marked turn alternation that becomes full circles, increasing in diameter as the bee elevates above the ground. Among insects, only the aculeate hymenoptera are known to produce an exploration flight, which might be phylogenetically derived from the search flight encountered in many other species. In bees, the search flight resembles the exploration flight, although it is less stereotyped and covers a more extended zone allowing the flight over novel, potential targets (Jander, 1997). Contrary to exploration flights, search flights may lead to experiencing uncertainty but they are believed to be only performed by scouts (Degen et al., 2015). In other words, bees that explore (recruits and scouts) are not very often exposed to delay and ratio type of uncertainty, while bees that search for new targets (scouts only) can experience uncertainty.

There is evidence that bees are sensitive to time intervals (e.g., Cheng, Pena, Porter, & Irwin, 2002). Boisvert and Sherry (2006) showed that B. impatiens waits significantly longer before responding to a fixed interval of 36 s compared to a fixed interval of 6 s, presented separately. However, if the two intervals are randomly mixed up within a session, the animal fails to distinguish them, exhibiting similar wait times for the two intervals. In general, there is a single burst of responding between the two interval values, sometimes enhanced by a second burst in anticipation of the long value. More thorough studies are here necessary to examine this interesting topic, because it is unknown whether bees may show a preference for an unpredictable delay (e.g., 6 or 36 s) relative to a predictable delay (here, 21 s). For recruits, delays might be unimportant because the flight time depends more on the terrain over which the bees fly than on the distance estimation reported by waggle dancers (Barron, Zhu, Robinson, & Srinivasan, 2005). But delays might be a stronger determinant of preference in scouts. If correct, testing bees irrespective of their functional status (recruit or scout) within a hive is likely to produce results showing indifference or a preference for certain rewards because of the greater proportion of recruits relative to scouts.

The hypothesis that scouts may prefer delay-related uncertainty seems to contradict the evidence that the solitary bee X. micans, which has to find food by itself, shows indifference to uncertainty (Perez & Waddington, 1996). However, such a comparison is perhaps irrelevant because, as noted earlier, the constraints that solitary bees and social forager bees have to satisfy are different. Solitary bees have to feed for themselves, in addition to managing their reproduction as well as predation risk. In contrast, social worker bees are sterile and can take refuge in their hive in case of predatory attacks. Here, the point is that solitary individuals can spend less time and energy foraging than social individuals while experiencing a higher risk of energy shortfall. As a result, feeding rate is perhaps more important than the opportunity to collect more food occasionally. In other words, solitary bees should ignore variability and feed as often as they can—an occasional small amount is better than nothing. If correct, this would mean that social and solitary bees exhibit uncertainty indifference for distinct reasons.

Conclusion

Overall, bees react to reward uncertainty in a similar way to “higher” vertebrates, but recruits (as opposed to scouts) might essentially be exposed to variable food amounts and might therefore have an innate aversion for uncertainty. Original experiments in which the reliability of CSs is manipulated could be of interest in a comparative perspective (bees vs. vertebrates or recruits vs. scouts). Of course, not all bee species have been tested, and more thorough investigations are necessary, but it is likely that no more mammalian and avian species have been tested in comparison. In the future, the reactivity of scouts (compared with recruits) would be worth being tested. If scouts exhibited a massive preference for reward uncertainty, this might be a behavioural indicator of their genetic predisposition to forage differently from recruits.

Notes

Acknowledgements

This work was partly supported by the DFG (An-1067/1-1).

References

  1. Adamo, S. A., & Baker, J. L. (2011). Conserved features of chronic stress across phyla: The effects of long-term stress on behavior and the concentration of the neurohormone octopamine in the cricket, Gryllus texensis. Hormones and Behavior, 60, 478–483.PubMedGoogle Scholar
  2. Ahearn, W., Hineline, P. N., & David, F. D. (1992). Relative preference for various bivalued ratio schedules. Animal Learning and Behavior, 20, 407–415.Google Scholar
  3. Alcaro, A., Panksepp, J., & Huber, R. (2011). D-amphetamine stimulates unconditioned exploration/approach behaviors in crayfish: Towards a conserved evolutionary function of ancestral drug reward. Pharmacology, Biochemistry, and Behavior, 99, 75–80.PubMedPubMedCentralGoogle Scholar
  4. Ammon, D., Abramson, C. I., & Bitterman, M. E. (1986). Partial reinforcement and resistance to extinction in honeybees. Animal Learning and Behavior, 14, 232–240.Google Scholar
  5. Amsel, A. (1958). The role of frustrative nonreward in noncontinuous reward situations. Psychological Bulletin, 55, 102–119.PubMedGoogle Scholar
  6. Anderson, C. (2001). The adaptive value of inactive foragers and the scout-recruit system in honey bee (Apis mellifera) colonies. Behavioral Ecology, 12, 111–119.Google Scholar
  7. Anselme, P. (2013). Sensitivity to tactile novelty in the terrestrial isopod, Porcellio scaber. Behavioural Processes, 92, 52–59.PubMedGoogle Scholar
  8. Anselme, P. (2015a). Does reward unpredictability reflect risk? Behavioural Brain Research, 280, 119–127.PubMedGoogle Scholar
  9. Anselme, P. (2015b). Enhanced exploratory activity in woodlice exposed to random visuo-tactile patterns. Learning and Motivation, 50, 48–58.Google Scholar
  10. Anselme, P., Edeş, N., Tabrik, S., & Güntürkün, O. (2018). Long-term behavioral sensitization to apomorphine is independent of conditioning and increases conditioned pecking, but not preference, in pigeons. Behavioural Brain Research, 336, 122–134.PubMedGoogle Scholar
  11. Anselme, P., Robinson, M. J. F., & Berridge, K. C. (2013). Reward uncertainty enhances incentive salience attribution as sign-tracking. Behavioural Brain Research, 238, 53–61.PubMedGoogle Scholar
  12. Arenas, A., Fernández, V. M., & Farina, W. M. (2007). Floral odor learning within the hive affects honeybees’ foraging decisions. Naturwissenschaften, 94, 218–222.PubMedGoogle Scholar
  13. Banschbach, V. S., & Waddington, K. D. (1994). Risk-sensitivity in honey-bees—No consensus among individuals and no effect of colony honey stores. Animal Behaviour, 47, 933–941.Google Scholar
  14. Barnard, C. J., & Brown, C. A. J. (1985a). Competition affects risk-sensitivity in foraging shrews. Behavioral Ecology and Sociobiology, 16, 379–382.Google Scholar
  15. Barnard, C. J., & Brown, C. A. J. (1985b). Risk-sensitive foraging in common shrews (Sorex araneus L.). Behavioral Ecology and Sociobiology, 16, 161–164.Google Scholar
  16. Barron, A. B., Søvik, E., & Cornish, J. L. (2010). The roles of dopamine and related compounds in reward-seeking behavior across animal phyla. Frontiers in Behavioral Neuroscience, 4, 163.PubMedPubMedCentralGoogle Scholar
  17. Barron, A. B., Zhu, H., Robinson, G. E., & Srinivasan, M. V. (2005). Influence of flight time and flight environment on distance communication by dancing honey bees. Insectes Sociaux, 52, 402–407.Google Scholar
  18. Bateson, M., Desire, S., Gartside, S. E., & Wright, G. A. (2011). Agitated honeybees exhibit pessimistic cognitive bias. Current Biology, 21, 1070–1073.PubMedPubMedCentralGoogle Scholar
  19. Bean, D., Mason, G. J., & Bateson, M. (1999). Contrafreeloading in starlings: Testing the information hypothesis. Behaviour, 136, 1267–1282.Google Scholar
  20. Belke, T. W., & Spetch, M. L. (1994). Choice between reliable and unreliable reinforcement alternatives revisited: Preference for unreliable reinforcement. Journal of the Experimental Analysis of Behavior, 62, 353–366.PubMedPubMedCentralGoogle Scholar
  21. Berridge, K. C. (2007). The debate over dopamine’s role in reward: The case for incentive salience. Psychopharmacology, 19, 391–431.Google Scholar
  22. Biernaskie, J. M., Cartar, R. V., & Hurly, T. A. (2002). Risk-averse inflorescence departure in hummingbirds and bumble bees: Could plants benefit from variable nectar volumes? Oikos, 98, 98–104.Google Scholar
  23. Bitterman, M. E. (1996). Comparative analysis of learning in honeybees. Animal Learning and Behavior, 24, 123–141.Google Scholar
  24. Boisvert, M. J., & Sherry, D. F. (2006). Interval timing by an invertebrate, the bumble bee Bombus impatiens. Current Biology, 16, 1636–1640.PubMedGoogle Scholar
  25. Brown, C. R. (1988). Social foraging in cliff swallows: Local enhancement, risk sensitivity, competition, and the avoidance of predators. Animal Behaviour, 36, 780–792.Google Scholar
  26. Brydges, N. M., Leach, M., Nicol, K., Wright, R., & Bateson, M. (2011). Environmental enrichment induces optimistic cognitive bias in rats. Animal Behaviour, 81, 169–175.Google Scholar
  27. Cakmak, I., Cook, P., Hollis, J., Shah, N., Huntley, D., van Valkenburg, D., & Wells, H. (1999). Africanized honey bee response to differences in reward frequency. Journal of Apicultural Research, 38, 125–136.Google Scholar
  28. Cakmak, I., Song, D. S., Mixon, T. A., Serrano, E., Clement, M. L., Savitski, A., ... Wells, H. (2010). Foraging response of Turkish honey bee subspecies to flower color choices and reward consistency. Journal of Insect Behavior, 23, 100–116.Google Scholar
  29. Caraco, T., Blanckenhorn, W. U., Gregory, G. M., Newman, J. A., Recer, G. M., & Zwicker, S. M. (1990). Risk-sensitivity: Ambient temperature affects foraging choice. Animal Behaviour, 39, 338–345.Google Scholar
  30. Cardinal, R. N. (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural Networks, 19, 1277–1301.PubMedGoogle Scholar
  31. Cartar, R. V. (1991). A test of risk sensitive foraging in bumble bees. Ecology, 72, 888–895.Google Scholar
  32. Cartar, R. V., & Dill, L. M. (1990). Why are bumble bees risk-sensitive foragers? Behavioral Ecology and Sociobiology, 26, 121–127.Google Scholar
  33. Cheng, K., Pena, J., Porter, M. A., & Irwin, J. D. (2002). Self-control in honeybees. Psychonomic Bulletin and Review, 9, 259–263.PubMedGoogle Scholar
  34. Couvillon, P. A., & Bitterman, M. E. (1984). The overlearning-extinction effect and successive negative contrast in honeybees (Apis mellifera). Journal of Comparative Psychology, 98, 100–109.PubMedGoogle Scholar
  35. Day, J. J., Jones, J. L., Wigthtman, R. M., & Carelli, R. M. (2010). Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biological Psychiatry, 68, 306–309.PubMedPubMedCentralGoogle Scholar
  36. Degen, J., Kirbach, A., Reiter, L., Lehmann, K, Norton, P., Storms, M., ... Menzel, R. (2015). Exploratory behaviour of honeybees during orientation flights. Animal Behaviour, 102, 45–57.Google Scholar
  37. Douglas, C., Bateson, M., Walsh, C., Bédué, A., & Edwards, S. A. (2012). Environmental enrichment induces optimistic cognitive biases in pigs. Applied Animal Behaviour Science, 139, 65–73.Google Scholar
  38. Dreller, C. (1998). Division of labor between scouts and recruits: Genetic influence and mechanisms. Behavioral Ecology and Sociobiology, 43, 191–196.Google Scholar
  39. Drezner-Levy, T., & Shafir, S. (2007). Parameters of variable reward distributions that affect risk sensitivity of honey bees. Journal of Experimental Biology, 210, 269–277.PubMedGoogle Scholar
  40. Dukas, R., & Real, L. A. (1991). Learning foraging tasks by bees: A comparison between social and solitary species. Animal Behaviour, 42, 269–276.Google Scholar
  41. Dunn, R., & Spetch, M. L. (1990). Choice with uncertain outcomes: Conditioned reinforcement effects. Journal of the Experimental Analysis of Behavior, 53, 201–218.PubMedPubMedCentralGoogle Scholar
  42. Ekman, J., & Hake, M. (1988). Avian flocking reduces starvation risk: An experimental demonstration. Behavioral Ecology and Sociobiology, 22, 91–94.Google Scholar
  43. Elwood, R. W., Barr, S., & Patterson, L. (2009). Pain and stress in crustaceans? Applied Animal Behaviour Science, 118, 128–136.Google Scholar
  44. Estle, S. J., Green, L., Myerson, J., & Holt, D. D. (2006). Differential effects of amount on temporal and probability discounting of gains and losses. Learning and Memory, 34, 914–928.Google Scholar
  45. Ferguson, A. W. & Free, J. B. (1979). Production of forage-marking pheromone by the honeybee. Journal of Apiculture Research, 18, 128–135.Google Scholar
  46. Field, D. P., Tonneau, F., Ahearn, W., & Hineline, P. N. (1996). Preference between variable-ratio and fixed-ratio schedules: Local and extended relations. Journal of the Experimental Analysis of Behavior, 66, 283–295.PubMedPubMedCentralGoogle Scholar
  47. Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902.PubMedGoogle Scholar
  48. Fischer, M. E., Couvillon, P. A., & Bitterman, M. E. (1993). Choice in honeybees as a function of the probability of reward. Animal Learning and Behavior, 21, 187–195.Google Scholar
  49. Flaherty, C. F. (1996). Incentive relativity. Cambridge, UK: Cambridge University Press.Google Scholar
  50. Forkman, B. (1996). The foraging behaviour of Mongolian gerbils: A behavioural need or a need to know? Behaviour, 133, 129–143.Google Scholar
  51. Fülöp, A., & Menzel, R. (2000). Risk-indifferent foraging behaviour in honeybees. Animal Behaviour, 60, 657–666.PubMedGoogle Scholar
  52. Gathmann, A., & Tscharntke, T. (2002). Foraging ranges of solitary bees. Journal of Animal Ecology, 71, 757–764.Google Scholar
  53. Gibbon, J., Farrell, L., Locurto, C. M., Duncan, H. J., & Terrace, H. S. (1980). Partial reinforcement in autoshaping with pigeons. Animal Learning and Behavior, 8, 45–59.Google Scholar
  54. Gipson, C. D., Alessandri, J. J. D., Miller, H. C., & Zentall, T. R. (2009). Preference for 50% reinforcement over 75% reinforcement by pigeons. Learning and Behavior, 37, 289–298.PubMedGoogle Scholar
  55. Giray, T., Galindo-Cardona, A., & Oskay, D. (2007). Octopamine influences honey bee foraging preference. Journal of Insect Physiology, 53, 691–698.PubMedPubMedCentralGoogle Scholar
  56. Giurfa, M. (1993). The repellent scent-mark of the honeybee Apis mellifera ligustica and its role as communication cue during foraging. Insectes Sociaux, 40, 59–67.Google Scholar
  57. Gottlieb, D. A. (2004). Acquisition with partial and continuous reinforcement in pigeon autoshaping. Learning and Behavior, 32, 321–334.PubMedGoogle Scholar
  58. Goulson, D., Hawson, S. A., & Stout, J. C. (1998). Foraging bumblebees avoid flowers already visited by conspecifics or by other bumblebee species. Animal Behaviour, 55, 199–206.PubMedGoogle Scholar
  59. Grüter, C., Balbuena, M. S., & Farina, W. M. (2008). Informational conflicts created by the waggle dance. Proceedings of the Royal Society Series B, 275, 1321–1327.Google Scholar
  60. Grüter, C., & Ratnieks, F. L. W. (2011). Flower constancy in insect pollinators: Adaptive foraging behaviour or cognitive limitation? Communicative and Integrative Biology, 4, 633–636.PubMedGoogle Scholar
  61. Harder, L. D., & Real, L. A. (1987). Why are bumble bees risk averse? Ecology, 68, 1104–1108.Google Scholar
  62. Heilbronner, S. R., Rosati, A. G., Stevens, J. R., Hare, B., & Hauser, M. B. (2008). A fruit in the hand or two in the bush: Divergent risk preferences in chimpanzees and bonobos. Biology Letters, 4, 246–249.PubMedPubMedCentralGoogle Scholar
  63. Humphries, M. A., Fondrk, M. K., & Page, R. E. (2005). Locomotion and the pollen hoarding behavioral syndrome of the honey bee (Apis mellifera L.). Journal of Comparative Physiology A, 191, 669–674.Google Scholar
  64. Hurly, T. A. (1992). Energetic reserves of marsh tits (Parus palustris): Food and fat storage in response to variable food supply. Behavioral Ecology, 3, 181–188.Google Scholar
  65. Ishida, M., Couvillon, P. A., & Bitterman, M. E. (1992). Acquisition and extinction of a shuttling response in honeybees (Apis mellifera) as a function of the probability of reward. Journal of Comparative Psychology, 106, 262–269.Google Scholar
  66. Jander, R. (1997). Macroevolution of a fixed action pattern for learning: The exploratory flights of bees and wasps. In G. Greenberg & E. Tobach (Eds.), Comparative psychology of invertebrates: The field and laboratory study of insect behavior (pp. 79–99). New York, NY: Garland.Google Scholar
  67. Jha, S., & Kremen, C. (2013). Resource diversity and landscape-level homogeneity drive native bee foraging. Proceedings of the National Academy of Science, 110, 555–558.Google Scholar
  68. Johnson, D. D. P., & Fowler, J. H. (2011). The evolution of overconfidence. Nature, 477, 317–320.PubMedGoogle Scholar
  69. Johnson, P. S., Madden, G. J., Brewer, A. T., Pinkston, J. W., & Fowler, S. C. (2011). Effects of acute pramipexole on preference for gambling-like schedules of reinforcement in rats. Psychopharmacology, 213, 11–18.PubMedGoogle Scholar
  70. Kacelnik, A., & Bateson, M. (1996). Risky theories: The effects of variance on foraging decisions. American Zoologist, 36, 402–434.Google Scholar
  71. Kacelnik, A., & Mouden, C.E. (2013). Triumphs and trials of the risk paradigm. Animal Behaviour, 86, 1117–1129.Google Scholar
  72. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–292.Google Scholar
  73. Kato, M. (1988). Bumble bee visits to Impatiens spp.: Pattern and efficiency. Oecologia, 76, 364–370.PubMedGoogle Scholar
  74. Knight, F. (1921). Risk, uncertainty, and profit. Boston, MA: Houghton Mifflin.Google Scholar
  75. Laude, J. R., Stagner, J. P., & Zentall, T. R. (2014). Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement. Journal of Experimental Psychology: Animal Learning and Cognition, 40, 12–21.Google Scholar
  76. Lendvai, A. Z., Barta, Z., Liker, A., & Bokony, V. (2004). The effect of energy reserves on social foraging: Hungry sparrows scrounge more. Proceedings of the Royal Society Series B, 271, 2467–2472.Google Scholar
  77. Leonard, A. S., Dornhaus, A., & Papaj, D. R. (2011). Flowers help bees cope with uncertainty: Signal detection and the function of floral complexity. Journal of Experimental Biology, 214, 113–121.PubMedGoogle Scholar
  78. Liang, Z. S., Nguyen, T., Mattila, H. R., Rodriguez-Zas, S. L., Seeley, T. D., & Robinson, G. E. (2012). Molecular determinants of scouting behavior in honey bees. Science, 335, 122–125.Google Scholar
  79. Lucas, J. R. (1994). Regulation of cache stores and body mass in Carolina chickadees (Parus carolinensis). Behavioral Ecology, 5, 171–181.Google Scholar
  80. Mailleux, A.-C., Devigne, C., Deneubourg, J.-L., & Detrain, C. (2010). Impact of starvation on Lasius niger’s exploration. Ethology, 116, 248–256.Google Scholar
  81. Marshall, J. A. R., Trimmer, P. C., Houston, A. I., & McNamara, J. M. (2013). On evolutionary explanations of cognitive biases. Trends in Ecology and Evolution, 28, 469–473.PubMedGoogle Scholar
  82. Matheson, S. M., Asher, L., & Bateson, M. (2008). Larger, enriched cages are associated with ‘optimistic’ response biases in captive European starlings (Sturnus vulgaris). Applied Animal Behaviour Science, 109, 374–383.Google Scholar
  83. Mayack, C., & Naug, D. (2011). A changing but not an absolute energy budget dictates risk-sensitive behaviour in the honeybee. Animal Behaviour, 82, 595–600.Google Scholar
  84. Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analyses of behavior: V. The effect of delay and of intervening events on reinforcement value (pp. 55–73). Mahwah, NJ: Erlbaum.Google Scholar
  85. Mazur, J. E. (1991). Choice with probabilistic reinforcement: Effects of delay and conditioned reinforcers. Journal of the Experimental Analysis of Behavior, 55, 63–77.PubMedPubMedCentralGoogle Scholar
  86. McDevitt, M. A., Dunn, R. M., Spetch, M. L., & Ludvig, E. A. (2016). When good news leads to bad choices. Journal of the Experimental Analysis of Behavior, 105, 23–40.PubMedGoogle Scholar
  87. McNamara, J. M., & Houston, A. I. (1987). A general framework for understanding the effects of variability and interruptions on foraging behaviour. Acta Biotheoretica, 36, 3–22.PubMedGoogle Scholar
  88. McNamara, J. M., & Houston, A. I. (1992). Risk-sensitive foraging: A review of the theory. Bulletin of Mathematical Biology, 54, 355–378.Google Scholar
  89. Ottaviani, E., & Franceschi, C. (1996). The neuroimmunology of stress from invertebrates to man. Progress in Neurobiology, 48, 421–440.PubMedGoogle Scholar
  90. Papini, M. R. (2006). Role of surprising nonreward in associative learning. Japanese Journal of Animal Psychology, 56, 35–54.Google Scholar
  91. Papini, M. R., Thomas, B. L., & McVicar, G. (2002). Between-subject PREE and within-subject reversed PREE in spaced-trial extinction with pigeons. Learning and Motivation, 33, 485–509.Google Scholar
  92. Pattison, K. F., Laude, J. R., & Zentall, T. R. (2013). Environmental enrichment affects suboptimal, risky, gambling-like choice by pigeons. Animal Cognition, 16, 429–434.PubMedGoogle Scholar
  93. Pearce, J. M., Redhead, E. S., & Aydin, A. (1997). Partial reinforcement in appetitive Pavlovian conditioning with rats. Quarterly Journal of Experimental Psychology, 50B, 273–294.Google Scholar
  94. Perez, S. M., & Waddington, K. D. (1996). Carpenter bee (Xylocopa micans) risk indifference and a review of nectarivore risk-sensitivity studies. American Zoologist, 36, 435–446.Google Scholar
  95. Perry, C. J., & Barron, A. B. (2013). Neural mechanisms of reward in insects. Annual Review of Entomology, 58, 543–562.PubMedGoogle Scholar
  96. Pyke, G. H. (1984). Optimal foraging theory: A critical review. Annual Review of Ecology and Systematics, 15, 523–575.Google Scholar
  97. Rathcke, B. (1992). Nectar distributions, pollinator behavior, and plant reproductive success. In M. D. Hunter, T. Ohgushi, & P. W. Price (Eds.), Effects of resource distribution on animal-plant interactions (pp 113–138). San Diego, CA: Academic Press.Google Scholar
  98. Real, L. A. (1981). Uncertainty and pollinator-plant interactions: The foraging behavior of bees and wasps on artificial flowers. Ecology, 62, 20–26.Google Scholar
  99. Real, L. A., Ott, J., & Silverfine, E. (1982). On the trade-off between mean and variance in foraging: An experimental analysis with bumblebees. Ecology, 63, 1617–1623.Google Scholar
  100. Rescorla, R. A. (1999). Within-subject partial reinforcement extinction effect in autoshaping. Quarterly Journal of Experimental Psychology, 52B, 75–87.Google Scholar
  101. Roeder, T. (1999) Octopamine in invertebrates. Progress in Neurobiology, 59, 533–561.PubMedGoogle Scholar
  102. Roulston, T. H., & Crane, J. H. (2000). Pollen nutritional content and digestibility for animals. Plant Systematics and Evolution, 222, 187–209.Google Scholar
  103. Schmickl, T., & Crailsheim, K. (2004). Costs of environmental fluctuations and benefits of dynamic decentralized foraging decisions in honey bees. Adaptive Behavior, 12, 263–277.Google Scholar
  104. Schultheiss, P., Cheng, K., & Reynolds, A.M. (2015). Searching behavior in social hymenoptera. Learning and Motivation, 50, 59–67.Google Scholar
  105. Seeley, T. D. (1983). Division of labor between scouts and recruits in honeybee foraging. Behavioral Ecology and Sociobiology, 12, 253–259.Google Scholar
  106. Shafir, S. (2000). Risk-sensitive foraging: the effect of relative variability. Oikos, 88, 663–669.Google Scholar
  107. Shafir, S., Bechar, A., & Weber, E.U. (2003). Cognition-mediated coevolution-context-dependent evaluations and sensitivity of pollinators to variability in nectar rewards. Plant Systematics and Evolution, 238, 195–209.Google Scholar
  108. Shafir, S., Menda, G., & Smith, B.H. (2005). Caste-specific differences in risk sensitivity in honeybees, Apis mellifera. Animal Behaviour, 69, 859–868.Google Scholar
  109. Shafir, S., Wiegmann, D. D., Smith, B. H., & Real, L. A. (1999). Risk-sensitive foraging: Choice behaviour of honeybees in response to variability in volume of reward. Animal Behaviour, 57, 1055–1061.PubMedGoogle Scholar
  110. Shapiro, M. S., Couvillon, P. A., & Bitterman, M. E. (2001). Quantitative tests of an associative theory of risk-sensitivity in honeybees. Journal of Experimental Biology, 204, 565–573.PubMedGoogle Scholar
  111. Sherman, J. A., & Thomas, J. R. (1968). Some factors controlling preference between fixed-ratio and variable-ratio schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 11, 689–702.PubMedPubMedCentralGoogle Scholar
  112. Smith, A. P., & Zentall, T. R. (2016). Suboptimal choice in pigeons: Choice is primarily based on the value of the conditioned reinforcer rather than overall reinforcement rate. Journal of Experimental Psychology: Animal Learning and Cognition, 42, 212–220.Google Scholar
  113. Spetch, M. L., Belke, T. W., Barnet, R. C., Dunn, R., & Pierce, W. D. (1990). Suboptimal choice in a percentage-reinforcement procedure: Effects of signal condition and terminal-link length. Journal of the Experimental Analysis of Behavior, 53, 219–234.PubMedPubMedCentralGoogle Scholar
  114. Spivey, J. E., & Hess, D. T. (1968). Effect of partial reinforcement trial sequences on extinction performance. Psychonomic Science, 10, 375–376.Google Scholar
  115. Stagner, J. P., & Zentall, T. R. (2010). Suboptimal choice behavior by pigeons. Psychonomic Bulletin and Reviews, 17, 412–416.Google Scholar
  116. Stefano, G. B., Cadet, P., Zhu, W., Rialas, C. M., Mantione, K., Benz, D., Fuentes, R., … Slingsby, B. (2002). The blueprint for stress can be found in invertebrates. Neuroendocrinology Letters, 23, 85–93.PubMedGoogle Scholar
  117. Stephens, D. W., & Krebs, J. R. (1986). Foraging theory. Princeton, NJ: Princeton University Press.Google Scholar
  118. Strausfeld, N. J., & Hirth, F. (2013). Deep homology of arthropod central complex and vertebrate basal ganglia. Science, 340, 157–161.PubMedGoogle Scholar
  119. Thomson, D. (2004). Competitive interactions between the invasive European honey bee and native bumble bees. Ecology, 85, 458–470.Google Scholar
  120. Vasconcelos, M., Monteiro, T., & Kacelnik, A. (2015). Irrational choice and the value of information. Scientific Reports, 5, 13874.PubMedPubMedCentralGoogle Scholar
  121. von Hippel, W., & Trivers, R. (2011). The evolution and psychology of self-deception. Behavioral and Brain Sciences, 34, 1–16.Google Scholar
  122. Waddington, K. D. (1985). Cost-intake information used in foraging. Journal of Insect Physiology, 31, 891–897.Google Scholar
  123. Waddington, K. D. (1995). Bumblebees do not respond to variance in nectar concentration. Ethology, 101, 33–38.Google Scholar
  124. Waddington, K. D., Allen, T., & Heinrich, B. (1981). Floral preferences of bumblebees (Bombus edwardsii) in relation to intermittent versus contiguous rewards. Animal Behaviour, 29, 779–784.Google Scholar
  125. Waddington, K. D., & Gottlieb, N. (1990). Actual versus perceived profitability: A study of floral choice of honey bees. Journal of Insect Behavior, 3, 429–441.Google Scholar
  126. Waddington, K. D., & Kirchner, W. H. (1992). Acoustical and behavioral correlates of profitability of food sources in honey bee round dances. Ethology, 92, 1–6.Google Scholar
  127. Waddington, K. D., Visscher, P. K., Herbert, T. J., & Richter, M. R. (1994). Comparisons of forager distributions from matched honey bee colonies in suburban environments. Behavioral Ecology and Sociobiology, 35, 423–429.Google Scholar
  128. Weidenmüller, A., & Seeley, T. D. (1999). Imprecision in waggle dances of the honeybee (Apis mellifera) for nearby food sources: Error or adaptation? Behavioral Ecology and Sociobiology, 46, 190–199.Google Scholar
  129. Wells, H., & Wells, P. H. (1983). Honey bee foraging ecology: Optimal diet, minimal uncertainty or individual constancy? Journal of Animal Ecology, 52, 829–836.Google Scholar
  130. Wiegmann, D. D., Wiegmann, D. A., & Waldron, F. A. (2003). Effects of a reward downshift on the consummatory behavior and flower choices of bumblebee foragers. Physiology and Behavior, 79, 561–566.PubMedGoogle Scholar
  131. Wilson, M. M., & Fowler, H. (1976). Variables affecting alternation behaviour in the cockroach, Blatta orientalis. Animal Learning and Behavior, 4, 490–494.Google Scholar
  132. Zurbuchen, A., Landert, L., Klaiber, J., Müller, A., Hein, S., & Dorn, S. (2010). Maximum foraging ranges in solitary bees: Only few individuals have the capability to cover long foraging distances. Biological Conservation, 143, 669–676.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Department of BiopsychologyUniversity of BochumBochumGermany

Personalised recommendations