Dual contributions of noradrenaline to behavioural flexibility and motivation
While several theories have highlighted the importance of the noradrenergic system for behavioral flexibility, a number of recent studies have also shown a role for noradrenaline in motivation, particularly in effort processing. Here, we designed a novel sequential cost/benefit decision task to test the causal influence of noradrenaline on these two functions in rhesus monkeys.
We manipulated noradrenaline using clonidine, an alpha-2 noradrenergic receptor agonist, which reduces central noradrenaline levels and examined how this manipulation influenced performance on the task.
Clonidine had two specific and distinct effects: first, it decreased choice variability, without affecting the cost/benefit trade-off; and second, it reduced force production, without modulating the willingness to work.
Together, these results support an overarching role for noradrenaline in facing challenging situations in two complementary ways: by modulating behavioral volatility, which would facilitate adaptation depending on the lability of the environment, and by modulating the mobilization of resources to face immediate challenges.
KeywordsNoradrenaline Behavioral flexibility Motivation Clonidine Monkey
Noradrenaline is among the most widespread neuromodulators in the brain. It was initially associated with vigilance and arousal (Kety 1972; Harley 1987; Aston-Jones et al. 1991; Berridge 1991; Waterhouse et al. 1998; Berridge and Waterhouse 2003), but several authors suggested that this role could extend to cognitive functions such as attention, learning, and memory (Usher et al. 1999; Arnsten 2000; Harley 2007; Arnsten et al. 2012; Sara and Bouret 2012; Sara 2015) and with a particular emphasis on various forms of behavioral flexibility (Devauges and Sara 1990; Bouret and Sara 2004; Chamberlain et al. 2006; Tait et al. 2007; McGaughy et al. 2008; Guedj et al. 2016). This led several authors to put behavioral flexibility at the heart of the functional role of noradrenaline in cognition (Yu and Dayan 2005; Aston-Jones and Cohen 2005; Bouret and Sara 2005). More recently, we and others have also emphasized the potential role of noradrenaline in motivation, with a strong role in effort processing (Ventura et al. 2008; Bouret and Richmond 2009; Zénon et al. 2014; Varazzani et al. 2015).
While aspects of these theories overlap, it has nonetheless been difficult to determine how to reconcile these different ideas as they have seldom been directly tested in the same experiment. A further complication is the strong relation between autonomic arousal and cognitive functions, classically attributed to the noradrenergic system (Einhäuser et al. 2008; Preuschoff et al. 2011; Nassar et al. 2012; Gee et al. 2017). This is based on the correlation between locus ceruleus (LC) firing and pupil diameter, even if this relation is far from being specific. Indeed, LC activity correlates with multiple measures of arousal, and these measures of autonomic arousal are associated with the activation of several other brain regions (Abercrombie and Jacobs 1987; Berridge and Waterhouse 2003; Joshi et al. 2016; Gee et al. 2017). Thus, one hypothesis would be that noradrenaline is simply associated with autonomic arousal and contributes to all processes linked with arousal in a highly non-specific fashion. In that frame, the various functions classically attributed to noradrenaline (behavioral flexibility and motivation), and measures associated with these (e.g., response latencies, willingness-to-work, effort sensitivity, force production or patterns of choices), would all co-vary across states of arousal and/or vigilance. Alternatively, noradrenaline could have separable influences on specific and independent processes, over and above its influence on vigilance and arousal.
The goal of this experiment was to test the causal role of noradrenaline in behavioral flexibility and motivation using a quantitative approach. Behavioral flexibility, the adaptation of the behavior to changes in the environment, could be achieved through two distinct (but non-exclusive) classes of processes: a high level executive control of the behavior (directed exploration) and a low level change of variability in the behavior (random exploration), which would allow the animal to randomly sample several alternatives. To do this, we developed an original sequential cost-benefit decision-making task, used computational modeling to identify precisely the cognitive processes of interest, and then examined the consequences of manipulating central noradrenergic neurotransmission using systemic injection of clonidine, an alpha-2 noradrenergic receptor agonist, which has been shown to decrease the firing of LC neurons and reduce central noradrenaline levels (Abercrombie and Jacobs 1987; Abercrombie et al. 1988; Grant et al. 1988; Berridge and Abercrombie 1999; Bouret and Richmond 2009).
Clonidine had two distinct effects. First, in line with the role of noradrenaline in the simplest aspect of behavioral flexibility, clonidine dose-dependently decreased choice variability: under clonidine, monkeys’ choices became more consistent with the cost-benefit analysis, but did not change their overall switch probabilities. Second, in line with the putative role of noradrenaline in motivation and effort, clonidine dose-dependently reduced force production during the task, though left animals’ willingness-to-work and cost-benefit trade-offs unaffected. Importantly, these effects on choice variability and force production did not co-vary across drug doses, as one would expect if they were simply driven by fluctuations of arousal/vigilance.
Materials and methods
Two male rhesus monkeys (monkey A, 15 kg, 5 years old; monkey D, 15 kg, 6 years old) and one female (monkey E, 4.5 kg, 3 years old) were used for the experiment. Their access to water was regulated. During testing days (Monday to Friday), they received water as reward and they received water according to their physiological needs over the weekend. All experimental procedures were designed in association with the Institut du Cerveau et de la Moelle Epiniere (ICM) veterinarians, approved by the Regional Ethical Committee for Animal Experiment (CREEA IDF no. 3), and performed in compliance with the European Community Council Directives (86/609/EEC).
Each monkey sat in a primate chair positioned in front of a monitor on which visual stimuli were displayed. Two electronic grips (M2E Unimecanique, Paris, France) were mounted on the chair at the level of the monkey’s hands. Monkeys were not constrained to use one hand or the other to squeeze the grips. Each grip corresponded to one side of the screen. Water rewards were delivered from a tube positioned between the monkey’s lips. Behavioral paradigm was controlled using the REX system (NIH, MD, USA) and Presentation software (Neurobehavioral systems, Inc., CA, USA).
In 30% of trials, monkeys were presented with no alternatives and so had to complete the initial sequence to gain the reward. However, in 70% of trials, monkeys were given the choice during the sequence to take an alternative option (Fig. 1a). The alternative option was presented on the opposite side of the screen and occurred at least three squeezes after the beginning of the initial sequence and at most three squeezes before the end of it (Fig. 1b). To choose this option, monkeys had to switch to squeezing the corresponding grip when the dot turned green. Only one alternative option was offered per trial and it was presented only once during the sequence. In 10% of trials, the alternative option was the same as the current option (i.e., the same reward size and the same remaining sequence length). In 20% of trials, the two dimensions of the choice were congruent: the alternative option had either a longer/same sequence length and a smaller/same reward size or a shorter/same sequence length and a larger/same reward. Note that the two options could not have the same reward size and sequence length. In the remaining 40% of trials, the two dimensions of the choice were incongruent: the alternative option either had a longer sequence length but a bigger reward size or a shorter sequence length but a smaller reward. The sequence length and reward size of the alternative option were drawn so that (i) if option A was offered as a current option and B as alternative on one trial, it was equally probable that A would be offered as an alternative and B as a current option on another trial, (ii) all sequence lengths (three to eight squeezes) were equally probable for the current and the alternative options, and (iii) before the choice, the numbers of squeezes performed were counterbalanced across sequences. As a consequence, starting with a long sequence was more probable (11 out of 18 trials) than a medium (5 out of 11 trials) and a short (2 out of 11 trials) sequence. All reward sizes and sides were equally probable.
All three monkeys followed the same training procedure. They were first exposed (forced choice) to different sequence lengths, then had to choose between two options differing only in sequence length. Then, we exposed them to different reward sizes, and they had to choose between two options differing only in reward sizes. Finally, they had to choose between options differing both in reward sizes and sequence lengths. At the beginning of this training phase, both dimensions were favorable to the same option (congruent). We progressively increased the proportion of incongruent choices to 40%. Incongruent choices were more difficult for the monkeys since the reward size was greater for one option whereas the sequence length was shorter for the other option. We started recording their behavior and using pharmacological manipulations a least a month after they reached asymptotic performance on the final version of the task.
We used three doses of clonidine—2.5, 5.0, and 7.5 μg/kg—which is a selective alpha-2 noradrenergic receptor agonist that, as well as potentially acting on post-synaptic alpha-2 receptors, previous studies have shown acts directly on LC neurons to suppress firing and consequently noradrenaline levels at the doses that we used (Kawahara et al. 1999; Fernández-Pastor et al. 2005). The doses that we used were below the doses that affected working memory functions in monkeys, which have been suggested to depend on the action of clonidine on post-synaptic alpha-2 receptors (Franowicz and Arnsten 1999). They were also below the sedative effect threshold determined in rats (Sara et al. 1995; Lapiz and Morilak 2006) and monkeys (Bouret and Richmond 2009), but elicited a subjective feeling of sedation humans (Jäkälä et al. 1999). Clonidine hydrochloride (C7897, Sigma-Aldrich, St. Louis, MO, USA) solutions were prepared freshly each day by dissolution in 1 mL saline for monkeys A and D and 0.5 mL for monkey E. The same volume of saline solution was given in saline condition. Drug or saline solution was injected intramuscularly 20 min before testing in monkeys’ home cages. Each dose or vehicle was given for five consecutive days (Monday to Friday). Order of drug and saline weeks (one drug week per dose and two saline weeks) was randomly assigned for each animal.
Data were analyzed with Matlab software (MathWorks).
All sessions lasted 1 h. Monkeys’ overall willingness to work in the session was assessed by looking at all offered squeezes as long as monkeys were engaged in the task. Indeed, since monkeys tended to completely disengage before the end of the session, we removed these series of terminal rejected trials at the end of each session. The behavior during the sequence was examined by looking the %accepted squeezes during the completion of a trial. To look at the progression in the sequence effect, we looked only at trials where no choice was offered. The willingness to work on the first squeeze of a sequence was assessed in the same way but by taking all first choices since the subjects could not know whether a choice was going to be offered or not in a particular trial. Reaction time corresponds to the time between the display of the green dot and the crossing of the minimum force threshold for correct squeezes. Note that the green dot always appeared 2 s after the onset of visual cues indicating sequence length and reward size. Reaction time distributions for each grip and each monkey were log-transformed (to obtain Gaussian-like distributions) and z-scored (mean set to zero and variance to one).
To assess changes in the decision process, we looked at three variables: (i) the proportion of alternative options chosen per session, (ii) average chosen number of squeezes, (iii) the average chosen reward size, and (iv) the stability in choices. The stability is the mean probability to repeat the same choice (i.e., either taking the alternative or taking the current option) in each possible combination of differences in reward size (five possibilities − 2, − 1, 0, + 1, + 2) and sequence length (five possibilities − 4, − 2, 0, + 2, + 4) between the current and the alternative options. The stability is therefore maximal if monkeys are perfectly stable in their choices and minimal if they are completely random across all choices.
Reaction time corresponds to the time between the display of the green dot and the crossing of the minimum force threshold for correct squeezes. Reaction time distributions for each grip and each monkey were log-transformed and z-scored (mean set to zero and variance to one). We compared reaction times across different squeeze types. When comparing choice and no choice reaction times, no choice reaction time corresponds to trials where no choice was offered but in principle could have been (Fig. 1b) and choice reaction time corresponds to points in the sequence where monkeys were presented with an alternative option but stayed with the original option. To evaluate the effect of choice difficulty on reaction time, we examined the effect of the absolute difference in value between two options, since choices get more difficult when this difference decreases. We binned the data according to the absolute difference in value of the options (five bins, same as used to plot the Fig. 4d, e) and took only choices where monkeys stayed with the current option and made a correct decision.
Motivational changes with treatment were assessed by two variables: force peak and willingness to work.
To calculate force peak, force time series for both grips were low-pass filtered at 15 Hz (zero-phase second-order Butterworth filter) and we took the maximal value of the force signal between two crossings of the minimal force threshold.
Overview of the task and subjects’ performance
We trained three monkeys to perform the task depicted in Fig. 1. In each trial, monkeys performed series of actions (squeezing a grip six, eight, or ten times) to get a small, medium, or large fluid reward (Fig. 1a). Note that the amount of force required to complete the trial was minimal and monkeys always succeeded to complete a squeeze when they tried. In 70% of trials, we introduced a choice by presenting an alternative option before monkeys could complete the trial. This alternative option, presented on the opposite side of the monitor compared to the current option, was also characterized by a given number of squeezes and a given reward size. Thus, monkeys could either choose to continue with the current option by squeezing the same grip as before, or switch to the other grip to start completing the alternative option and obtain the corresponding reward.
Monkeys performed on average (across treatment conditions) 61, 71, and 55 trials per session (monkeys A, D, and E respectively). Note that since the amount of force required to complete the squeeze was minimal, monkeys always succeeded to complete it when they tried. Thus, a failure to squeeze the bar was always interpreted as a rejection of the current offer, either at the onset or in the middle of a sequence. Overall, when monkeys were performing the task, they engaged in respectively 90.4, 95.8, and 78.9% of the offered forced choice squeezes (all “no choice” squeezes included) (monkeys A, D, and E respectively, mean across treatment conditions, no significant effect of treatment condition: linear effect taking into account the variability across subjects, t(10) = − 0.52, p = 0.61). Note that these relatively low percentages are due to the fact that the first squeezes of the sequence were often refused. On the choice squeezes, they were also very unlikely to disengage, and they accepted 99.4, 99.6, and 99.8% of choices (monkeys A, D, and E respectively, the same as above t(10) = − 0.02, p = 0.79).
Behavior under saline
This analysis of monkeys’ behavior under saline showed that their performance in the task was similar. However, we could observe different profiles. Monkey A had a general bias toward staying with the current option. The other male (monkey D) had the opposite bias, and the female (monkey E) had no bias. These idiosyncratic features of the monkeys’ behavior were taken into account in all our subsequent analyses. Indeed, as for the other behavioral measures (force, reaction times, stability in choices) and parameter estimates, we fitted a different intercept for each monkey.
Effects of clonidine on behavioral flexibility: choices
We first measured the influence of clonidine on choices. We first looked at the average number of squeezes chosen and the reward size chosen (Fig. 3a), no matter whether it was a stay or switch choice, which provides a global estimate of the animals’ relative sensitivity to costs and benefits. This measure was also not reliably affected by the treatment (linear regression taking into account subject variability, t(10) = 1.85, p = 0.09 and t(10) = 0.65, P = 0.53, for squeezes and reward size respectively).
We next examined the influence of clonidine on the choice bias toward the current versus alternative option, over and above the influence of expected costs (number of squeezes) and benefits (reward sizes). Because all options were offered in equal proportions as current and alternative options, any departure of the probability to take the alternative option from 50% would represent a bias toward staying or switching. At the group level, there was no significant bias toward staying or switching across all treatment conditions (linear regression taking into account subject variability, t(10) = 0.52, p = 0.52), and the bias was not different from zero in any condition (T test, p > 0.47, for all doses) (Fig. 3b).
We then looked the stability in choices across doses. As shown in Fig. 3c, there was a clear linear increase in stability across doses of clonidine, which means that with increasing doses of clonidine, the monkeys became increasingly likely to make the same decisions when faced with the same type of choice (linear regression taking into account variability across subjects, β = 0.559 ± 0.183, t(10) = 3.06, p = 0.01).
To capture the specific influence of clonidine on distinct components of decision-making, we built a simple choice model depicted in Eqs. 1 and 2 in “Material and methods”. In this model, the value of each option corresponds to a trade-off between reward at stake and sequence length, controlled by a parameter kcost-benefit. The probability to select a given option depends on (i) the value difference with the alternative and (ii) a fixed bias, e.g., a preference for either staying with the current option or taking the alternative, as well as (iii) the choice consistency, which determines the degree to which choices are consistent with the evaluation.
To ensure that our model accurately captured monkeys’ choices, we computed the model balanced accuracy for each subject and each treatment condition and it was overall between 0.80 and 0.86 (Fig. 4f). There was close but not significant positive effect of dose on balanced accuracy (linear regression taking into account variability across subjects, β = 0.008 ± 0.004, t(10) = 2.08, p = 0.06) and close but not significant correlation between the balanced accuracy and the consistency parameter (linear regression taking into account variability across subjects, β = 12.39 ± 6.13, t(10) = 2.02, p = 0.07). However, because these statistical tests were close to the significance threshold and to ensure that the balanced accuracy was not driving the effect on the consistency parameter, we compared a model in which the consistency parameter is linearly dependent on the balanced accuracy (linear regression taking into account variability across subjects, Bayesian information criterion: BIC = 23.9) and linearly dependent on the dose (same, BIC = 18.8). Since the later model won the comparison (∆BIC = 5.1 > 3, meaning that there is a strong evidence in favor of the dose-effect model), the dose effect was better explained by a change on the consistency parameter than by a change in goodness of fit, evaluated by the balanced accuracy.
Finally, we used a more complex model (full model) to try to capture other factors potentially affecting choices. Notably, we generally found a significant side bias in monkeys’ choices but it was not systematically significant and changed direction across treatment conditions (or weeks of testing) in the same animal. We also added a parameter capturing the effect of the number of squeezes done before the choice and found a significant positive effect of this parameter on the probability to take the alternative option. Hence, monkeys were more likely to take the alternative option if they had done more squeezes to get to the choice. We also added a parameter to capture an imbalance in the evaluation of the options, but this parameter was not different from 1 across all subjects and doses (T test, p = 0.08). Overall, including these parameters in the choice model did not affect the results presented above, and there was no effect of the treatment on any of them (linear regression taking into account the variability across subjects: all P > 0.41), except on the consistency parameter (β(consistency) = 0.265 ± 0.088, t(10) = 3.00, P = 0.01) consistency (see online resource ESM_3 for subjects’ full model parameters estimates).
Effect of clonidine on reaction times
Together, our analyses therefore revealed two effects of clonidine on behavior: it dose-dependently increased both choice consistency (as captured by the model-based analysis) and choice reaction times. We then examined the relation between these two effects across treatments and animals. We found a positive correlation (linear regression taking into account variability across subjects, β = 0.321 ± 0.095, t(10) = 3.47, p < 0.01) between the estimated consistency parameter and the choice reaction time (Fig. 5c). This correlation between the effect of treatments on reaction time and choice consistency suggests that clonidine affects a single functional entity, which we will refer as the “speed-consistency trade-off.”
Effect of clonidine on motivation: willingness to work
We next examined the effect of clonidine on animals’ trial-by-trial willingness to work trial as a function of the upcoming effort cost and future reward size. To do this, we first examined willingness to work on the first squeeze of all sequences. We included every trial, since there was no way for monkeys to predict at the start of a sequence if a choice was going to be offered later in that sequence. We again examined the influence of reward size and sequence length on the willingness to perform the first squeeze using a linear regression taking into account the variability across subjects. This analysis revealed a significant negative effect of sequence length (i.e., the animals were less willing to engage on long sequences: β = − 18.81 ± 2.44, t(104) = − 7.71, p < 0.001) and a marginally significant positive effect of reward size (animals were more willing to engage for greater reward: β = − 4.79 ± 2.44, t(104) = 1.96, P = 0.052), but no effect of dose of clonidine (t(104) = 1.43, P = 0.16) and no interaction with sequence length (t(102) = 0.20, p = 0.84) or reward (t(102) = − 0.65, p = 0.52). We also looked at the effect of clonidine on reaction times during the first squeeze of the sequence and found a positive effect of sequence length (multi-level linear regression, β = 0.08 ± 0.04, t(104) = 2.05, p = 0.04) but no effect of reward size (t(104) = 0.08, p = 0.93). We found a positive main effect of dose (β = 0.09 ± 0.03, t(104) = 2.93, p = 0.004) but no interaction with sequence length and reward size (both p > 0.40). Hence, the treatment did not interfere with subjects’ evaluation of whether or not to engage in the sequence.
We then examined the influence of clonidine on adjustments to willingness to work across the steps of a trial (Fig. 6b). In all treatment conditions, monkeys displayed a sharp increase in their willingness to work after the first squeeze. We fitted the curves depicted in Fig. 6b with Eq. 4 for all trials during which no choice was offered. In this model, kintercept controls the intercept (initial willingness to work) and kslope the slope of the rise of the willingness to work across the sequence. We then examined the influence of reward and sequence length on each of these parameter estimates using a multi-level linear regression taking into account the variability across monkeys. The parameter kintercept displayed a significant positive linear effect of sequence length (β = 0.160 ± 0.019, t(104) = 8.09, p < 0.001) and a negative effect of reward size (β = − 0.048 ± 0.019, t(104) = − 2.44, p < 0.05). Importantly, however, there was again no significant linear effect of dose (t(104) = − 1.85, p = 0.07) and no interaction with either the sequence length (t(102) = 0.32, p = 0.75) or the reward size (t(102) = − 0.12, p = 0.91). Moreover, neither the task factors (reward and sequence length) nor the dose of clonidine and its interaction with the tasks factors affected the parameter kslope, which captured the slope of the change in willingness to work across the sequence (all p > 0.30).
In short, while monkeys’ willingness to start and persist with an action sequence was sensitive to the sequence length and reward size on each trial, none of these parameters were affected by clonidine.
Effect of clonidine on motivation: force production
Lastly, we examined the effect of clonidine on another key component of motivation: force production. As shown in Fig. 6c, force peak was significantly decreased under clonidine treatment (linear regression, t(10) = − 3.24, p < 0.01).
As described in an earlier section, clonidine also increased overall the reaction times (linear regression, t(10) = 2.63, P = 0.02). Therefore, we considered the possibility that clonidine was having non-specific motivational effect, for instance on arousal or vigilance, which would then be responsible for both longer reaction times and smaller force peaks. In such a scenario, we might therefore expect a strong relation between the effects of clonidine on force peak and reaction time.
To assess whether the effect of dose was the same on the two measures, we performed a two-way ANOVA on z-scored force peak and reaction as the dependent variables. Independent variables were dose and measure type (force peak vs. reaction time). We found a significant main effect of dose (F3,16 = 5.25, P = 0.01) and measure (F1,16 = 4.72, p = 0.04), but importantly also a significant interaction between the two (F3,16 = 8.59, p = 0.001), indicating that the effect of clonidine differs significantly between the two measures. We further explored this using a linear regression (Fig. 6d); there was no reliable correlation between the two measures (r(10) = − 0.4, p = 0.19). Moreover, we ran separate linear regressions excluding the highest dose of clonidine, where the effect on reaction times appeared most prominently (see Fig. 5a). While the effect on force peak was still significant even when only analyzing the low and mid doses (t(7) = − 3.93, p = 0.005), this was not the case for reaction time (t(7) = 0.16, p = 0.88). This was coherent with a stronger linear effect of dose on force peak (β = − 0.222 ± 0.068) than on reaction time (β = 0.106 ± 0.040). Therefore, effort production appears more sensitive to clonidine manipulations than any reaction time measures.
Finally, given that we had found a relationship between changes in reaction times and choice consistency under different doses of clonidine, we explored whether there was any similar connection between each animals’ average force peak and consistency parameter. However, we found no significant linear relationship between measures (t(10) = − 1.39, p = 0.15). Together, this demonstrates that clonidine has a specific, separable, and dose-dependent influence on different aspects of motivation and that the effect of clonidine on force production and choice behavior (as indexed by the monkeys’ speed-consistency trade-off) is at least partially dissociable.
We used a novel decision-making task, in which monkeys make sequential actions for reward and, on most trials, choose whether to stay with their current sequence or to switch to an alternative based on the costs and benefits of the options. Using systemic injections of clonidine, at doses known to decrease LC activity (Kawahara et al. 1999; Fernández-Pastor et al. 2005), we showed that noradrenaline was causally involved in controlling both the variability in choices (which can be interpreted as a simple form of behavioral flexibility) and the response speed. We also showed that noradrenaline was involved in regulating the intensity of the force produced, in line with a potential role in promoting effortful actions. Importantly, the effects on force were distinct to those on response speed, and both measures were dissociable from animals’ willingness to work, which remained unaffected at all doses of clonidine. Similarly, the dose-dependent increase in choice consistency under clonidine occurred in the absence of any changes in the rate of switching choices. These data therefore confirm, refine, and potentially reconcile recent theories of implicating noradrenaline in a behavioral flexibility and motivation, beyond its known general influence on vigilance and arousal.
Overall, we found a coherent effect of the treatment across the three subjects. For instance, in spite of a difference in their initial bias for the current versus the alternative option, we found that the treatment had no effect on this aspect of the behavior. The only potential difference was that we only found a marginally significant effect of the treatment in monkey E. This could be due to a subtle difference between the effect of treatment in monkey E and the others. Indeed, monkey E is a small young female (as opposed to two heavy adult males), and it is possible that the pharmacokinetic of the drug was different in this animal. Critically, however, she shows the same effect of the drug on our main measures of interest (reaction time, force, and choice consistency) as the two other animals.
The first major effect of clonidine treatment was to decrease choice variability, thus extending the causal role of noradrenaline to value-based decision-making. With increasing doses of clonidine, monkeys became increasingly more likely to repeat the same choice when presented with the same pairs of options throughout a session. This was confirmed by our model-based analysis, where we separated the evaluation from the option selection components (Padoa-Schioppa 2011). These two processes were controlled by three parameters: kcost-benefit for the evaluation, bias, and consistency for the option selection. This model-based approach linked the behavioral change induced by clonidine with an increase of the consistency parameter, independent of the valuation process, and the bias parameter. Hence, clonidine induced an increase in choice consistency rather than a systematic bias of the choices in any direction (such as by reward, cost, or current vs. alternative option). This is in line with the results of three recent studies. First, specifically enhancing LC inputs to rat anterior cingulate cortex triggered behavioral variation (Tervo et al. 2014). Second, systemic clonidine injections increased decisiveness in rats performing a spatial decision-making task by reducing both the deliberative search process and the representation of the unchosen path in the hippocampus (Amemiya and Redish 2016). Finally, chemogenetic LC stimulation increased exploration in rats performing a patch-leaving task, though the effects were less specific than in our experiments, as LC stimulation also affected rats’ participation and performance (Kane et al. 2017). All together, this implies that the action of noradrenaline on choice consistency and potentially exploratory behavior is systematic and generic, rather than dependent upon specific task contingency. Thus, noradrenaline would promote adaptation mainly by making it more likely that animals randomly stumble into a novel option.
Variability in choices is assumed in many models of decision-making, but its functional role remains debated. It has been proposed that it arises from random noise driven by internal neural variability (Wang 2002; Faisal et al. 2008; Drugowitsch et al. 2016), and in that case, noradrenaline would control the amount of noise (Aston-Jones and Cohen 2005). Note that we chose to reason in terms of noradrenergic system activity, because clonidine has been shown to decrease LC firing rate, rather than shift its tonic/phasic firing mode. Moreover, we and others have failed to observe these modes, even in learning tasks (Bouret and Sara 2004; Bouret and Richmond 2015; Kalwani et al. 2014; Joshi et al. 2016). In a constant and deterministic environment, such as in our task, choices are optimal when they follow exactly the values of the options. But in uncertain and dynamic environments, it could be beneficial to try alternative options, should an unexpectedly better alternative appear. In that frame, noradrenaline is thought to control the adaptation to changes in action-outcome contingencies (Bouret and Sara 2005; Aston-Jones and Cohen 2005; Yu and Dayan 2005; Cohen et al. 2007; Nassar et al. 2012; Wilson et al. 2014). Hence, the clonidine-induced increase in choice consistency might decrease efficacy (reward rate) if animals were in a labile or uncertain environment, by hampering the ability to explore potentially better options.
The effect of clonidine on choice consistency was associated with an increase in reaction times, which we called a “speed-consistency trade-off.” In the frame of a drift diffusion model, this effect could be captured by the noise in the accumulation process or the setting of the decision boundaries, factors which are distinguishable by their effects on the variance of decision times. However, our task was not designed to capture decision times as we imposed a waiting period before the monkeys could respond, and our analyses showed that we could not reliably tease apart these two factors (data not shown). Nonetheless, this may be an elegant way in future studies to capture the effects of noradrenaline manipulations described here.
In line with the conclusions of our recent electrophysiological studies (Bouret and Richmond 2015; Varazzani et al. 2015), our direct manipulation of the noradrenergic system also highlighted its causal importance for effort processes. Clonidine dose-dependently reduced the amount of force produced, independent of the influence on reaction time, ruling out a global motor impairment. Moreover, it is also unlikely to be caused by a simple effect on arousal or vigilance, as clonidine had no impact on either the animals’ willingness to work or their ability to switch to the alternative option. The influence of clonidine on force production was also independent of task conditions. Indeed, the amount of force produced was not contingent in this task, and the effect of clonidine was equivalent across reward conditions. Hence, clonidine did not affect incentive processes, as it is often the case with dopaminergic treatments (Denk et al. 2005; Le Bouc et al. 2016; Yohn et al. 2016; Zénon et al. 2016).
Instead, the effect of clonidine on force production seems to be relatively specific to action production, but not necessarily to action initiation or persistence through the sequence as neither the initial choice to engage with the sequence nor the choice in the middle of the sequence was affected by the treatment. At first, this might appear surprising since sequences of actions are often considered to be an effort. But since the minimal force to validate a squeeze was very small and monkeys always succeeded if they initiated the action, physical exertion is unlikely to be a major component of the cost of each trial compared to the time over which the animal had to persist to gain the reward (Minamimoto et al. 2012). This is also in line with recent electrophysiological data in an effort-reward trade-off task showing an activation of LC neurons correlated with the amount of force produced on a grip at the time of executed action, but not when evaluating this option (Varazzani et al. 2015). This, together with work by Kalwani et al. (2014) showing a stronger relation between LC activation and action initiation rather than to decision-making, reinforces the idea that noradrenaline plays a specific role in actually producing the effort, mobilizing energy to face a challenge (Bouret and Richmond 2015; Varazzani et al. 2015). It is intriguing that such a role is complementary yet distinct from the influence of the other major catecholamine, dopamine, which is known to be a key for assessing the value of working through sequences of actions for reward but is perhaps not required to overcome force constraints (Ishiwari et al. 2004; Gan et al. 2010; Pasquereau and Turner 2013; Varazzani et al. 2015; Salamone et al. 2016). A key question for future studies will be to directly contrast the precise roles these neurotransmitters play in effort-based decision-making.
Last, we found that the effects on force production were not correlated with choice consistency, suggesting that noradrenaline plays specific, and at least partially separable, roles in behavioral flexibility and effort motivation. One possibility is that this could be mediated by two sets of LC target networks. Indeed, recent studies in rodents indicate that distinct LC neurons could target specific forebrain regions and support separate behavioral functions (Chandler et al. 2014; Uematsu et al. 2017). It may also relate to the fact that, as well as influencing alpha-2 auto-receptors and reducing central noradrenaline levels (Kawahara et al. 1999; Fernández-Pastor et al. 2005), clonidine also stimulates post-synaptic alpha-2 receptors and thus can alter synaptic transmission in, for example, prefrontal cortex. Note, however, the doses that we used are below those that affect working memory functions in monkeys, which was interpreted as clonidine-only post-synaptic alpha-2 receptors at higher doses (Franowicz and Arnsten 1999).
To conclude, these results demonstrate specific, complementary causal roles for noradrenaline in choice variability and motivation. Nonetheless, both functions might be subsumed under the overarching notion that noradrenaline plays a central role when facing challenges by (i) promoting an increase in behavioral volatility and (ii) mobilizing physical resources to respond to immediate challenges. This proposal relies on the assumption that these two processes are adaptive to solve most challenges. While the former would facilitate adaptation in uncertain or changing environments, the latter is clearly advantageous in environments where you must compete for resources and achieve the goals that are set.
We thank Morgan Weissenburger and the animal housing facility staff for their assistance and care of the animals. We also thank Dr. Alizée Lopez for her priceless help in using the drift diffusion model, Dr. Jill O’Reillly for her insights on mixed models, and Nicolas Borderies for discussion.
This research was funded by the ERC BIOMOTIV, the Paris Descartes University doctoral and mobility grants, and the Wellcome Trust fellowships (MEW: 090051 and 202831/Z/16/Z, JS: WT1005651MA).
Compliance with ethical standards
All experimental procedures were designed in association with the Institut du Cerveau et de la Moelle Epiniere (ICM) veterinarians, approved by the Regional Ethical Committee for Animal Experiment (CREEA IDF no. 3), and performed in compliance with the European Community Council Directives (86/609/EEC).
Conflict of interest
The authors declare that they have no competing interests.
- Brodersen KH, Ong CS, Stephan KE, Buhmann JM (2010) The balanced accuracy and its posterior distribution. In: 2010 20th International Conference on Pattern Recognition, pp 3121–3124Google Scholar
- Gee JW de, Colizoli O, Kloosterman NA, Knapen T, Nieuwenhuis S, Donner TH (2017) Dynamic modulation of decision biases by brainstem arousal systems. eLife 6:e23232Google Scholar
- Guedj C, Monfardini E, Reynaud AJ, Farnè A, Meunier M, Hadj-Bouziane F (2016) Boosting norepinephrine transmission triggers flexible reconfiguration of brain networks at rest. Cereb CortexGoogle Scholar
- Kety SS (1972) The possible role of the adrenergic systems of the cortex in learning. Res Publ Assoc Res Nerv Ment Dis 50:376–389Google Scholar
- Yohn SE, Errante EE, Rosenbloom-Snow A, Somerville M, Rowland M, Tokarski K, Zafar N, Correa M, Salamone JD (2016) Blockade of uptake for dopamine, but not norepinephrine or 5-HT, increases selection of high effort instrumental activity: implications for treatment of effort-related motivational symptoms in psychopathology. Neuropharmacology 109:270–280CrossRefGoogle Scholar
- Zénon A, Sidibé M, Olivier E (2014) Pupil size variations correlate with physical effort perception. Front Behav Neurosci 8Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.