The computational pharmacology of oculomotion
Many physiological and pathological changes in brain function manifest in eye-movement control. As such, assessment of oculomotion is an invaluable part of a clinical examination and affords a non-invasive window on several key aspects of neuronal computation. While oculomotion is often used to detect deficits of the sort associated with vascular or neoplastic events; subtler (e.g. pharmacological) effects on neuronal processing also induce oculomotor changes. We have previously framed oculomotor control as part of active vision, namely, a process of inference comprising two distinct but related challenges. The first is inferring where to look, and the second is inferring how to implement the selected action. In this paper, we draw from recent theoretical work on the neuromodulatory control of active inference. This allows us to simulate the sort of changes we would expect in oculomotor behaviour, following pharmacological enhancement or suppression of key neuromodulators—in terms of deciding where to look and the ensuing trajectory of the eye movement itself. We focus upon the influence of cholinergic and GABAergic agents on the speed of saccades, and consider dopaminergic and noradrenergic effects on more complex, memory-guided, behaviour. In principle, a computational approach to understanding the relationship between pharmacology and oculomotor behaviour affords the opportunity to estimate the influence of a given pharmaceutical upon neuronal function, and to use this to optimise therapeutic interventions on an individual basis.
KeywordsActive inference Bayesian Computational pharmacology Neuromodulation Oculomotion
Oculomotor behaviour relies upon the coordination of a distributed network of regions throughout the brain (Parr and Friston 2017a; Robinson 1968). Assessment of oculomotion therefore offers a simple (non-invasive) way to measure brain function. While disruption of normal neurological (Anderson and MacAskill 2013) or psychiatric (Lipton et al. 1983) function can induce a range of characteristic eye-movement deficits; subtler modulations of neuronal function may also be detected in oculomotion. In this paper, we focus upon the neurochemical aspects of oculomotor control, and the sorts of oculomotor syndromes that may be induced by therapeutic agents (Naicker et al. 2017; Reilly et al. 2008). In doing so, we draw from recent theoretical work addressing the computational anatomy of oculomotion (Parr and Friston 2018a; Parr and Friston 2018c) and emerging themes in computational accounts of neuromodulation (Friston et al. 2014; Marshall et al. 2016; Parr et al. 2018; Parr and Friston 2017b; Sales et al. 2018; Schwartenbeck et al. 2015). These accounts are based upon the idea that the brain uses a generative model to infer the causes of its sensations, and that this model is equipped with beliefs1 about the precision (inverse variance) of the relationships between different kinds of latent (i.e. unobserved) variables generating sensory (i.e. observed) samples. The precision of a belief can be thought of as the confidence in that belief (as opposed to its content). As such, precisions are generally associated with neuromodulatory influences over synaptic gain (Feldman and Friston 2010; Marder and Thirumalai 2002; Nadim and Bucher 2014), as opposed to driving postsynaptic responses (i.e. modulating transmembrane conductance as opposed to depolarisation).
We have previously argued for an association (i) between acetylcholine and beliefs about how precisely hidden variables in the world give rise to sensory data, (ii) between noradrenaline and beliefs about how hidden variables in the present cause those in the future, and (iii) between dopamine and beliefs about how we will act upon the world (Friston et al. 2014; Parr and Friston 2017b). In what follows, we first provide an overview of oculomotion in terms of these three kinds of precision, their associated neurotransmitter systems, and active inference. We then introduce a simple delay period oculomotor task—of the sort used extensively in primate electrophysiological studies (Funahashi 2015). Through manipulating various precision terms, we will see that the resulting oculomotor syndromes reproduce those induced by pharmacological agents acting upon their associated neurochemical systems. The implication here is that if one can generate pathological eye movements from selective deficits in neuromodulatory systems in silico, it is possible to estimate these deficits using empirical observations, such as eye tracking [see Adams et al. (2016) for a proof of principle using slow pursuit eye movements].
Active inference and oculomotor control
In this section, we briefly overview the neuroanatomical networks involved in ocular control, with a special focus on the synapses on which different neurotransmitters are thought to act. In describing this functional anatomy, one can associate these neurotransmitters with putative computational roles. The direct cortical control of eye movements involves predominantly dorsal brain areas, including the frontal eye fields (Künzle and Akert 1977), which communicate with the nearby dorsolateral prefrontal cortex (Buschman and Miller 2007). The former area is thought to represent the position of the eyes (Moore and Fallah 2001), while the latter is associated with the maintenance of beliefs about cued targets (Goldman-Rakic 1987). Like most of the cortex, these areas receive distributed projections from the locus coeruleus and the basal forebrain via the cingulum (Avery and Krichmar 2017; Doya 2008). As such, these cortical regions are modulated by noradrenaline and acetylcholine. Under active inference, noradrenaline is thought to represent the precision of transitions (i.e. confidence in probabilistic beliefs about the dynamics of the world—such as motion and occlusion). This encoding of precision is crucial in prefrontal cortical regions involved in the maintenance of a remembered stimulus, as the persistence of a belief over time rests upon a precise belief that the target does not change between viewing the stimulus and enacting the appropriate response (Parr and Friston 2017c).
Acetylcholine has been linked to the precision of beliefs about how latent or hidden states of the world—that cannot be directly observed (e.g. eye position)—give rise to sensory (visual or proprioceptive) data (Marshall et al. 2016; Moran et al. 2013; Vossel et al. 2014). Loss of acetylcholine, as observed in conditions such as Lewy body dementia, can lead to a failure of sensory data to constrain perceptual inference in the right sort of way. Complex visual hallucinations—namely, false positive perceptual inference (Collerton et al. 2005; Parr et al. 2018)—represent a dramatic example of this failure. In the context of motor control, imprecise predictions about desired movements may lead to a failure of descending predictions (i.e. motor commands) to elicit the predicted proprioceptive signals via motor reflexes.
Delayed oculomotor task
To simulate oculomotor performance under this paradigm, we have to specifying a model of how sensory outcomes are generated by latent or hidden states of the world—and how those states can be changed by selecting particular actions or movements. This generative model then specifies belief updating in a synthetic brain under ideal Bayesian assumptions (see the equations in Fig. 1). The beliefs in question here are expectations about states of the world generating sensations—and the plausible actions that can change those states. Crucially, the synthetic subject believes she will select those actions that maximise the evidence for her model of the world. It transpires that this is the same as selectively sampling in sensory outcomes (e.g. directing saccades to particular parts of the visual field) that resolve uncertainty. This resolution of uncertainty comes in two flavours. First, the information gained by sampling new observations and, second, ensuring that these observations are consistent with the generative model (i.e. conform to prior preferences). In the following generative model, we have to deal with two sorts of states, namely, discrete and continuous states. Discrete states correspond to different locations, different stages of each trial etc., while continuous states refer to things like eye position and velocity.
The generative model we use to simulate this task uses a Markov decision process (MDP) model of discrete states (Friston et al. 2015) that generates a set of hypothetical saccadic targets. Each of these hypotheses represents the equilibrium (attracting) point (Feldman and Levin 2009) in a continuous state-space model (described in detail in (Parr and Friston 2018a)) of the eyes themselves. The MDP part of the model comprises three types of hidden (unobservable) state—that jointly generate discrete predictions for the continuous part of model dealing with continuous oculomotor trajectories (Friston et al. 2017b). The discrete states generating predictions include the current fixation location, the target location, and the current stage of the trial. The last of these states includes the target presentation, delay period, and saccade-to-target stages. Fixation location is a controllable state, meaning that any of the five (fixation cross, up, down, left, right) locations may change to any other location depending upon the saccade selected. The target location is static over time, ensuring it is the same at the end of the trial as it was at the beginning. It is this enduring context that gives rise to the delay period activity—thought to be supported by recurrent glutamatergic connections in Layer III of the cortex (Kritzer and Goldman-Rakic 1995)—characteristic of the prefrontal cortex. The third hidden state models transition to the next stage of the trial, at each time-step.
During the fixation stage of the trial, the visual outcome indicates the target location. During the delay period, the cross turns red and no cues are shown. An outcome ‘incorrect’ occurs if a saccade is performed during this step. A priori, this outcome is not preferred and is therefore avoided. At the final stage of the trial, the cross changes from back to blue, and a saccade is permitted. ‘Correct’ outcomes ensue if the fixation location hidden state at this time matches the target location, and ‘incorrect’ pursues otherwise. Prior preferences dictate that ‘correct’ outcomes mark a particular saccade as more likely (in terms of maximising the model evidence or minimising free energy expected following a saccade). At all stages, the proprioceptive outcome is generated through an identity mapping from the fixation location (i.e. the subject has precise sensory evidence about where she is looking). We do not explicitly model free free-viewing during inter-trial intervals and assume that feedback is given immediately following the response. Equipped with this model, we can now examine the effects of changing the precisions (i.e. simulated neuromodulators) on task performance.
Cholinergic or anticholinergic effects are common to many drug classes (Campbell et al. 2009; Ness et al. 2006) and also represent an important mode of action of several toxins (e.g. organophosphate pesticides (Minton and Murray 1988)). As with the benzodiazepines, cholinergic effects have been associated with the velocity of a saccadic eye movement (Naicker et al. 2017). This is interesting from the perspective of the scheme in Fig. 1, as cholinergic modulation is hypothesised to occur at the level of inference about categorical variables (which saccade to perform and which location is the target). It is not immediately obvious how such inferences could influence continuous variables such as velocity. In addition to its cortical site of action, acetylcholine is vital in the normal function of the striatum and has actions on brainstem nuclei directly (Dautan et al. 2014; Kobayashi and Isa 2002; Maurice et al. 2015). The cholinergic (precision) manipulations illustrated in Fig. 3 suggest that these categorical inferences can influence velocity in a consistent way. As cholinergic transmission increases so does the peak saccade velocity. This is an example of a functional diaschisis (Carrera and Tononi 2014; Fornito et al. 2015; Price et al. 2001), in which altering one part of a network has implications for all other parts. The effect here is due to the fact that the precision of the state-outcome mapping determines the precision of the predictive distribution over alternative saccadic locations. If this distribution becomes less precise, the expected (average) anticipated location in continuous coordinates becomes a mixture of all of the possible locations, weighted by their relative probability. When the precision is low, this means saccades towards more central locations become more probable and that, as precision increases, the anticipated location will move further towards a specific target.
Hyoscine (a.k.a. scopolamine), an antimuscarinic (anticholinergic) drug (Corallo et al. 2009) used to treat motion sickness, slows the velocity of saccades (Oliva et al. 1993), consistent with Fig. 3. Interestingly, it additionally causes saccades to become hypo-metric and impairs the stability of fixation. These effects are shown clearly in Fig. 3, with lower levels of cholinergic signalling leading to shorter saccades, and an increase in the number of inappropriate saccades. The latter are due to the fact that precision sharpens or flattens the distribution of plausible saccadic targets. As this distribution becomes flatter, saccades previously deemed inappropriate acquire plausibility. Agonists of the cholinergic system, notably nicotine, improve the performance of saccade (specifically ‘anti-saccade’) tasks if performance is suboptimal (e.g. on first exposure to a task) (Rycroft et al. 2006). These effects tend to saturate fairly quickly, implying that nicotine adds no additional benefit (or deficit) when task performance has already been optimised—and this may be why some studies report no improvement or changes in velocity with nicotine administration (Sherr et al. 2002). This is consistent with the saturation of responses we see in our simulations, with increasing levels of precision asymptotically approaching optimal saccadic trajectories. It is encouraging that these studies show similar results to the computationally focal manipulations performed in our simulations, despite the fact that these drugs do not act in an anatomically specific way.
These simulations provide further face validity to the idea that dopamine is involved in signalling the precision of beliefs about deep policies. Previous theoretical accounts have reproduced aspects of the phenomenology of dopamine signalling based upon this (Friston et al. 2013) and have inspired empirical studies, including the use of simulated precision updates as regressors in functional imaging studies (implicating the dopaminergic midbrain) (Schwartenbeck et al. 2015), and modelling of behavioural responses under pharmacological manipulations (Marshall et al. 2016). A simple experiment that could be performed to further test these ideas would be to fit the model described here to the (saccadic) decisions made in this task (Mirza et al. 2018) and to see whether the prior precision over policies estimated from real participants correlates with their spontaneous blink rate—a peripheral manifestation of central dopamine function (Karson 1983).
Intuitively, an inability to project precise beliefs into the future, or from the past to the present, should undermine the performance of this task. Under active inference, a simple explanation for the preservation of performance even in the absence of precise beliefs about transitions rests upon the use of deep (sequential) policies. If we are able to infer, based upon early observations, the course of action we will pursue, performance becomes robust to the degradation of memories about those observations. In other words, if I know I have to perform a saccade to the left location, whether I believe that this is the target location or not has no influence over my task performance. This illustrates the dissociation between beliefs about states of the world and beliefs about ‘how I will act’. This explanation, and the redundancy it implies, lends itself to an empirical hypothesis. Under a noradrenergic blockade, the sensitivity of task performance to dopamine depletion should increase. Similarly, increasing noradrenaline or dopamine signalling should be able to rescue impaired performance due to depletion of the other transmitter. This sort of hypothesis, concerning an interaction between precision terms, highlights an important future research direction. While, for simplicity, we have highlighted the behavioural or physiological responses that build an intuition about the role of precision parameters, these quantitative behaviours also enable us to plot the responses shown above alongside any of the manipulations employed here. This affords an opportunity to investigate the interaction between neuromodulators and the influence of, for example, noradrenergic modulations on dopamine firing.
Interestingly, selective noradrenaline uptake inhibitors have been shown to be efficacious in treating anxiety disorders (Montoya et al. 2016). These drugs increase signalling at α2-receptors (Grandoso et al. 2004), implying that anxiety may be partly mitigated through an induction of the belief that environmental dynamics are more precise. Under the view that stress is a manifestation of uncertainty (Peters et al. 2017), this conclusion makes a great deal of sense. The reduction in uncertainty about what will happen next, by setting up a belief that the world is actually quite predictable, may be an important part of the computational mechanism of action of these pharmacological agents.
In the above, we have demonstrated the face validity of the use of active inference to simulate the effects of pharmacological therapies on oculomotor behaviour. Previous accounts of these behaviours have proposed their utility as biomarkers for the action of therapeutic (or toxic) agents in individual patients (de Visser et al. 2001; de Visser et al. 2003; Reilly et al. 2008). Complementing this approach, we offer a mechanistic (computational) account that bridges the gap between chemical and behavioural changes. The advantage of casting this in computational terms is that the model used here can be fit to empirical (eye-tracking) data to estimate the changes in precision brought about by specific drugs (Adams et al. 2016; Mirza et al. 2018; Schwartenbeck and Friston 2016). This offers a mechanistically informed method for non-invasive evaluation of synaptic function in individual patients. In doing so, it may be possible to titrate drug doses to achieve an optimal change in central nervous system function or to avoid adverse psychopharmacological effects.
Part of the strength of this method is the appeal to behaviours that depend upon inferences in two different, but connected domains (Parr and Friston 2018c). These are categorical decisions between alternative saccadic targets that depend upon working memory and delay period activity, and the continuous implementation of these decisions through oculomotion. This means that it is possible to draw inferences, based upon behaviour, about the function of anatomically disparate brain regions using a single model. An important caveat here is that the model we have used is overly simple from a pharmacological perspective. Notably, we have neglected the fact that neuromodulatory compounds act at many different anatomical sites and have different effects on different receptor subtypes. These omissions undoubtedly have important computational consequences. This may be why, although we have captured some aspects of the oculomotor changes induced by different drugs, there are others that are not reproduced. For example, the changes in speed of saccades associated with dopaminergic changes (Lynch et al. 1997) were not seen here. Despite these limitations, the correspondence between drug effects and the behaviours resulting from changes to precision parameters adds further weight to computational accounts of neuromodulatory systems and offers a tool to evaluate these theoretical accounts empirically.
This paper offers a computational perspective on the influence of commonly used drugs on behaviour. We considered oculomotor behaviour as a specific example that is known to vary with drug administration, and that is easy to measure with non-invasive techniques. The simulations presented above illustrate that some of the key features of oculomotor responses to pharmacological interventions can be replicated in silico through an appeal to active inference. This rests upon the idea that planning is inference about how to act, and that these inferences entail predictions about the sensory consequences of action. Each stage of this process is sensitive to the precision associated with the relationship between different kinds of variable, and these precisions are thought to manifest biologically as synaptic gain—subject to neuromodulatory chemicals. Ultimately, we hope that this approach will be useful in a clinical setting, enabling quantitative characterisations of pharmacologically induced synaptic modulations using non-invasive measures.
Although the generative model changes from application to application, the belief updates described in this article are generic and can be implemented using standard routines (here spm_MDP_VB_X.m). These routines are available as Matlab code in the SPM academic software: http://www.fil.ion.ucl.ac.uk/spm/. Simulations of the sort reported above can be reproduced (and customized) via a graphical user interface by typing in >>DEM and selecting ‘Decisions to movements’ and associated demos.
In this article, beliefs are used in a non-propositional sense to denote a posterior probability distribution. In other words, beliefs are used technically in the sense of Bayesian belief updating and belief propagation.
TP is supported by the Rosetrees Trust (Award Number 173346). KJF is a Wellcome Principal Research Fellow (Ref: 088130/Z/09/Z).
Conflict of interest
The authors declare that they have no conflict of interest.
- Campbell N, Boustani M, Limbil T, Ott C, Fox C, Maidment I, Schubert CC, Munger S, Fick D, Miller D, Gulati R (2009) The cognitive impact of anticholinergics: a clinical review. Clin Interv Aging 4:225–233Google Scholar
- Chan F, Armstrong IT, Pari G, Riopelle RJ, Munoz DP (2005) Deficits in saccadic eye-movement control in Parkinson’s disease. Neuropsychologia 43:784–796. https://doi.org/10.1016/j.neuropsychologia.2004.06.026 CrossRefGoogle Scholar
- Dautan D, Huerta-Ocampo I, Witten IB, Deisseroth K, Bolam JP, Gerdjikov T, Mena-Segovia J (2014) A major external source of cholinergic innervation of the striatum and nucleus accumbens originates in the brainstem. J Neurosci 34:4509–4518. https://doi.org/10.1523/JNEUROSCI.5071-13.2014 CrossRefGoogle Scholar
- Feldman H, Friston K (2010) Attention, uncertainty, and free-energy. Front Hum Neurosci 4. https://doi.org/10.3389/fnhum.2010.00215
- Friston K, Schwartenbeck P, Fitzgerald T, Moutoussis M, Behrens T, Dolan R (2013) The anatomy of choice: active inference and agency. Front Hum Neurosci 7. https://doi.org/10.3389/fnhum.2013.00598
- Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. Comprehensive PhysiologyGoogle Scholar
- Griffin CE 3rd, Kaye AM, Bueno FR, Kaye AD (2013) Benzodiazepine pharmacology and central nervous system-mediated effects. Ochsner J 13:214–223Google Scholar
- Ness J, Hoth A, Barnett MJ, Shorr RI, Kaboli PJ (2006) Anticholinergic medications in community-dwelling older veterans: prevalence of anticholinergic symptoms, symptom burden, and adverse drug events. Am J Geriatr Pharmacother 4:42–51. https://doi.org/10.1016/j.amjopharm.2006.03.008 CrossRefGoogle Scholar
- O’Driscoll GA, Dépatie L, Holahan A-LV, Savion-Lemieux T, Barr RG, Jolicoeur C, Douglas VI (2005) Executive functions and methylphenidate response in subtypes of attention-deficit/hyperactivity disorder. Biol Psychiatry 57:1452–1460. https://doi.org/10.1016/j.biopsych.2005.02.029 CrossRefGoogle Scholar
- Parr T, Friston KJ (2017a) The active construction of the visual world. Neuropsychologia 104:92–101. https://doi.org/10.1016/j.neuropsychologia.2017.08.003 CrossRefGoogle Scholar
- Parr T, Friston KJ (2017b) Uncertainty, epistemics and active inference Journal of The Royal Society Interface 14Google Scholar
- Parr T, Friston KJ (2018a) Active inference and the anatomy of oculomotion. Neuropsychologia. https://doi.org/10.1016/j.neuropsychologia.2018.01.041
- Parr T, Friston KJ (2018b) The anatomy of inference: generative models and brain structure. Front Comput Neurosci 12. https://doi.org/10.3389/fncom.2018.00090
- Parr T, Friston KJ (2018c) The discrete and continuous brain: from decisions to movement—and back again. Neural Comput:2319–2347Google Scholar
- Parr T, Benrimoh DA, Vincent P, Friston KJ (2018) Precision and false perceptual inference. Front Integr Neurosci 12. https://doi.org/10.3389/fnint.2018.00039
- Sales AC, Friston KJ, Jones MW, Pickering AE, Moran RJ (2018) Locus Coeruleus tracking of prediction errors optimises cognitive flexibility: an Active Inference model bioRxiv:340620Google Scholar
- Schwartenbeck P, Friston K (2016) Computational phenotyping in psychiatry: a worked example eNeuro 3:ENEURO0049-0016.2016 doi: https://doi.org/10.1523/ENEURO.0049-16.2016
- Suzuki TW, Tanaka M (2017) Causal role of noradrenaline in the timing of internally generated saccades in monkeys. Neuroscience 366:15–22. https://doi.org/10.1016/j.neuroscience.2017.10.003 CrossRefGoogle Scholar
- Veale R, Hafed ZM, Yoshida M (2017) How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling. Philos Trans R Soc B: Biol Sci 372Google Scholar
- de Visser SJ, van der Post JP, de Waal PP, Cornet F, Cohen AF, van Gerven JMA (2003) Biomarkers for the effects of benzodiazepines in healthy volunteers. Br J Clin Pharmacol 55:39–50. https://doi.org/10.1046/j.1365-2125.2002.t01-10-01714.x CrossRefGoogle Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.