Editorial: bridging the gap with computational and translational psychopharmacology
The application of theoretical and computational approaches to the analysis of complex behavior has a rich history in psychology. A shining example of this is the modeling of learning encapsulated elegantly by Rescorla and Wagner (1972), who demonstrated that classical conditioning can be described by a simple mathematical equation. The explanatory power of the Rescorla-Wagner rule and its subsequent expansion into additional areas of behavioral plasticity has enabled the precise mapping of learning parameters onto neural structures and even individual neurons. A plethora of other mathematical models have since been used to describe a variety of behaviors, and to map those behaviors onto their underlying neurobiology. This so-called computational phenotyping is now gaining momentum as a translational tool that can be used to identify process characteristics in both humans and animals with the potential of transforming the field of psychopharmacology. The contributions to this Special Issue on Computational and Translational Psychopharmacology stem from the European Behavioural Pharmacology Society (EBPS) Workshop that was held at the University of Cambridge, in August of 2018. The overarching goal of the workshop was to foster discussion around the nascent subfield we refer to as Computational and Translational Psychopharmacology, and to identify points of convergence for which computational approaches could be used to enhance the translational value of animal and human studies. The manuscripts contained herein demonstrate the potential utility of such approaches and provide a foundation for continuous growth towards a better mechanistic understanding of the complex behaviors that characterize psychiatric conditions and the development of more predictive translational probes.
There are several advantages to using mathematical algorithmic approaches to advance a quantitative mechanistic understanding of the processes that underlie animal and human behavior, mental health, and disease. First, process-based hypotheses can be made explicit and quantitative using mathematical models, which increase the precision of underlying theories. Second, competing models of such processes can be directly compared and evaluated based on the evidence provided by empirical data. Third, computational model parameters can be modified based on experimental observations to arbitrate which processes best describe the experimental results. Fourth, individuals exhibiting maladaptive or psychopathological behaviors can be reconceptualized as exhibiting computational “failure modes”, i.e., a constellation of model parameters associated with dysfunctions. These “failure modes” can then be used across diagnostic categories and levels of analyses to reveal common underlying brain mechanisms. Moreover, using computational models for animal behavior can help to determine whether these “failure modes” occur across species. Finally, computational approaches allow us to arbitrate between individual-level analyses and group level analyses, i.e., one can determine whether a group of individuals that are characterized by similar disorder also show similar “failure modes.” Taken together, computational approaches within a translational framework can provide greater explanatory depth, but also the potential for better translational prediction.
The contributions in this special issue highlight some of the emerging (or re-emerging) constructs which provide the basis for current or to-be-developed computational models. A number of these publications extend from the work of Rescorla and Wagner (1972), modeling different aspects of learning. For example, Sebold et al. ( 2019, this issue) use a model-free, or temporal difference, reinforcement learning model to explain the existence of an emotional bias on decision-making. Further, they report that subjects with subclinical depressive symptoms have increased emotional biases, supporting the notion that amplified Pavlovian influences on action selection may serve as a vulnerability factor for mood disorders (e.g., (Nord et al. 2018). In relation, Walters et al. (2019, this issue) describe how decision-making on an approach-avoidance conflict task in rodents can be modeled as a partially observable Markov Decision Process and characterize a potential relationship between anxiety-like behavior, the effect of diazepam on hippocampal theta oscillations, and hippocampal representations of the future. These examples illustrate the utility of such models for delineating inferred states of an individual (human or animal) and associated brain mechanisms, ultimately aiding in diagnosis and treatment.
As an alternative to decision-making and reinforcement learning models, Gu et al. (2019, this issue) introduce interoceptive inference as a candidate framework for modeling psychopathology. Interoceptive inference is the degree to which internal homeostatic processes influence cognitive and affective dysfunctions. Here, the feasibility of using interoceptive inference to model the psychopathology of subjective states is illustrated using drug-craving as an example. The report by Lim et al. (2019, this issue) supports the notion that computational models of drug addiction should extend beyond reinforcement learning. Specifically, different constructs likely contribute to different phases of the addiction process. In particular, Lim et al. emphasize the transition from goal-directed behavior during the initiation of drug use to habitual behavior associated with persistent drug use. The persistence of drug-taking behavior that characterizes addiction has also been postulated to result from increased risk-preference and/or insensitivity to adverse consequences (Bechara 2003, Hester et al. 2013). Langdon et al. (2019, this issue) use computational modeling to demonstrate that the emergence of risk-preferring choice derives from insensitivity to punishment on a rat gambling task. These studies rightfully recognize the need to deconstruct the addiction process in order to capture the mechanisms that may render an individual more susceptible to drug-taking, drug-craving, and relapse. They point to the notion that a “unitary theory of addiction” is likely a misnomer, as different classes of drugs (e.g., opioids versus psychostimulants) can differentially impact experienced and decision utility (Badiani et al., 2019, this issue). Of course, more work is needed to confirm this notion and to fully characterize addictive behaviors; it is hoped that computational models will help expedite the discoveries that will yield successful treatment options for individuals with addiction.
Computational models provide a platform to develop testable and quantifiable hypotheses to relate different levels of analyses, including the impact of pharmacological agents and the role of specific brain structures in psychopathology. In this issue, Weigard et al. (2019) apply evidence accumulation models to determine how methylphenidate modulates the speed accuracy tradeoff on a relatively simple cognitive conflict task. Kanen et al. (2019, this issue) provide a computational account of how perseverative tendencies and associated maladaptive learning strategies differentially contribute to substance use disorder (SUD) and obsessive compulsive disorder (OCD). Furthermore, they relate specific neurotransmitter receptor dysfunction to these component processes, which may inform therapeutic effects. Alsio et al. (2019, this issue) applied computational reinforcement learning algorithms to elucidate the role of D1- and D2-like receptors on reversal learning using a rodent model. To highlight specific brain structures, Murray et al. (2019, this issue) fitted a Q-learning computational model to fMRI prediction error responses to demonstrate abnormally strong signaling in the anterior cingulate cortex in OCD patients during reward omission, and this effect was normalized following administration of dopamine D2/D3 receptor agents.
Beyond exploring relationships across levels of analyses, computational models can provide a quantitative approach to formulate distinct behavioral mechanisms (e.g., Robbins and Cardinal 2019, this issue). Moreover, if parameter estimates indicate changes in behavioral function in an animal paradigm, one can then use a translational approach to determine whether a similar parametric change can be observed in humans. This strategy goes beyond predictive validity (e.g., (Markou et al. 1993), which has relied on changes in observable behavior rather than on changes in model parameters. Yet, there is much room for improvement to develop more sophisticated models that can capture both strategic and tactical behavioral adjustments, and this is only the beginning.
It is becoming increasingly recognized that individual differences should be exploited (rather than ignored) to better inform us of the processes that may render one more or less susceptible to a given disorder or more or less responsive to a certain treatment. There is a need for tasks that allow for greater inter-individual variability; development of such tasks is an important step for future computational models (Palminteri et al. 2017). Such paradigms, constructed with an a-priori model in mind, can be as simple as a two-lever conflict task (e.g., Oberrauch et al. 2019, this issue) or as sophisticated as a multistage behavioral choice task (Sweis et al. 2018). Moreover, it may be worth expanding outcome measures to include outputs such as oculomotor behavior (Parr and Friston 2019, this issue), which could be used in the context of active inference to examine the influence of pharmacological manipulations on sophisticated choice behavior between individuals. Finally, capturing individual differences in animal models can provide a rich experimental framework for translation. One excellent example of this is the sign-tracker/goal-tracker animal model, which reflects individual differences in Pavlovian cue-reward learning and has been associated with individual differences in vulnerability to addiction (Flagel et al. 2009; Huys et al. 2014; Robinson et al. 2014). Computational explanations have been developed that account for both the behavioral and pharmacological components of this animal model (Lesaint et al. 2014a, b), including that put forth in the current issue by Cinotti and colleagues (2019). Thus, it is hoped that this animal model can serve as a blueprint for the application of computational models to enhance our understanding of the behavioral and neural processes that drive behaviors characteristic of psychopathology. Furthermore, such models can be used as a platform for translation to identify those at risk (e.g., Garofalo and di Pellegrino 2015, Joyner et al. 2018) and to determine how interventions alter model parameters to re-regulate behavior and minimize dysfunction.
We have previously argued that explanations and accurate predictions are the fundamental deliverables for a mechanistic or pragmatic approach that academic psychiatric research can provide to stakeholders (Paulus and Thompson 2019). Latent variable approaches such as principal components or factor analysis can be useful unsupervised statistical methods to uncover relationships between variables, within and across units of analyses. However, the underlying assumption is that these latent variables reflect common relationships among all individuals. Instead, it is more likely that relationships differ across individuals and may even differ across states within an individual. Recent approaches to addressing individual variation employ both latent variable and mixture approaches to differentiate subgroups of depressed subjects (Drysdale et al. 2017). These approaches should also be used together with computational models to derive latent variable models of computational processes and to determine individual differences that have important predictive implications. There is a need for a more concerted effort to advance the field, and we have previously argued (Paulus et al. 2016) that achieving this challenging goal will require a systematic, focused approach, akin to that used in drug development. The collection in this special issue represents the breadth of research that could benefit from computational modeling and sets the stage for the advancement of Computational and Translational Psychopharmacology in the twenty-first century.
Funding for the associated European Behavioural Pharmacology Society (EBPS) Workshop – Animal and Human Behavior: Using Computational Approaches to Build a Two-way Bridge – held at the University of Cambridge in August, 2018, was made possible (in part) by 1 R13 DA 046190-01 from the National Institute on Drug Abuse (NIDA). The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention by trade names, commercial practices, or organizations imply endorsement by the U.S. Government. We would also like to acknowledge the University of Michigan Depression Center and Department of Psychiatry for their support of the workshop and the William K Warren Foundation for travel support.
Compliance with ethical standards
Conflict of interest
Dr. Paulus is an advisor to Spring Care, Inc., a behavioral health startup, he has received royalties for an article about methamphetamine in UpToDate and he has received support from the National Institute of General Medical Sciences (P20GM121312, Paulus).
- Alsio J, Phillips BU, Sab-Bayo J, Nilsson SRO, Calafat-Pla TC, Rizwand A, Plumbridge JM, Lopez-Cruz L, Dalley JW, Cardinal RN, Mar AC and Robbins TW (2019). Dopamine D2-like receptor stimulation blocks negative feedback in visual and spatial reversal learning in the rat: behavioural and computational evidence. Psychopharmacology this issueGoogle Scholar
- Badiani, Caprioli and De Pirro (2019) Opposite environmental gating of the experienced utility and decision utility of heroin versus cocaine in animals and humans: implications for computational neuroscience. Psychopharmacology, this issueGoogle Scholar
- Cinotti F, Marchand AR, Roesch MR, Girard B and Khamassi M (2019) Impacts of inter-trial interval duration on a computational model of sign-tracking vs. goal-tracking behaviour. Psychopharmacology, this issueGoogle Scholar
- Drysdale AT, Grosenick L, Downar J, Dunlop K, Mansouri F, Meng Y, Fetcho RN, Zebley B, Oathes DJ, Etkin A, Schatzberg AF, Sudheimer K, Keller J, Mayberg HS, Gunning FM, Alexopoulos GS, Fox MD, Pascual-Leone A, Voss HU, Casey BJ, Dubin MJ, Liston C (2017) Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat Med 23(1):28–38CrossRefGoogle Scholar
- Gu X, FitzGerald THB and Friston KJ (2019). Modeling subjective belief states in computational psychiatry: interoceptive inference as a candidate framework. Psychopharmacology, this issueGoogle Scholar
- Kanen, Ersche, Fineberg, Robbins and Cardinal (2019) Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2/3 receptor agents. Psychopharmacology, this issueGoogle Scholar
- Langdon AJ, Hathaway BA, Zorowitz S, Harris CBW, Winstanley CA (2019) Relative insentivity to time-out punishments induced by win-paired cues in a rat gambling task. Psychopharmacology, this issueGoogle Scholar
- Lesaint F, Sigaud O, Clark JJ, Flagel SB and Khamassi M (2014a). Experimental predictions drawn from a computational model of sign-trackers and goal-trackers. J Physiol ParisGoogle Scholar
- Lim, Cardinal, Savulich, Jones, Moustafa, Robbins, Ersche (2019) Impairments in reinforcement learning do not explain enhanced habit formation in cocaine use disorder. Psychopharmacology, this issueGoogle Scholar
- Murray GK, Knolle F, Ersche KD, Craig KJ, Abbott S, Shabbir SS, Fineberg NA, Suckling J, Sahakian BJ, Bullmore ET and Robbins TW (2019) Dopaminergic drug treatment remediates exaggerated cingulated prediction error responses in obsessive-compulsive disorder. Psychopharmacology, this issueGoogle Scholar
- Oberrauch, Sigrist, Sautter, Bach and Pryce (2019). Establishing operant conflict tests for the translational study of anxiety in mind. Psychopharmacology, this issueGoogle Scholar
- Parr T and Friston KJ (2019) The computational pharmacology of oculomotion. Psychopharmacology, this issueGoogle Scholar
- Paulus MP and Thompson WK (2019) The challenges and opportunities of small effects: the new normal in academic psychiatry. JAMA PsychiatryGoogle Scholar
- Rescorla, Wagner (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II. Appleton-Century-Crofts, pp 64–99Google Scholar
- Robbins and Cardinal (2019) Computational psychopharmacology: a translational and pragmatic approach. Psychopharmacology, this issueGoogle Scholar
- Sebold M, Garbusow M, Jetzschmann P, Schad DJ, Nebe S, Schlagenhauf F, Heinz A, Rapp M and Romanczuk-Seiferth N (2019). Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms. Psychopharmacology, this issueGoogle Scholar
- Walters CJ, Jubran J, Sheehan A, Erickson MT, Redish AD (2019). Avoid-approach conflict behaviours differentially affected by anxiolytics: implications for a computational model of risky decision-making. Psychopharmacology, this issueGoogle Scholar
- Weigard, Heathcote and Sripada (2019). Modeling the effects of methyphenidate on conflict, top-down control and evidence accumulation using the conflict linear ballistic accumulator. Psychopharmacology, this issueGoogle Scholar