Abstract
One of the earliest attempts at building a theory of the basal ganglia (BG) is based on the clinical findings that lesions to the direct and indirect pathways of the BG produce quite opposite motor manifestations (Albin et al., in Trends Neurosci 12(10):366–375, 1989). While lesions of the direct pathway (DP), affecting particularly the projections from the striatum to GPi, are associated with hypokinetic disorders (distinguished by a paucity of movement), lesions of the indirect pathway (IP) produce hyperkinetic disorders, such as chorea and tremor. In this chapter, we argue that describing the two BG pathways as having mutually opponent actions has limitations. We argue that the BG indirect pathway also plays a role in exploration. We should evidence from various motor learning and decision-making tasks that exploration is a necessary process in various behavioral processes. Importantly, we use the exploration mechanism explained here to simulate various processes of the basal ganglia which we discuss in the following chapters.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366–375.
Baunez, C., Humby, T., Eagle, D. M., Ryan, L. J., Dunnett, S. B., & Robbins, T. W. (2001). Effects of STN lesions on simple vs choice reaction time tasks in the rat: Preserved motor readiness, but impaired response selection. European Journal of Neuroscience, 13(8), 1609–1616.
Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of Parkinsonism. Journal of Neurophysiology, 72(2), 507–520.
Brown, P., Oliviero, A., Mazzone, P., Insola, A., Tonali, P., & Di Lazzaro, V. (2001). Dopamine dependency of oscillations between subthalamic nucleus and pallidum in Parkinson’s disease. The Journal of Neuroscience, 21(3), 1033–1038.
Brunel, N. (2000). Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons. Journal of Computational Neuroscience 8, 183–208.
Chakravarthy, V. S., Joseph, D., & Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological Cybernetics, 103(3), 237–253. https://doi.org/10.1007/s00422-010-0401-y.
Contreras-Vidal, J., & Stelmach, G. E. (1995). Effects of Parkinsonism on motor control. Life Sciences, 58(3), 165–176.
Crick, F. (1984). Function of the thalamic reticular complex: The searchlight hypothesis. Proceedings of the National Academy of Sciences, 81(14), 4586–4590.
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879.
Flores-Hernandez, J., Cepeda, C., Hernandez-Echeagaray, E., Calvert, C. R., Jokel, E. S., Fienberg, A. A., … Levine, M. S. (2002). Dopamine enhancement of NMDA currents in dissociated medium-sized striatal neurons: Role of D1 receptors and DARPP-32. Journal of Neurophysiol, 88(6), 3010–3020.
Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51–72. https://doi.org/10.1162/0898929052880093.
Gangadhar, G., Joseph, D., & Chakravarthy, V. S. (2008). Understanding Parkinsonian handwriting through a computational model of basal ganglia. Neural Computation, 20(10), 2491–2525.
Gillies, A., Willshaw, D., Atherton, J., & Arbuthnott, G. (2002). Functional interactions within the subthalamic nucleus. In The basal ganglia VII (pp. 359–368). Boston: Springer.
Grabli, D., McCairn, K., Hirsch, E. C., Agid, Y., Féger, J., François, C., et al. (2004). Behavioural disorders induced by external globus pallidus dysfunction in primates: I. Behavioural study. Brain, 127(9), 2039–2054.
Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401–410.
Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1(4), 304–309.
Houk, J. C., Davis, J. L., & Beiser, D. G. (1995). Models of information processing in the basal ganglia. Cambridge: The MIT press.
Humphries, M., & Gurney, K. (2002). The role of intra-thalamic and thalamocortical circuits in action selection. Network: Computation in Neural Systems, 13(1), 131–156.
Humphries, M. D., & Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Progress in Neurobiology, 90(4), 385–417. https://doi.org/10.1016/j.pneurobio.2009.11.003.
Hurtado, J. M., Gray, C. M., Tamas, L. B., & Sigvardt, K. A. (1999). Dynamics of tremor-related oscillations in the human globus pallidus: a single case study. Proceedings of the National Academy of Sciences, 96(4), 1674–1679.
Joel, D., Niv, Y., & Ruppin, E. (2002). Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural Networks, 15(4–6), 535–547.
Kalva, S. K., Rengaswamy, M., Chakravarthy, V. S., & Gupte, N. (2012). On the neural substrates for exploratory dynamics in basal ganglia: A model. Neural Networks, 32, 65–73. https://doi.org/10.1016/j.neunet.2012.02.031.
Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (2000). Principles of neural science (Vol. 4). New York: McGraw-Hill.
Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680.
Kliem, M. A., Maidment, N. T., Ackerson, L. C., Chen, S., Smith, Y., & Wichmann, T. (2007). Activation of nigral and pallidal dopamine D1-like receptors modulates basal ganglia outflow in monkeys. Journal of Neurophysiology, 98(3), 1489–1500.
Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. Journal of Neuroscience, 21(16), 159.
Kravitz, A. V., Freeze, B. S., Parker, P. R., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of Parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature, 466(7306), 622–626.
Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V. S., & Rengaswamy, M. (2011). Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer? Neural Networks, 24(8), 801–813. https://doi.org/10.1016/j.neunet.2011.06.002.
Magdoom, K. N., Subramanian, D., Chakravarthy, V. S., Ravindran, B., Amari, S., & Meenakshisundaram, N. (2011). Modeling basal ganglia for understanding Parkinsonian reaching movements. Neural Computation, 23(2), 477–516. https://doi.org/10.1162/NECO_a_00073.
Magill, P., Bolam, J., & Bevan, M. (2001). Dopamine regulates the impact of the cerebral cortex on the subthalamic nucleus–globus pallidus network. Neuroscience, 106(2), 313–330.
Mandali, A., & Chakravarthy, V. S. (2016). Probing the role of medication, DBS electrode position, and antidromic activation on impulsivity using a computational model of basal ganglia. Frontiers in Human Neuroscience, 10, 450.
Mandali, A., Rengaswamy, M., Chakravarthy, S., & Moustafa, A. A. (2015). A spiking Basal Ganglia model of synchrony, exploration and decision making. Frontiers in Neuroscience, 9, 191.
Michmizos, K. P., & Nikita, K. S. (2011). Local field potential driven Izhikevich model predicts a subthalamic nucleus neuron activity. Paper presented at the Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE.
Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936–1947.
Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Gilat, M., Lewis, S. J., & Moustafa, A. A. (2017). A neurocomputational model of the effect of cognitive load on freezing of gait in Parkinson’s disease. Frontiers in Human Neuroscience, 10, 649. https://doi.org/10.3389/fnhum.2016.00649.
Muralidharan, V., Balasubramani, P. P., Chakravarthy, V. S., Lewis, S. J., & Moustafa, A. A. (2014). A computational model of altered gait patterns in Parkinson’s disease patients negotiating narrow doorways. Frontiers in Computational Neuroscience, 7, 190. https://doi.org/10.3389/fncom.2013.00190.
O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669), 452–454.
Plenz, D., & Kital, S. T. (1999). A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature, 400(6745), 677–682.
Pragathi Priyadharsini, B., Ravindran, B., & Srinivasa Chakravarthy, V. (2012). Understanding the role of serotonin in basal ganglia through a unified model. In A. P. Villa, W. Duch, P. Érdi, F. Masulli, & G. Palm (Eds.), Artificial Neural Networks and Machine Learning—ICANN 2012 (Vol. 7552, pp. 467–473). Berlin: Springer.
Redgrave, P., Prescott, T. J., & Gurney, K. (1999). The basal ganglia: A vertebrate solution to the selection problem? Neuroscience, 89(4), 1009–1023.
Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11(4), 389–397.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599.
Sridharan, D., Prashanth, P. S., & Chakravarthy, V. S. (2006). The role of the basal ganglia in exploration in a neural model based on reinforcement learning. International Journal of Neural Systems, 16(2), 111–124.
Stein, P. S., Grillner, S., Selverston, A. I., & Stuart, D. G. (1997). Neurons, networks, and behavior. Cambridge, MA: MIT Press.
Steiner, H., & Tseng, K. Y. (2010). Handbook of basal ganglia structure and function: A decade of progress (Vol. 20). Access Online via Elsevier.
Sukumar, D., Rengaswamy, M., & Chakravarthy, V. S. (2012). Modeling the contributions of Basal ganglia and Hippocampus to spatial navigation using reinforcement learning. PLoS ONE, 7(10), e47467. https://doi.org/10.1371/journal.pone.0047467.
Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Adaptive computations and machine learning. MIT Press/Bradford.
Terman, D., Rubin, J., Yew, A., & Wilson, C. (2002). Activity patterns in a model for the subthalamopallidal network of the basal ganglia. The Journal of Neuroscience, 22(7), 2963–2976.
Willshaw, D., & Li, Z. (2002). Subthalamic–pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia. Proceedings of the Royal Society of London, Series B: Biological Sciences, 269(1491), 545–551.
Yoshida, W., & Ishii, S. (2006). Resolution of uncertainty in prefrontal cortex. Neuron, 50(5), 781–789.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
The system of equations for single oscillator is given by,
Differentiating (i)
Substituting (ii) and (iii) in (iv)
Substituting (i) and (ii) in (v)
on rearranging
is similar to Lienard’s equation \( \frac{{{\text{d}}^{2} x}}{{{\text{d}}^{2} t}} + \frac{{{\text{d}}x}}{{{\text{d}}t}}f\left( x \right) + g\left( x \right) = 0 \) where \( f\left( x \right) = 2 - \lambda \,\sec \,h^{2} \left( {\lambda x} \right) \), and \( g(x) = x - I \).
Checking for the Lienard’s conditions, let us assume I = 0.
-
1.
Both \( f(x) \) and \( g(x) \) are continuously differentiable for all \( x \).
-
2.
\( g( - x) = - g(x) \) for all \( x \) (i.e., \( g(x) \) is an odd function).
-
3.
\( g(x) > 0\;{\text{for}}\;x > 0 \).
-
4.
For all \( x \)(i.e., \( f(x) \) is an even function); The odd function \( F(x) = \int\nolimits_{0}^{x} {f(u)du} \) = \( 2x - \tan \,h\left( {\lambda x} \right) \) has exactly one positive zero at \( x = x_{o} \), is negative for \( 0 < x < x_{o} \), is positive and non-decreasing for \( x > x_{o} \), and \( F(x) \to \infty \) as \( x \to \infty \) (one can estimate \( x_{o} \) from graph of \( F(x) \)). So the system has a unique stable limit cycle surrounding the origin in the phase plane.
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Srinivasa Chakravarthy, V., Balasubramani, P.P. (2018). The Basal Ganglia System as an Engine for Exploration. In: Computational Neuroscience Models of the Basal Ganglia. Cognitive Science and Technology. Springer, Singapore. https://doi.org/10.1007/978-981-10-8494-2_5
Download citation
DOI: https://doi.org/10.1007/978-981-10-8494-2_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-8493-5
Online ISBN: 978-981-10-8494-2
eBook Packages: EngineeringEngineering (R0)