Skip to main content

Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia

Abstract

Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. 1.

    Byrne JH, Baxter DA, Buonomano DV, Cleary LJ, Eskin A, Goldsmith JR, McClendon E, Nazif FA, Noel F, Scholz KP (1991) Neural and molecular bases of nonassociative and associative learning in Aplysia. Ann N Y Acad Sci 627:124–149

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    Glanzman DL (2006) The cellular mechanisms of learning in Aplysia: of blind men and elephants. Biol Bull 210:271–279

    Article  PubMed  Google Scholar 

  3. 3.

    Hawkins RD, Kandel ER, Bailey CH (2006) Molecular mechanisms of memory storage in Aplysia. Biol Bull 210:174–191

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Reynolds GS (1975) A primer of operant conditioning, 2nd edn. Scott Foresman, Glenview

    Google Scholar 

  5. 5.

    Thorndike EL (1911) Animal intelligence: experimental studies. Macmillan, New York

    Google Scholar 

  6. 6.

    Skinner BF (1938) The behavior of organisms: an experimental analysis. Appleton-Century-Crofts, New York

    Google Scholar 

  7. 7.

    Cook DG, Carew TJ (1986) Operant conditioning of head-waving in Aplysia. Proc Natl Acad Sci USA 83:1120–1124

    CAS  Article  PubMed  Google Scholar 

  8. 8.

    Susswein AJ, Schwarz M, Feldman E (1986) Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. J Neurosci 6:1513–1527

    CAS  PubMed  Google Scholar 

  9. 9.

    Skinner BF (1981) Selection by consequences. Science 213:501–504

    CAS  Article  PubMed  Google Scholar 

  10. 10.

    Brembs B (2003) Operant conditioning in invertebrates. Curr Opin Neurobiol 13:710–717

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8:1481–1489

    CAS  Article  PubMed  Google Scholar 

  12. 12.

    Pitman RK (1989) Animal models of compulsive behavior. Biol Psychiatry 26:189–198

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Cardinal RN, Everitt BJ (2004) Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol 14:156–162

    CAS  Article  PubMed  Google Scholar 

  14. 14.

    Kelley AE (2004) Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev 27:765–776

    Article  PubMed  Google Scholar 

  15. 15.

    Balleine BW (2005) Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86:717–730

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ (2009) Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res 199:89–102

    Article  PubMed  Google Scholar 

  17. 17.

    Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal striatum in reward and decision-making. J Neurosci 27:8161–8165

    CAS  Article  PubMed  Google Scholar 

  18. 18.

    Hoyle G (1980) Learning, using natural reinforcement, in insect preparations that permit cellular neuronal analysis. J Neurobiol 11:323–354

    CAS  Article  PubMed  Google Scholar 

  19. 19.

    Wustmann G, Rein K, Wolf R, Heisenberg M (1996) A new paradigm for operant conditioning of Drosophila melanogaster. J Comp Physiol A 179:429–436

    CAS  Article  PubMed  Google Scholar 

  20. 20.

    Spencer GE, Syed NI, Lukowiak K (1999) Neural changes after operant conditioning of the aerial respiratory behavior in Lymnaea stagnalis. J Neurosci 19:1836–1843

    CAS  PubMed  Google Scholar 

  21. 21.

    Gardner D (1977) Interconnections of identified multiaction interneurons in buccal ganglia of Aplysia. J Neurophysiol 40:349–361

    CAS  PubMed  Google Scholar 

  22. 22.

    Susswein AJ, Byrne JH (1988) Identification and characterization of neurons initiating patterned neural activity in the buccal ganglia of Aplysia. J Neurosci 8:2049–2061

    CAS  PubMed  Google Scholar 

  23. 23.

    Plummer MR, Kirk MD (1990) Premotor neurons B51 and B52 in the buccal ganglia of Aplysia californica: synaptic connections, effect on ongoing motor rhythms, and peptide modulation. J Neurophysiol 63:539–558

    CAS  PubMed  Google Scholar 

  24. 24.

    Hurwitz I, Susswein AJ (1996) B64, a newly identified central pattern generator element producing a phase switch from protraction to retraction in buccal motor programs of Aplysia californica. J Neurophysiol 75:1327–1344

    CAS  PubMed  Google Scholar 

  25. 25.

    Hurwitz I, Kupfermann I, Susswein AJ (1997) Different roles of neurons B63 and B34 that are active during the protraction phase of the buccal motor programs in Aplysia californica. J Neurophysiol 78:1305–1319

    CAS  PubMed  Google Scholar 

  26. 26.

    Kabotyanski EA, Baxter DA, Byrne JH (1998) Identification and characterization of catecholaminergic neuron B65, which initiates and modifies patterned activity in the buccal ganglia of Aplysia. J Neurophysiol 79:605–621

    CAS  PubMed  Google Scholar 

  27. 27.

    Jing J, Weiss KR (2002) Interneuronal basis of the generation of related but distinct motor programs in Aplysia: implications for current neuronal models of vertebrate interlimb coordination. J Neurosci 22:6228–6238

    CAS  PubMed  Google Scholar 

  28. 28.

    Jing J, Cropper EC, Hurwitz I, Weiss KR (2004) The construction of movement with behavior-specific and behavior-independent modules. J Neurosci 24:6315–6325

    CAS  Article  PubMed  Google Scholar 

  29. 29.

    Elliott CJH, Susswein AJ (2002) Comparative neuroethology of feeding control in molluscs. J Exp Biol 205:877–896

    CAS  PubMed  Google Scholar 

  30. 30.

    Kupfermann I (1974) Feeding behavior in Aplysia: a simple system for the study of motivation. Behav Biol 10:1–26

    CAS  Article  PubMed  Google Scholar 

  31. 31.

    Nargeot R, Petrissans C, Simmers J (2007) Behavioral and in vitro correlates of compulsive-like food-seeking induced by operant conditioning in Aplysia. J Neurosci 27:8059–8070

    CAS  Article  PubMed  Google Scholar 

  32. 32.

    Botzer D, Markovich S, Susswein AJ (1998) Multiple memory processes following training that a food in inedible in Aplysia. Learn Mem 5:204–219

    CAS  PubMed  Google Scholar 

  33. 33.

    Alberini CM, Ghirardi M, Metz R, Kandel ER (1994) C/EBP is an immediate-early gene required for the consolidation of long-term facilitation in Aplysia. Cell 76:1099–1114

    CAS  Article  PubMed  Google Scholar 

  34. 34.

    Taubenfeld SM, Milekic MH, Monti B, Alberini CM (2001) The consolidation of new but not reactivated memory requires hippocampal C/EBPβ. Nat Neurosci 4:813–818

    CAS  Article  PubMed  Google Scholar 

  35. 35.

    Levitan D, Lyons LC, Perelman A, Green CL, Motro B, Eskin A, Susswein AJ (2008) Training with inedible food in Aplysia causes expression of C/EBP in the buccal but not cerebral ganglion. Learn Mem 15:412–416

    CAS  Article  PubMed  Google Scholar 

  36. 36.

    Brembs B, Lorenzetti FD, Reyes FD, Baxter DA, Byrne JH (2002) Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science 296:1706–1709

    CAS  Article  PubMed  Google Scholar 

  37. 37.

    Lechner HA, Baxter DA, Byrne JH (2000) Classical conditioning of feeding in Aplysia: I. Behavioral Analysis. J Neurosci 20:3369–3376

    CAS  PubMed  Google Scholar 

  38. 38.

    Nargeot R, Baxter DA, Byrne JH (1997) Contingent-dependent enhancement of rhythmic motor patterns: an in vitro analog of operant conditioning. J Neurosci 17:8093–8105

    CAS  PubMed  Google Scholar 

  39. 39.

    Nargeot R, Baxter DA, Byrne JH (1999) In vitro analog of operant conditioning in Aplysia. I. Contingent reinforcement modifies the functional dynamics of an identified neuron. J Neurosci 19:2247–2260

    CAS  PubMed  Google Scholar 

  40. 40.

    Nargeot R, Baxter DA, Byrne JH (1999) In vitro analogue of operant conditioning in Aplysia. II. Modifications of the functional dynamics of an identified neuron contribute to motor pattern selection. J Neurosci 19:2261–2272

    CAS  PubMed  Google Scholar 

  41. 41.

    Brembs B, Baxter DA, Byrne JH (2004) Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation. Learn Mem 11:412–420

    Article  PubMed  Google Scholar 

  42. 42.

    Lechner HA, Baxter DA, Byrne JH (2000) Classical conditioning of feeding in Aplysia: II. Neurophysiological correlates. J Neurosci 20:3377–3386

    CAS  PubMed  Google Scholar 

  43. 43.

    Lorenzetti FD, Mozzachiodi R, Baxter DA, Byrne JH (2006) Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron. Nat Neurosci 9:17–19

    CAS  Article  PubMed  Google Scholar 

  44. 44.

    Lorenzetti FD, Baxter DA, Byrne JH (2008) Molecular mechanisms underlying a cellular analog of operant reward learning. Neuron 59:815–828

    CAS  Article  PubMed  Google Scholar 

  45. 45.

    Mozzachiodi R, Lorenzetti FD, Baxter DA, Byrne JH (2008) Changes in neuronal excitability serve as a mechanism of long-term memory for operant conditioning. Nat Neurosci 11:1146–1148

    CAS  Article  PubMed  Google Scholar 

  46. 46.

    Morton DW, Chiel HJ (1993) In vivo buccal nerve activity that distinguishes ingestion from rejection can be used to predict behavioral transitions in Aplysia. J Comp Physiol A 172:17–32

    CAS  Article  PubMed  Google Scholar 

  47. 47.

    Briggman KL, Abarbanel HDI, Kristan WB (2005) Optical imaging of neuronal populations during decision-making. Science 307:896–901

    CAS  Article  PubMed  Google Scholar 

  48. 48.

    Kristan WB (2008) Neuronal decision-making circuits. Curr Biol 18:R928–R932

    CAS  Article  PubMed  Google Scholar 

  49. 49.

    Schall JD (2005) Decision making. Curr Biol 15:R9–R11

    CAS  Article  PubMed  Google Scholar 

  50. 50.

    Nargeot R, Baxter DA, Patterson GW, Byrne JH (1999) Dopaminergic synapses mediate neuronal changes in an analogue of operant conditioning. J Neurophysiol 81:1983–1987

    CAS  PubMed  Google Scholar 

  51. 51.

    Mozzachiodi R, Byrne JH (2010) More than synaptic plasticity: role of nonsynaptic plasticity in learning and memory. Trends Neurosci 33:17–26

    CAS  Article  PubMed  Google Scholar 

  52. 52.

    Schwarz M, Susswein AJ (1986) Identification of the neural pathway for reinforcement of feeding when Aplysia learn that food is inedible. J Neurosci 6:1528–1536

    CAS  PubMed  Google Scholar 

  53. 53.

    Horn CC, Zhurov Y, Orekhova IV, Proekt A, Kupfermann I, Weiss KR, Brezina V (2004) Cycle-to-cycle variability of neuromuscular activity in Aplysia feeding behavior. J Neurophysiol 92:157–180

    Article  PubMed  Google Scholar 

  54. 54.

    Reyes FD, Mozzachiodi R, Baxter DA, Byrne JH (2005) Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: blockade by a dopamine antagonist. Learn Mem 12:216–220

    Article  PubMed  Google Scholar 

  55. 55.

    Nargeot R, Cornet C, Simmers J (2008) Dopaminergic modulation of oscillatory membrane properties of buccal motor-pattern initiating neurons and their electrical coupling in Aplysia. FENS Abstr 4:021.13

    Google Scholar 

  56. 56.

    Balleine BW, Liljeholm M, Ostlund SB (2009) The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res 199:43–52

    Article  PubMed  Google Scholar 

  57. 57.

    Kemenes G, Staras K, Benjamin PR (2001) Multiple types of control by identified interneurons in a sensory-activated rhythmic motor pattern. J Neurosci 21:2903–2911

    CAS  PubMed  Google Scholar 

  58. 58.

    Nargeot R, Le Bon-Jego M, Simmers J (2009) Cellular and network mechanisms of operant learning–induced compulsive behaviour in Aplysia. Curr Biol 19:975–984

    CAS  Article  PubMed  Google Scholar 

  59. 59.

    Chay TR (1996) Electrical bursting and luminal calcium oscillation in excitable cell models. Biol Cybern 75:419–431

    CAS  Article  PubMed  Google Scholar 

  60. 60.

    Selverston AI, Rabinovich MI, Abarbanel HDI, Elson R, Szücs A, Pinto RD, Huerta R, Varona P (2000) Reliable circuits from irregular neurons: a dynamical approach to understanding central pattern generators. J Physiol (Paris) 94:357–374

    CAS  Article  Google Scholar 

  61. 61.

    Varona P, Torres JJ, Huerta R, Abarbanel HDI, Rabinovich MI (2001) Regularization mechanisms of spiking-bursting neurons. Neural Netw 14:865–875

    CAS  Article  PubMed  Google Scholar 

  62. 62.

    Falcke M, Huerta R, Rabinovich MI, Abarbanel HDI, Elson RC, Selverston AI (2000) Modeling observed chaotic oscillations in bursting neurons: the role of calcium dynamics and IP3. Biol Cybern 82:517–527

    CAS  Article  PubMed  Google Scholar 

  63. 63.

    Lewis TJ, Rinzel J (2000) Self-organized synchronous oscillations in a network of excitable cells coupled by gap junctions. Network 11:299–320

    CAS  Article  PubMed  Google Scholar 

  64. 64.

    Placantonakis DG, Bukovsky AA, Aicher SA, Kiem H-P, Welsh JP (2006) Continuous electrical oscillations emerge from a coupled network: a study of the inferior olive using lentiviral knockdown of connexin36. J Neurosci 26:5008–5016

    CAS  Article  PubMed  Google Scholar 

  65. 65.

    Lasater EM (1987) Retinal horizontal cell gap junctional conductance is modulated by dopamine through a cyclic AMP-dependent protein kinase. Proc Natl Acad Sci USA 84:7319–7323

    CAS  Article  PubMed  Google Scholar 

  66. 66.

    Yang XD, Korn H, Faber DS (1990) Long-term potentiation of electrotonic coupling at mixed synapses. Nature 348:542–545

    CAS  Article  PubMed  Google Scholar 

  67. 67.

    Johnson BR, Peck JH, Harris-Warrick RM (1993) Amine modulation of electrical coupling in the pyloric network of the lobster stomatogastric ganglion. J Comp Physiol A 172:715–732

    CAS  Article  PubMed  Google Scholar 

  68. 68.

    Kokarovtseva L, Jaciw-Zurakiwsky T, Mendizabal Arbocco R, Frantseva MV, Perez Velazquez JL (2009) Excitability and gap junction-mediated mechanisms in nucleus accumbens regulate self-stimulation reward in rats. Neuroscience 159:1257–1263

    CAS  Article  PubMed  Google Scholar 

  69. 69.

    Schultz W (1997) Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 7:191–197

    CAS  Article  PubMed  Google Scholar 

  70. 70.

    Schultz W (2010) Dopamine signals for reward value and risk: basic and recent data. Behav Brain Funct 23:6–24

    Google Scholar 

  71. 71.

    Wise RA, Rompre PP (1989) Brain dopamine and reward. Annu Rev Psychol 40:191–225

    CAS  Article  PubMed  Google Scholar 

  72. 72.

    Wise RA (2004) Dopamine and food reward: back to the elements. Am J Physiol Regul Integr Comp Physiol 286:R13

    CAS  PubMed  Google Scholar 

  73. 73.

    Di Chiara G, Bassareo V (2007) Reward system and addiction: what dopamine does and doesn’t do. Curr Opin Pharmacol 7:69–76

    CAS  Article  PubMed  Google Scholar 

  74. 74.

    Dalley JW, Everitt BJ (2009) Dopamine receptors in the learning, memory and drug reward circuitry. Semin Cell Dev Biol 20:403–410

    CAS  Article  PubMed  Google Scholar 

  75. 75.

    Fulton S (2010) Appetite and reward. Front Neuroendocrinol 31:85–103

    Article  PubMed  Google Scholar 

  76. 76.

    Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427

    CAS  Article  PubMed  Google Scholar 

  77. 77.

    Le Moal M, Simon H (1991) Mesocorticolimbic dopaminergic network: Functional and regulatory roles. Physiol Rev 71:155–234

    CAS  PubMed  Google Scholar 

  78. 78.

    Ettenberger A (1989) Dopamine, neuroleptics and reinforced behavior. Neurosci Biobehav Rev 13:105–111

    Article  Google Scholar 

  79. 79.

    Smith-Roe SL, Kelley AE (2000) Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci 20:7737–7742

    CAS  PubMed  Google Scholar 

  80. 80.

    Liu Z, Richmond BJ, Murray EA, Saunders RC, Steenrod S, Stubblefield BK, Montague DM, Ginns EI (2004) DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward. Proc Natl Acad Sci USA 101:12336–12341

    CAS  Article  PubMed  Google Scholar 

  81. 81.

    Fadok JP, Dickerson TMK, Palmiter RD (2009) Dopamine is necessary for cue-dependent fear conditioning. J Neurosci 29:11089–11097

    CAS  Article  PubMed  Google Scholar 

  82. 82.

    Martínez-Rubio C, Serrano GE, Miller MW (2009) Localization of biogenic amines in the foregut of Aplysia californica: catecholaminergic and serotonergic innervation. J Comp Neurol 514:329–342

    Article  PubMed  Google Scholar 

  83. 83.

    Ascher P (1972) Inhibitory and excitatory effects of dopamine on Aplysia neurons. J Physiol (London) 225:173–209

    CAS  Google Scholar 

  84. 84.

    Swann JW, Sinback CN, Carpenter DO (1978) Evidence for identified dopamine motor neurons to the gill of Aplysia. Neurosci Lett 10:275–280

    CAS  Article  PubMed  Google Scholar 

  85. 85.

    Wright NJ, Walker RJ (1984) The possible site of action of 5-hydroxytryptamine, 6-hydroxytryptamine, tryptamine and dopamine on identified neurons in the central nervous system of the snail, Helix aspersa. Comp Biochem Physiol C 78:217–225

    CAS  Article  PubMed  Google Scholar 

  86. 86.

    Teyke T, Rosen SC, Weiss KR, Kupfermann I (1993) Dopaminergic neuron B20 generates rhythmic neuronal activity in the feeding motor circuitry of Aplysia. Brain Res 630:26–237

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Romuald Nargeot.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nargeot, R., Simmers, J. Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia . Cell. Mol. Life Sci. 68, 803–816 (2011). https://doi.org/10.1007/s00018-010-0570-9

Download citation

Keywords

  • Aplysia
  • Feeding
  • Operant conditioning
  • Learning
  • Central pattern generator
  • Endogenous bursting
  • Synaptic plasticity