Neurobiology of pavlovian and operant conditioning - A narrative review
DOI:
https://doi.org/10.33448/rsd-v13i11.47523Keywords:
Conditioning, Classical; Conditioning, Operant; Neurobiology; Brain; Neurosciences.Abstract
Pavlovian and operant conditioning are forms of associative learning in which humans and other animals learn to associate events or stimuli that occur in sequence or together. Skinner coined the term "operant conditioning" to describe how behavior is shaped by its consequences. Recent advances in neuroscience can provide insights into behavior that were unthinkable in Skinner's time. This study aimed to show a narrative review of the literature on recent advances in understanding the neural bases involved in Pavlovian conditioning and operant conditioning. A literature review was conducted on the neurobiology of Pavlovian and operant conditioning. Studies in invertebrates have highlighted similarities with the forms of learning described in vertebrates by Pavlov, Thorndike, and Skinner. Experiments in simpler nervous systems show that it is possible to advance understanding at the level of neural circuits and molecular mechanisms of psychological concepts inferred from purely behavioral studies.
References
Alberini, C. M., Ghirardl, M., Metz, R., & Kandel, E. R. (1994). C/EBP is an immediate-early gene required for the consolidation of long-term facilitation in Aplysia. Cell, 76 (6), 1099–114. https://doi.org/10.1016/0092-8674(94)90386-7.
Bacskai, B. J., Hochner, B., Mahaut-Smith, M., Adams, S. R., Kaang, B.-K., Kandel, E. R., & Tsien, R. Y. (1993). Spatially Resolved Dynamics of cAMP and Protein Kinase A Subunits in Aplysia Sensory Neurons. Science, 260 (5105), 222–6. https://doi.org/10.1126/science.7682336.
Bailey, C. H., & Chen, M. (1988). Long-term memory in Aplysia modulates the total number of varicosities of single identified sensory neurons. Proceedings of the National Academy of Sciences, 85 (7), 2373–7. https://doi.org/10.1073/pnas.85.7.2373.
Barco, A., Alarcon, J. M., & Kandel, E. R. (2002). Expression of Constitutively Active CREB Protein Facilitates the Late Phase of Long-Term Potentiation by Enhancing Synaptic Capture. Cell, 108 (5), 689–703. https://doi.org/10.1016/S0092-8674(02)00657-8.
Bartsch, D., Ghirardi, M., Casadio, A., Giustetto, M., Karl, K. A., Zhu, H., & Kandel, E. R. (2000). Enhancement of Memory-Related Long-Term Facilitation by ApAF, a Novel Transcription Factor that Acts Downstream from Both CREB1 and CREB2. Cell, 103 (4), 595–608.
https://doi.org/10.1016/S0092-8674(00)00163-X.
Berleant, D., White, M., Pierce, E., Tudoreanu, E., Boeszoermenyi, A., Shtridelman, Y., & Macosko, J. C. (2009). The Genetic Code—More Than Just a Table. Cell Biochemistry and Biophysics, 55 (2), 107–16. https://doi.org/10.1007/s12013-009-9060-9.
Bouton, M. E., Maren, S., & McNally, G. P. (2021). Behavioral and neurobiological mechanisms of pavlovian and instrumental extinction learning. Physiological Reviews, 101 (2), Artigo 2. https://doi.org/10.1152/physrev.00016.2020.
Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant Reward Learning in Aplysia: Neuronal Correlates and Mechanisms. Science, 296 (5573), 1706–9. https://doi.org/10.1126/science.1069434.
Brunelli, M., Castellucci, V., & Kandel, E. R. (1976). Synaptic Facilitation and Behavioral Sensitization in Aplysia: Possible Role of Serotonin and Cyclic AMP. Science, 194 (4270), 1178–81. https://doi.org/10.1126/science.186870.
Byrne, J. H. (Org.). (2019). The Oxford handbook of invertebrate neurobiology. Oxford University Press.
Carew, T., Castellucci, V. F., & Kandel, E. R. (1979). Sensitization in Aplysia: Restoration of Transmission in Synapses Inactivated by Long-Term Habituation. Science, 205 (4404), 417–9. https://doi.org/10.1126/science.451611.
Castellucci, V. F., Carew, T. J., & Kandel, E. R. (1978). Cellular Analysis of Long-Term Habituation of the Gill-Withdrawal Reflex of Aplysia californica. Science, 202 (4374), 1306–8. https://doi.org/10.1126/science.214854.
Catania, A. C. (2007). Learning (4th interim ed). Sloan Pub.
Cavalcante, L. T. C. & Oliveira, A. A. S. (2020). Métodos de revisão bibliográfica nos estudos científicos. Psicol. Rev. 26 (1). https://doi.org/10.5752/P.1678-9563.2020v26n1p82-100
Casarin, S. T., Porto, A. R., Gabatz, R. I. B., Bonow, C. A., Ribeiro, J. P., & Mota, M. S. (2020). Tipos de revisão de literatura: considerações das editoras do Journal of Nursing and Health/Types of literature review: considerations of the editors of the Journal of Nursing and Health. Journal of Nursing and Health,10(5). https://periodicos.ufpel.edu.br/index.php/enfermagem/article/view/19924.
Costa, R. M., Baxter, D. A., & Byrne, J. H. (2022). Neuronal population activity dynamics reveal a low-dimensional signature of operant learning in Aplysia. Communications Biology, 5 (1), 90. https://doi.org/10.1038/s42003-022-03044-1.
Croteau-Chonka, E. C., Clayton, M. S., Venkatasubramanian, L., Harris, S. N., Jones, B. M., Narayan, L., Winding, M., Masson, J.-B., Zlatic, M., & Klein, K. T. (2022). High-throughput automated methods for classical and operant conditioning of Drosophila larvae. eLife, 11, e70015. https://doi.org/10.7554/eLife.70015.
Dash, P. K., Hochner, B., & Kandel, E. R. (1990). Injection of the cAMP-responsive element into the nucleus of Aplysia sensory neurons blocks long-term facilitation. Nature, 345 (6277), 718–21. https://doi.org/10.1038/345718a0.
Denny, C. A., Kheirbek, M. A., Alba, E. L., Tanaka, K. F., Brachman, R. A., Laughman, K. B., Tomm, N. K., Turi, G. F., Losonczy, A., & Hen, R. (2014). Hippocampal Memory Traces Are Differentially Modulated by Experience, Time, and Adult Neurogenesis. Neuron, 83 (1), 189–201. https://doi.org/10.1016/j.neuron.2014.05.018.
Dinsmoor, J. A. (1998). Punishment. In W. T. O'Donohue (Ed.), Learning and behavior therapy (pp. 188-204). Allyn & Bacon.
Dudai, Y. (2012). The Restless Engram: Consolidations Never End. Annual Review of Neuroscience, 35 (1), 227–47. https://doi.org/10.1146/annurev-neuro-062111-150500.
Eichenbaum, H. (2016). Still searching for the engram. Learning & Behavior, 44 (3), 209–22. https://doi.org/10.3758/s13420-016-0218-1.
Eliot, L., Hawkins, R., Kandel, E., & Schacher, S. (1994). Pairing-specific, activity-dependent presynaptic facilitation at Aplysia sensory-motor neuron synapses in isolated cell culture. The Journal of Neuroscience, 14 (1), 368–83. https://doi.org/10.1523/JNEUROSCI.14-01-00368.1994.
Gazzaniga, M. S., Ivry, R. B., & Mangun, G. R. (2019). Cognitive neuroscience: The biology of the mind (Fifth edition). W.W. Norton & Company.
Ginsburg, S., & Jablonka, E. (2010). The evolution of associative learning: A factor in the Cambrian explosion. Journal of Theoretical Biology, 266 (1), 11–20. https://doi.org/10.1016/j.jtbi.2010.06.017.
Glanzman, D. L. (2010). Common Mechanisms of Synaptic Plasticity in Vertebrates and Invertebrates. Current Biology, 20 (1), R31–R36. https://doi.org/10.1016/j.cub.2009.10.023.
Guan, Z., Giustetto, M., Lomvardas, S., Kim, J.-H., Miniaci, M. C., Schwartz, J. H., Thanos, D., & Kandel, E. R. (2002). Integration of Long-Term-Memory-Related Synaptic Plasticity Involves Bidirectional Regulation of Gene Expression and Chromatin Structure. Cell, 111 (4), 483–93. https://doi.org/10.1016/S0092-8674(02)01074-7.
Hawkins, R. D., Abrams, T. W., Carew, T. J., & Kandel, E. R. (1983). A Cellular Mechanism of Classical Conditioning in Aplysia: Activity-Dependent Amplification of Presynaptic Facilitation. Science, 219 (4583), 400–5. https://doi.org/10.1126/science.6294833.
Hayes, S. C., Barnes-Holmes, D., & Roche, B. (Orgs.). (2001). Relational frame theory: A post-Skinnerian account of human language and cognition. Kluwer Academic/Plenum Publishers.
Hayes, S. C., & Hofmann, S. G. (Orgs.). (2018). Process-based CBT: The science and core clinical competencies of cognitive behavioral therapy. New Harbinger Publications, Inc.
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006). Acceptance and Commitment Therapy: Model, processes and outcomes. Behaviour Research and Therapy, 44 (1), Artigo 1. https://doi.org/10.1016/j.brat.2005.06.006.
Hegde, A. N., Inokuchi, K., Pei, W., Casadio, A., Ghirardi, M., Chain, D. G., Martin, K. C., Kandel, E. R., & Schwartz, J. H. (1997). Ubiquitin C-Terminal Hydrolase Is an Immediate-Early Gene Essential for Long-Term Facilitation in Aplysia. Cell, 89 (1), 115–26. https://doi.org/10.1016/S0092-8674(00)80188-9.
Heller, A. S. (2020). From Conditioning to Emotion: Translating Animal Models of Learning to Human Psychopathology. The Neuroscientist, 26 (1), Artigo 1. https://doi.org/10.1177/1073858419866820.
Hsieh, J., & Gage, F. H. (2005). Chromatin remodeling in neural development and plasticity. Current Opinion in Cell Biology, 17 (6), 664–71. https://doi.org/10.1016/j.ceb.2005.09.002.
Ito, M. (2001). Cerebellar Long-Term Depression: Characterization, Signal Transduction, and Functional Roles. Physiological Reviews, 81 (3), 1143–95. https://doi.org/10.1152/physrev.2001.81.3.1143.
Josselyn, S. A., Köhler, S., & Frankland, P. W. (2017). Heroes of the Engram. The Journal of Neuroscience, 37 (18), 4647–57. https://doi.org/10.1523/JNEUROSCI.0056-17.2017.
Kandel, E. R. (2001). The Molecular Biology of Memory Storage: A Dialogue Between Genes and Synapses. Science, 294 (5544), 1030–8. https://doi.org/10.1126/science.1067020.
Kandel, E. R., Dudai, Y., & Mayford, M. R. (2014a). The Molecular and Systems Biology of Memory. Cell, 157 (1), Artigo 1. https://doi.org/10.1016/j.cell.2014.03.001.
Kandel, E. R., Dudai, Y., & Mayford, M. R. (2014b). The Molecular and Systems Biology of Memory. Cell, 157 (1), 163–86. https://doi.org/10.1016/j.cell.2014.03.001.
Laing, P. A. F., Vervliet, B., Fullana, M. A., Savage, H. S., Davey, C. G., Felmingham, K. L., & Harrison, B. J. (2021). Characterizing human safety learning via Pavlovian conditioned inhibition. Behaviour Research and Therapy, 137, 103800. https://doi.org/10.1016/j.brat.2020.103800.
Lammel, S., Lim, B. K., & Malenka, R. C. (2014). Reward and aversion in a heterogeneous midbrain dopamine system. Neuropharmacology, 76, 351–9. https://doi.org/10.1016/j.neuropharm.2013.03.019.
LeDoux, J. (2003). The emotional brain, fear, and amygdala. Cellular and Molecular Neurobiology, 23 (4/5), 727–38. https://doi.org/10.1023/A:1025048802629.
LeDoux, J. E., Moscarello, J., Sears, R., & Campese, V. (2017). The birth, death and resurrection of avoidance: A reconceptualization of a troubled paradigm. Molecular Psychiatry, 22 (1), Artigo 1. https://doi.org/10.1038/mp.2016.166.
Lee, S.-H., Kwak, C., Shim, J., Kim, J.-E., Choi, S.-L., Kim, H. F., Jang, D.-J., Lee, J.-A., Lee, K., Lee, C.-H., Lee, Y.-D., Miniaci, M. C., Bailey, C. H., Kandel, E. R., & Kaang, B.-K. (2012). A cellular model of memory reconsolidation involves reactivation-induced destabilization and restabilization at the sensorimotor synapse in Aplysia. Proceedings of the National Academy of Sciences, 109 (35), 14200–5. https://doi.org/10.1073/pnas.1211997109.
Lewis, R. G., Florio, E., Punzo, D., & Borrelli, E. (2021). The Brain’s Reward System in Health and Disease. Em O. Engmann & M. Brancaccio (Orgs.), Circadian Clock in Brain Health and Disease. 1344, 57–69). Springer International Publishing. https://doi.org/10.1007/978-3-030-81147-1_4.
Liu, X., Ramirez, S., Pang, P. T., Puryear, C. B., Govindarajan, A., Deisseroth, K., & Tonegawa, S. (2012). Optogenetic stimulation of a hippocampal engram activates fear memory recall. Nature, 484 (7394), 381–5. https://doi.org/10.1038/nature11028.
Marco, A., Meharena, H. S., Dileep, V., Raju, R. M., Davila-Velderrain, J., Zhang, A. L., Adaikkan, C., Young, J. Z., Gao, F., Kellis, M., & Tsai, L.-H. (2020). Mapping the epigenomic and transcriptomic interplay during memory formation and recall in the hippocampal engram ensemble. Nature Neuroscience, 23 (12), 1606–17. https://doi.org/10.1038/s41593-020-00717-0.
Martin, K. C., Michael, D., Rose, J. C., Barad, M., Casadio, A., Zhu, H., & Kandel, E. R. (1997). MAP Kinase Translocates into the Nucleus of the Presynaptic Cell and Is Required for Long-Term Facilitation in Aplysia. Neuron, 18 (6), 899–912. https://doi.org/10.1016/S0896-6273(00)80330-X.
Montarolo, P. G., Goelet, P., Castellucci, V. F., Morgan, J., Kandel, E. R., & Schacher, S. (1986). A Critical Period for Macromolecular Synthesis in Long-Term Heterosynaptic Facilitation in Aplysia. Science, 234 (4781), 1249–54. https://doi.org/10.1126/science.3775383.
Moore, J. (1997). Some thoughts on the s-r issue and the relation between Behavior Analysis and Behavioral Neuroscience. Journal of the Experimental Analysis of Behavior, 67 (2), 242–5. https://doi.org/10.1901/jeab.1997.67-242.
Morton, D. W., & Chiel, H. J. (1993). In vivo buccal nerve activity that distinguishes ingestion from rejection can be used to predict behavioral transitions in Aplysia. Journal of Comparative Physiology A, 172 (1), 17–32. https://doi.org/10.1007/BF00214712.
Murphy, G. G., & Glanzman, D. L. (1997). Mediation of Classical Conditioning in Aplysia californica by Long-Term Potentiation of Sensorimotor Synapses. Science, 278 (5337), 467–71. https://doi.org/10.1126/science.278.5337.467.
Nader, K., Schafe, G. E., & Le Doux, J. E. (2000). Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406 (6797), 722–6. https://doi.org/10.1038/35021052.
Nargeot, R., Baxter, D. A., & Byrne, J. H. (1999). In Vitro Analog of Operant Conditioning in Aplysia. I. Contingent Reinforcement Modifies the Functional Dynamics of an Identified Neuron. The Journal of Neuroscience, 19 (6), 2247–60. https://doi.org/10.1523/JNEUROSCI.19-06-02247.1999.
Nargeot, R., Le Bon-Jego, M., & Simmers, J. (2009). Cellular and Network Mechanisms of Operant Learning-Induced Compulsive Behavior in Aplysia. Current Biology, 19 (12), 975–84. https://doi.org/10.1016/j.cub.2009.05.030.
Nargeot, R., Petrissans, C., & Simmers, J. (2007). Behavioral and In Vitro Correlates of Compulsive-Like Food Seeking Induced by Operant Conditioning in Aplysia. The Journal of Neuroscience, 27 (30), 8059–70. https://doi.org/10.1523/JNEUROSCI.1950-07.2007.
Nargeot, R., & Simmers, J. (2011). Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia. Cellular and Molecular Life Sciences, 68 (5), 803–16. https://doi.org/10.1007/s00018-010-0570-9.
Ortega-de San Luis, C., & Ryan, T. J. (2022). Understanding the physical basis of memory: Molecular mechanisms of the engram. Journal of Biological Chemistry, 298 (5), 101866. https://doi.org/10.1016/j.jbc.2022.101866.
Perez, W. F., Kovac, R., de Almeida, J. H., & de Rose, J. C. (2022). Teoria das Molduras Relacionais (RFT)—Conceitos, pesquisa e aplicações. Paradigma.
Phelps, E. A., & Hofmann, S. G. (2019). Memory editing from science fiction to clinical practice. Nature, 572 (7767), 43–50. https://doi.org/10.1038/s41586-019-1433-7.
Ramirez, S., Liu, X., Lin, P.-A., Suh, J., Pignatelli, M., Redondo, R. L., Ryan, T. J., & Tonegawa, S. (2013). Creating a False Memory in the Hippocampus. Science, 341 (6144), 387–91. https://doi.org/10.1126/science.1239073.
Rogan, M. T., Stäubli, U. V., & LeDoux, J. E. (1997). Fear conditioning induces associative long-term potentiation in the amygdala. Nature, 390 (6660), 604–7. https://doi.org/10.1038/37601.
Rother, E. T. (2007). Revisão sistemática x revisão narrativa. Acta paul. enferm. 20 (2). https://doi.org/10.1590/S0103-21002007000200001.
Sara, S. J. (2000). Retrieval and Reconsolidation: Toward a Neurobiology of Remembering. Learning & Memory, 7 (2), 73–84. https://doi.org/10.1101/lm.7.2.73.
Semon, R. (1918). The Mneme. George Allen & Unwin.
Sidman, M. (1994). Equivalence relations and behavior: A research story. Authors Cooperative.
Skinner, B. F. (1965). Science and human behavior (First Free Press Paperback edition). The Free Press.
Skinner, B. F. (1976). About behaviorism. Vintage Books.
Sweatt, J. D. (Org.). (2013). Epigenetic regulation in the nervous system: Basic mechanisms and clinical impact. Academic Press.
Tanaka, K. Z., Pevzner, A., Hamidi, A. B., Nakazawa, Y., Graham, J., & Wiltgen, B. J. (2014). Cortical Representations Are Reinstated by the Hippocampus during Memory Retrieval. Neuron, 84 (2), 347–54. https://doi.org/10.1016/j.neuron.2014.09.037.
Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Columbia University Press. https://doi.org/10.1037/10780-000.
Todorov, J. C. (2007). A Psicologia como o estudo de interações. Psicologia: Teoria e Pesquisa, 23 (spe), 57–61. https://doi.org/10.1590/S0102-37722007000500011.
Von Essen, A. M. H. J., Pauls, D., Thum, A. S., & Sprecher, S. G. (2011). Capacity of visual classical conditioning in Drosophila larvae. Behavioral Neuroscience, 125 (6), 921–9. https://doi.org/10.1037/a0025758.
Walters, E. T., & Byrne, J. H. (1983). Associative Conditioning of Single Sensory Neurons Suggests a Cellular Mechanism for Learning. Science, 219 (4583), 405–8. https://doi.org/10.1126/science.6294834.
Zilio, D. (2016). Who, What, and When: Skinner’s Critiques of Neuroscience and His Main Targets. The Behavior Analyst, 39 (2), 197–218. https://doi.org/10.1007/s40614-016-0053-x.
Zovkic, I. B. (2021). Epigenetics and memory: An expanded role for chromatin dynamics. Current Opinion in Neurobiology, 67, 58–65. https://doi.org/10.1016/j.conb.2020.08.007.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Welington dos Santos Silva
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
