Skip to main content
SpringerLink
Log in

A Brief Introduction to Probabilistic Machine Learning and Its Relation to Neuroscience

  • Chapter
  • First Online:
Growing Adaptive Machines

Part of the book series: Studies in Computational Intelligence ((SCI,volume 557))

  • 1783 Accesses

Abstract

My aim in this chapter is to give a concise summary of what I consider the most important ideas in modern machine learning, and relate to one another different approaches, such as support vector machines and Bayesian networks, or reinforcement learning and temporal supervised learning. I begin with general comments on organizational mechanisms, then focus on unsupervised, supervised and reinforcement learning. I point out the links between these concepts and brain processes such as synaptic plasticity and models of the basal ganglia. Examples for each of the three main learning paradigms are also included to allow experimenting with these concepts.

Available at http://projects.cs.dal.ca/hallab/MLreview2013.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Available at http://code.google.com/p/bnt/, and used to implement Fig. 13; file at www.cs.dal.ca/~tt/repository/MLintro2012/PearlBurglary.m.

  2. 2.

    Markov models are often a simplification or abstraction of a real world. In this section, however, we discuss a “toy world” in which state transitions were designed to fulfill the Markov condition.

  3. 3.

    \(V^\pi (s)\) is usually called the state value function and \(Q^\pi (s, a)\) the state-action value function. Note, however, that the value depends in both cases on the states and the actions taken.

  4. 4.

    This formulation of the Bellman equation for an MDP [36–38] is slightly different from the formulation of Sutton and Barto in [39], as these authors define the value function to be the cumulative reward starting from the next state, not the current state. In their case, the Bellman equation reads \(V^\pi (s) = \sum _{s'} T(s'|s, a) (r(s') + \gamma \, V^\pi (s'))\). This is only a matter of convention about when we consider the prediction: just before getting the current reward of after taking the next step.

  5. 5.

    The same function name is used on both sides of this equation, but these are distinguished by the inclusion of parameters. The value functions all refer to the parametric model, which should be clear from the context.

  6. 6.

    Julian Miller made this point nicely at the aforementioned workshop.

References

  1. S. Geman, E. Bienenstock, R. Doursat, Neural networks and the bias/variance dilemma. Neural Comput. 4(1), 1–58 (1992)

    Article  Google Scholar 

  2. P. Smolensky, Information Processing in Dynamical Systems: Foundations of Harmony Theory, in Parallel Distributed Processing: Volume 1: Foundations, ed. by D.E. Rumelhart, J.L. McClelland (MIT Press, Cambridge, MA, 1986), pp. 194–281

    Google Scholar 

  3. G. Hinton, Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1711–1800 (2002)

    Article  Google Scholar 

  4. G. Hinton, A Practical Guide to Training Restricted Boltzmann Machines. University of Toronto Technical Report UTML TR 2010–003, 2010

    Google Scholar 

  5. A. Graps, An Introduction to Wavelets. http://www.amara.com/IEEEwave/IEEEwavelet.html

  6. N. Huang et al., The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. Lond. A 454, 903–995 (1998)

    Article  MATH  Google Scholar 

  7. H. Barlow (1961) Possible principles underlying the transformation of sensory messages. Sens. Commun. 217–234, (1961)

    Google Scholar 

  8. P. Földiák, Forming sparse representations by local anti-Hebbian learning. Biol. Cybern. 64, 165–170 (1990)

    Article  Google Scholar 

  9. P. Földiák, D. Endres, Sparse coding. Scholarpedia 3, 2984 (2008)

    Article  Google Scholar 

  10. B. Olshausen, D. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Article  Google Scholar 

  11. H. Lee, E. Chaitanya and A. Ng, Sparse deep belief net model for visual area V2, NIPS*2007

    Google Scholar 

  12. C. von der Malsburg, Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14, 85–100 (1973)

    Article  Google Scholar 

  13. S. Grossberg, Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors. Biol. Cybern. 23, 121–134 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  14. T. Kohonen, Self-Organizing Maps (Springer, Berlin, 1994)

    MATH  Google Scholar 

  15. P. Hollensen, P. Hartono, T. Trappenberg (2011) Topographic RBM as Robot Controller, JNNS 2011

    Google Scholar 

  16. S. Grossberg, Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw. 37, 1–47 (2012)

    Article  Google Scholar 

  17. T. Trappenberg, P. Hartono, D. Rasmusson, in Top-Down Control of Learning in Biological Self-Organizing Maps, ed. by J. Principe, R. Miikkulainen. Lecture Notes in Computer Science 5629, WSOM 2009 (Springer, 2009), pp. 316–324

    Google Scholar 

  18. K. Tanaka, H. Saito, Y, Fukada, M. Moriya, Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J. Neurophysiol. 66, 170–189 (1991)

    Google Scholar 

  19. S. Chatterjee, A. Hadi, Sensitivity Analysis in Linear Regression (John Wiley & Sons, New York, 1988)

    Book  MATH  Google Scholar 

  20. Judea Pearl, Causality: Models, Reasoning and Inference (Cambridge University Press, Cambridge, 2009)

    Book  Google Scholar 

  21. D. Cireşan, U. Meier, J. Masci, J. Schmidhuber, Multi-column deep neural network for traffic sign classification. Neural Netw. 32, 333–338 (2012)

    Article  Google Scholar 

  22. D. Rumelhart, G. Hinton, R. Williams, Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)

    Article  Google Scholar 

  23. K. Hornik, Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2), 251–257 (1991)

    Article  Google Scholar 

  24. A. Weigend, D. Rumelhart (1991) Generalization through minimal networks with application to forecasting, ed. by E.M. Keramidas. in Computing Science and Statistics (23rd Symposium INTERFACE’91, Seattle, WA), pp. 362–370

    Google Scholar 

  25. R. Caruana, S. Lawrence, C.L. Giles, Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping, in Proceedings of Neural Information Processing Systems Conference, 2000. pp. 402–408

    Google Scholar 

  26. D.J.C. MacKay, A practical Bayesian framework for backpropagation networks. Neural Comput. 4(3), 448–472 (1992)

    Article  Google Scholar 

  27. D. Silver, K. Bennett, Guest editor’s introduction: special issue on inductive transfer learning. Mach. Learn. 73(3), 215–220 (2008)

    Article  Google Scholar 

  28. S. Pan, Q. Yang, A survey on transfer learning. IEEE Trans. Knowl. Data Eng. (IEEE TKDE) 22(10), 1345–1359 (2010)

    Article  Google Scholar 

  29. B.E. Boser, I.M. Guyon, V. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, (ACM, 1992), pp. 144–152

    Google Scholar 

  30. V. Vapnik, The Nature of Statistical Learning Theory (Springer, Berlin, 1995)

    Book  MATH  Google Scholar 

  31. C. Cortes, V. Vapnik, Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  32. C. Burges, A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2(2), 121–167 (1998)

    Article  Google Scholar 

  33. A. Smola, B. Schölkopf, A tutorial on support vector regression. Stat. Comput. 14(3) (2004)

    Google Scholar 

  34. C.-C. Chang, C.-J. Lin, LibSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/cjlin/libsvm

  35. M. Boardman, T. Trappenberg, A heuristic for free parameter optimization with support vector machines, WCCI 2006, pp. 1337–1344, (2006). http://www.cs.dal.ca/boardman/wcci

  36. E. Alpaydim, Introduction to Machine Learning, 2e (MIT Press, Cambridge, 2010)

    Google Scholar 

  37. S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics (MIT Press, Cambridge, 2005)

    MATH  Google Scholar 

  38. S. Russel, P. Norvigm, Artificial Intelligence: A Modern Approach, 3rd edn. (Prentice Hall, New York, 2010)

    Google Scholar 

  39. R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 1998)

    Google Scholar 

  40. C.J.C.H. Watkins, Learning from Delayed Rewards. Ph.D. thesis, Cambridge University, Cambridge, England, 1989

    Google Scholar 

  41. H. van Hasselt, Reinforcement learning in continuous state and action spaces. Reinforcement Learn.: Adapt. Learn. Optim. 12, 207–251 (2012).

    Google Scholar 

  42. R. Sutton, Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9–44 (erratum p. 377) (1988)

    Google Scholar 

  43. B. Sallans, G. Hinton, Reinforcement learning with factored states and actions. J. Mach. Learn. Res. 5, 1063–1088 (2004)

    MATH  MathSciNet  Google Scholar 

  44. D.O. Hebb, The Organization of Behaviour (John Wiley & Sons, New York, 1949)

    Google Scholar 

  45. E.R. Caianiello, Outline of a theory of thought-processes and thinking machines. J. Theor. Biol. 1, 204–235 (1961)

    Article  MathSciNet  Google Scholar 

  46. T. Trappenberg, Fundamentals of Computational Neuroscience, 2nd edn. (Oxford University Press, Oxford, 2010)

    MATH  Google Scholar 

  47. R. Enoki, Y.L. Hu, D. Hamilton, A. Fine, Expression of long-term plasticity at individual synapses in hippocampus is graded, bidirectional, and mainly presynaptic: optical quantal analysis. Neuron 62(2), 242–253 (2009)

    Article  Google Scholar 

  48. T. Bliss, T. Lømo, Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. 232(2), 331–56 (1973)

    Google Scholar 

  49. D. Heinke, E. Mavritsaki (eds.), Computational Modelling in Behavioural Neuroscience: Closing the gap between neurophysiology and behaviour (Psychology Press, London, 2008)

    Google Scholar 

  50. R. Rescorla, A. Wagner, in A Theory of Pavlovian Conditioning: Variations, in the Effectiveness of Reinforcement and Nonreinforcement, ed. by W.F. Prokasy, A.H. Black, Classical Conditioning, II: Current Research and Theory, (Appleton Century Crofts, New York, 1972), pp. 64–99

    Google Scholar 

  51. W. Schultz, Predictive reward signal of dopamine neurons. J. Neurophysiol. 80(1), 1–27 (1998)

    Google Scholar 

  52. J. Houk, J. Adams, A. Barto in A Model of How the Basal Ganglia Generate and Use Neural Signals that Predict Reinforcement, ed. by J.C. Hauk, J.L. Davis, D.G. Breiser. Models of Information Processing in the Basal Ganglia (MIT Press, Cambridge, 1995)

    Google Scholar 

  53. P. Connor, T. Trappenberg, in Characterizing a Brain-Based Value-Function Approximator, ed. by E. Stroulia, S. Matwin, Advances in Artificial Intelligence LNAI 2056, (Springer, Berlin, 2011), pp. 92–103

    Google Scholar 

  54. J. Reynolds, J. Wickens, Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15(4–6), 507–521 (2002)

    Article  Google Scholar 

  55. P. Connor, V. LoLordo, T. Trappenberg (2012) An elemental model of retrospective revaluation without within-compound associations. Anim. Learn. 42(1), 22–38

    Google Scholar 

  56. T. Maia, M. Frank, From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 14, 154–162 (2011)

    Article  Google Scholar 

  57. Y. Bengio, Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1–127 (2009)

    Article  MATH  Google Scholar 

  58. J. Hawkins, On Intelligence (Times Books, New York, 2004)

    Google Scholar 

  59. G. Gigerenzer, P. Todd and the ABC Research Group, Simple Heuristics that Make Us Smart (Oxford University Press, Oxford, 1999)

    Google Scholar 

Download references

Acknowledgments

I would like to express my thanks to René Doursat for careful edits, Christian Albers, Igor Farkas, and Stephen Grossberg for useful comments of an earlier draft circulation, and all the colleagues that have provided me with encouraging comments.

Author information

Authors and Affiliations

  1. Dalhousie University, Halifax, Canada

    Thomas P. Trappenberg

Authors
  1. Thomas P. Trappenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas P. Trappenberg .

Editor information

Editors and Affiliations

  1. CNRS, Institut des Systèmes Complexes - Paris Île-de-France, Paris, France

    Taras Kowaliw

  2. Institute of Intelligent Systems and Robotics, CNRS UMR 7222, Université Pierre et Marie Curie, Paris, France

    Nicolas Bredeche

  3. School of Biomedical Engineering, Drexel University, Philadelphia, Pennsylvania, USA

    René Doursat

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Trappenberg, T.P. (2014). A Brief Introduction to Probabilistic Machine Learning and Its Relation to Neuroscience. In: Kowaliw, T., Bredeche, N., Doursat, R. (eds) Growing Adaptive Machines. Studies in Computational Intelligence, vol 557. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55337-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-55337-0_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-55336-3

  • Online ISBN: 978-3-642-55337-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation