A Spatial Small-World Graph Arising from Activity-Based Reinforcement

  • Markus Heydenreich
  • Christian HirschEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11631)


In the classical preferential attachment model, links form instantly to newly arriving nodes and do not change over time. We propose a hierarchical random graph model in a spatial setting, where such a time-variability arises from an activity-based reinforcement mechanism. We show that the reinforcement mechanism converges, and prove rigorously that the resulting random graph exhibits the small-world property. A further motivation for this random graph stems from modeling synaptic plasticity.


Random tree Reinforcement Neural network Small-world graph 



The authors thank all anonymous referees. We also thank C. Leibold for interesting discussions on the neuro-scientific background of synaptic plasticity and comments on an earlier version of the manuscript.


  1. 1.
    Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Lee, D., Bornschein, J., Lin, Z.: Towards biologically plausible deep learning. CoRR abs/1502.04156 (2015).
  3. 3.
    Bollobás, B., Riordan, O.: The diameter of a scale-free random graph. Combinatorica 24(1), 5–34 (2004)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bonacich, P., Liggett, T.M.: Asymptotics of a matrix valued Markov chain arising in sociology. Stoch. Process. Appl. 104(1), 155–171 (2003)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Couzinié, Y., Hirsch, C.: Infinite WARM graphs I. Weak reinforcement regime (in preparation)Google Scholar
  6. 6.
    Delattre, S., Fournier, N., Hoffmann, M.: Hawkes processes on large networks. Ann. Appl. Probab. 26(1), 216–261 (2016)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Embrechts, P., Klüppelberg, C., Mikosch, T.: Modelling Extremal Events. Springer, Berlin (1997). Scholar
  8. 8.
    Feller, W.: An Introduction to Probability Theory and its Applications, vol. II, 2nd edn. Wiley, New York (1971)zbMATHGoogle Scholar
  9. 9.
    Hirsch, C., Holmes, M., Kleptsyn, V.: Absence of WARM percolation in the very strong reinforcement regime, preprint available at
  10. 10.
    Van Der Hofstad, R., Holmes, M., Kuznetsov, A., Ruszel, W.: Strongly reinforced Pólya urns with graph-based competition. Ann. Appl. Probab. 26(4), 2494–2539 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Holmes, M., Kleptsyn, V.: Infinite WARM graphs. Critical regime (in preparation)Google Scholar
  12. 12.
    Holmes, M., Kleptsyn, V.: Proof of the WARM whisker conjecture for neuronal connections. Chaos 27(4), 043104 (2017)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Jacob, E., Mörters, P.: A spatial preferential attachment model with local clustering. In: Bonato, A., Mitzenmacher, M., Prałat, P. (eds.) WAW 2013. LNCS, vol. 8305, pp. 14–25. Springer, Cham (2013). Scholar
  14. 14.
    Kalisman, N., Silberberg, G., Markram, H.: The neocortical microcircuit as a tabula rasa. Proc. Natl. Acad. Sci. 102(3), 880–885 (2005)CrossRefGoogle Scholar
  15. 15.
    Liggett, T.M., Rolles, S.W.W.: An infinite stochastic model of social network formation. Stoch. Process. Appl. 113(1), 65–80 (2004)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Montague, P.R., Dayan, P., Sejnowski, T.J.: A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16(5), 1936–1947 (1996)CrossRefGoogle Scholar
  17. 17.
    Pemantle, R.: A survey of random processes with reinforcement. Probab. Surv. 4, 1–79 (2007)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Pemantle, R., Skyrms, B.: Network formation by reinforcement learning: the long and medium run. Math. Soc. Sci. 48(3), 315–327 (2004)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Pittel, B.: On a random graph evolving by degrees. Adv. Math. 223(2), 619–671 (2010)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Skyrms, B., Pemantle, R.: A dynamic model of social network formation. Proc. Natl. Acad. Sci. USA 97(16), 9340–9346 (2000)CrossRefGoogle Scholar
  21. 21.
    Stanovich, K.E.: Matthew effects in reading: some consequences of individual differences in the acquisition of literacy. J. Educ. 189(1–2), 23–55 (2009)CrossRefGoogle Scholar
  22. 22.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)zbMATHGoogle Scholar
  23. 23.
    Zhu, T.: Nonlinear Pólya urn models and self-organizing processes. Ph.D. thesis, University of Pennsylvania (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Mathematisches InstitutLudwig-Maximilians-Universität MünchenMünchenGermany
  2. 2.Institut für MathematikUniversität MannheimMannheimGermany

Personalised recommendations