Entropy Based Metrics of Sensory Motor Coordination

A Short Survey
  • Fabio BonsignorioEmail author
Part of the Cognitive Systems Monographs book series (COSMOS, volume 36)


This chapter reviews and summarises the main metrics of sensory motor coordination based on Shannon Entropy. Shannon Entropy and derived metrics provide useful tools for the study of processes that are inherently stochastic, multi-variated and continuous. They allow to associate measures of information and information content changes when the uncertainty associated to a process variates. This is crucial when studying embodied intelligent processes when thanks to mechanisms, such as morphological computation, part of the information processing is outsourced to the systems body dynamics. We are not describing all possible methods to measure sensory-motor coordination. Instead, we focus on the methods that can be applied to the study of the emergence of coordinated ‘intelligent’ behaviours in loosely coupled networks of agents (LCNA). In particular we briefly discuss the methods that allow to study a specific kind of emergent coordinated processes known by the IDSO acronym: Information Driven Self Organization. Moreover, as Information Theoretic metrics are still now not widely known in the Robotics and AI communities we start by shortly introducing the basic related metrics.


  1. 1.
    Amos, M.: Theoretical and Experimental DNA Computation. Springer, Berlin (2005)Google Scholar
  2. 2.
    Asano, F., Yamakita, M., Furuta, K.: Virtual passive dynamic walking and energy-based control laws. In: IEEE/RSJ Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2000) (2000)Google Scholar
  3. 3.
    Banzhaf, W., Yamamoto, L.: Artificial Chemistries. MIT Press, Cambridge (2015)Google Scholar
  4. 4.
    Barabasi, A.: Network Science. Cambridge University Press, Cambridge (2016)Google Scholar
  5. 5.
    Barnett, L., Barrett, A.B., Seth, A.K.: Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett. 103(23) (2009)Google Scholar
  6. 6.
    Bennequin, D., Fuchs, R., Berthoz, A., Flash, T.: Movement timing and invariance arise from several geometries. PLoS Comput Biol. 5(7) (2009)Google Scholar
  7. 7.
    Bongard, J.: Morphological change in machines accelerates the evolution of robust behavior. Proc. Natl. Acad. Sci. 108(4), 1234–1239 (2011)Google Scholar
  8. 8.
    Bonsignorio, F.: On the stochastic stability and observability of controlled serial kinematic chains. In: Proceedings of the ASME 2010 10th Biennial Conference on Engineering Systems Design and Analysis ESDA 2010, Istanbul (2010)Google Scholar
  9. 9.
    Bonsignorio, F.: Quantifying the evolutionary self-structuring of embodied cognitive networks. Artif. Life (MIT Press) 19(2) (2013)Google Scholar
  10. 10.
    Bonsignorio, F.P.: Preliminary considerations for a quantitative theory of networked embodied intelligence. In: Lungarella, M., et al. (eds.) 50 Years of AI, Festschrift. LNAI, vol. 4850, pp. 112–123. Springer, Berlin (2007)Google Scholar
  11. 11.
    Bonsignorio, F.P.: On some information metrics of intelligent material systems. In: ASME ESDA 2008, Haifa (2008)Google Scholar
  12. 12.
    Bonsignorio, F.P.: Steps to a cyber-physical model of networked embodied anticipatory behavior. In: Pezzulo, G., et al. (eds.) ABiALS 2008. LNAI, vol. 5499, pp. 77–94. Springer, Berlin (2009)Google Scholar
  13. 13.
    Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2, 14 (1986)Google Scholar
  14. 14.
    Brudno, A.A.: Entropy and the Complexity of the Trajectories of a Dynamical System. Transactions of the Moscow Mathematical Society, Moscow (1983)Google Scholar
  15. 15.
    Burfoot, D., Lungarella, M., Kuniyoshi, Y.: Toward a theory of embodied statistical learning, SAB 2008, Osaka (2008)Google Scholar
  16. 16.
    Chaitin, G.: Proving Darwin: Making Biology Mathematical. Vintage, New York (2012)Google Scholar
  17. 17.
    Chen, H.L., Doty, D., Soloveichik, D.: Deterministic function computation with chemical reaction networks (2013). arXiv:1204.4176
  18. 18.
    Chen, H.L., Doty, D., Soloveichik, D.: Deterministic function computation with chemical reaction networks. Nat. Comput. 13(4) (2013). International Workshop on DNA-Based ComputersGoogle Scholar
  19. 19.
    Chirikjian, G.S.: Information theory on Lie-groups and mobile robotics application. In: Proceedings of the ICRA 2010, Anchorage, USA (2010)Google Scholar
  20. 20.
    Chirikjian, G.: Stochastic Models, Information Theory, and Lie Groups, vol. 2. Birkhauser, Basel (2011)Google Scholar
  21. 21.
  22. 22.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley, New Jersey (2006)Google Scholar
  23. 23.
    Crutchfield, J.P., Young, K.: Inferring statistical complexity. Phys. Rev. Lett. 63, 105–108 (1989)Google Scholar
  24. 24.
    Der, R.: Selforganized robot behavior from the principle of homeokinesis. In: Gross, H.M., Debes,  K., Bohme, H.-J. (eds), Proceedings of Workshop SOAVE ’2000 (Selbstorganisation von adaptivem Verhalten). Fortschritt-Berichte VDI, vol. 643, Reihe 10, pp. 39–46, Ilmenau: VDI (2000)Google Scholar
  25. 25.
    Der, R.: Self-organized acquisition of situated behavior. Theory Biosci. 120, 179187 (2001)Google Scholar
  26. 26.
    Der, R., Hesse, F., Martius, G.: Rocking stamper and jumping snake from a dynamical system approach to artificial life. J. Adapt. Behav. 14(105), 116 (2005)Google Scholar
  27. 27.
    Der, R., Hesse, F., Martius, G.: Videos of self-organized creatures (2005).
  28. 28.
    Der, R., Martius, G., Hesse, F.: Let it roll emerging sensorimotor coordination in a spherical robot. In: Rocha, L.M. (ed.) Artificial Life X, pp. 192198. MIT Press, Cambridge (2006)Google Scholar
  29. 29.
    Di Paolo, E.A., Cuffari, E.C., De Jaegher, H.: Linguistic Bodies: The Continuity Between Life and Language. MIT Press, Cambridge (2018)Google Scholar
  30. 30.
    Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T.F.G., Qin, C., Zidek, A., Nelson, A., Bridgland, A., Penedones, H., Petersen, S., Simonyan, K., Crossan, S., Jones, D.T., Silver, D., Kavukcuoglu, K., Hassabis, D., Senior, A.W.: De Novo structure prediction with deep-learning based scoring. In: Thirteenth Critical Assessment of Techniques for Protein Structure Prediction (Abstracts) (2018)Google Scholar
  31. 31.
    Fuchslin, R.M., Dzyakanchuk, A., Flumini, D., Hauser, H., Hunt, K.J., Luchsinger, R.H., Reller, B., Scheidegger, S., Walker, R.: Morphological computation and morphological control: steps towards a formal theory and applications. Artif. Life 19(1), 9–34 (2013)Google Scholar
  32. 32.
    Garcia, M., Chatterjee, A., Ruina, A., Coleman, M.: The simplest walking model: stability, complexity, and scaling. J. Biomech Eng. 120(2), 281–8 (1998)Google Scholar
  33. 33.
    Ghazi-Zahedi, K., Ay, N.: Quantifying morphological computation (2013). arXiv:1301.6975
  34. 34.
    Ghazi-Zahedi, K., Ay, N.: Quantifying morphological computation. Entropy 15(5), 1887–1915 (2013)Google Scholar
  35. 35.
    Granger, C.W.J.: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3), 424–438 (1969)Google Scholar
  36. 36.
    Harremoes, P., Tishby, N.: The information bottleneck revisited or how to choose a good distortion measure. In: 2007 Proceedings of the International Symposium on Information Theory (ISIT) (2007)Google Scholar
  37. 37.
    Hauser, H., Ijspeert, A.J., Fchslin, R.M., Pfeifer, R., Maass, W.: Towards a theoretical foundation for morphological computation with compliant bodies. Biol. Cybern. 105(56), 355370 (2011)Google Scholar
  38. 38.
    Heess, N., Dhruva, T.B., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Ali Eslami, S.M., Riedmiller, M., Silver, D.: Emergence of locomotion behaviours in rich environments (2017). arXiv:1707.02286
  39. 39.
    Heess, N., Dhruva, T.B., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Ali Eslami, S.M., Riedmiller, M., Silver, D.: Emergence of locomotion behaviours in rich environments, Youtube Video (2017). Accessed November 2018
  40. 40.
    Iida, F., Gmez, G., Pfeifer, R.: Exploiting body dynamics for controlling a running quadruped robot. In: ICAR’05. Proceedings. 12th International Conference on Advanced Robotics (2005)Google Scholar
  41. 41.
    Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. (Elsevier) 21(4), 642 (2008)Google Scholar
  42. 42.
    Klyubin, A., Polani, D., Nehaniv, C.: Representations of space and time in the maximization of information flow in the perception-action loop. Neural Comput. 19(9), 2387–2432 (2007)Google Scholar
  43. 43.
    Klyubin, A.S., Polani, D., Nehaniv, C.L.: Keep your options open: an information-based driving principle for sensorimotor systems. PLoS ONE 3(12), e4018 (2008)Google Scholar
  44. 44.
    Lampe, A., Chatila, R.: Performance measure for the evaluation of mobile robot autonomy. In: 2006 Proceedings of ICRA (2006)Google Scholar
  45. 45.
    Liljebck, P., Pettersen, K.I., Stavdahl, Ø., Gravdahl, J.T.: Snake Robots: Modelling, Mechatronics, and Control. Springer Science and Business Media, Berlin (2012)Google Scholar
  46. 46.
    Linsker, R.: Self-organization in a perceptual network. Computer 21(3), 105–117 (1988)Google Scholar
  47. 47.
    Linsker, R.: How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Comput. 1(3), 402–411 (1989)Google Scholar
  48. 48.
    Long, A., Wolfe, K., Mashner, M., Chirikjian, G.: The banana distribution is Gaussian: a localization study with wxponential coordinates. In: 2012 Proceedings of Robotics Science and Systems, Sydney (2012)Google Scholar
  49. 49.
    Lungarella, M., Sporns, O.: Mapping information flow in sensorimotor networks. PLOS Comput. Biol. 2(10), 1301–1312 (2006)Google Scholar
  50. 50.
    Olsson, L., Nehaiv, C.L., Polani, D.: Information Trade-Offs and the Evolution of Sensory Layouts. In: Pollack, J., Bedau, M.A., Husbands, P., Ikegami, T., Watson, R.A. (eds.) Artificial Life IX, MIT Press (2004)Google Scholar
  51. 51.
    Pfeifer, R., Bongard, J.: How the Body Shapes the Way We Think: A New View of Intelligence. Bradford Books, Cambridge (2006)Google Scholar
  52. 52.
    Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT Press, Cambridge (1999)Google Scholar
  53. 53.
    Prokopenko, M., Gerasimov, V., Tanev, I.: Evolving spatiotemporal coordination in a modular robotic system. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) From Animals to Animats 9: 9th International Conference on the Simulation of Adaptive Behavior (SAB 2006), Rome, Italy. Lecture Notes in Computer Science, vol. 4095, pp. 558–569. Springer, Berlin (2006)Google Scholar
  54. 54.
    Sandamirskaya, Y., Zibner, S., Schneegans, S., Schoner, G.: Using dynamic field theory to extend the embodiment stance toward higher cognition. New Ideas Psychol. 31, 322–339 (2013)Google Scholar
  55. 55.
    Schoner, G.: Dynamical systems approaches to cognition. In: Sun, R. (ed.) The Cambridge Handbook of Computational Psychology, pp. 101–126. Cambridge University Press, Cambridge (2008)Google Scholar
  56. 56.
    Schoner, G.: Dynamic field theory of embodied cognition. Encyclopedia of Computational Neuroscience. Springer, Berlin (2014)Google Scholar
  57. 57.
    Schreiber, T.: Measuring information transfer. Phys. Review. Lett. 85, 461–464 (2000). arXiv:pdf/nlin.CD/0001042.pdf
  58. 58.
    Shannon, C.E.: The mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)Google Scholar
  59. 59.
    Tanev, I., Ray, T.S., Buller, A.: Automated evolutionary design, robustness, and adaptation of sidewinding locomotion of a simulated snake-like robot. IEEE Trans. Robot. 21(4), 632–645 (2005)Google Scholar
  60. 60.
  61. 61.
    Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. In: Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing, pp 368–377 (1999)Google Scholar
  62. 62.
    Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method (2000). arXiv:physics/0004057
  63. 63.
    Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle (2015). arXiv:1503.02406
  64. 64.
    Toda, M.: Man, Robot, and Society: Models and Speculations. Nijhoff Publisher, Boston (1982)Google Scholar
  65. 65.
    Touchette, H., Lloyd, S.: Information-theoretic approach to the study of control systems. Physica A 331, 140–172 (2003)Google Scholar
  66. 66.
    Wang, Y., Chirikjian, G.: Error propagation on the Euclidean group with applications to manipulator kinematics. IEEE Trans. Robot. 22(4), 591–602 (2006)Google Scholar
  67. 67.
    Wang, S., Sun, S., Li, Z., Zhang, R., Xu, J.: Accurate De Novo prediction of protein contact map by ultra-deep learning model. PLOS Comput. Biol. (2017)Google Scholar
  68. 68.
    Zibner, S.K.U., Faubel, C., Iossifidis, I., Schoner, G.: Dynamic neural fields as building blocks of a cortex-inspired architecture for robotic scene representation. IEEE Trans. Auton. Ment. Dev. 3(1), 74–91 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute of BioroboticsScuola Superiore Sant’AnnaPisaItaly
  2. 2.Heron RobotsGenovaItaly

Personalised recommendations