Neural theory for the perception of causal actions


The efficient prediction of the behavior of others requires the recognition of their actions and an understanding of their action goals. In humans, this process is fast and extremely robust, as demonstrated by classical experiments showing that human observers reliably judge causal relationships and attribute interactive social behavior to strongly simplified stimuli consisting of simple moving geometrical shapes. While psychophysical experiments have identified critical visual features that determine the perception of causality and agency from such stimuli, the underlying detailed neural mechanisms remain largely unclear, and it is an open question why humans developed this advanced visual capability at all. We created pairs of naturalistic and abstract stimuli of hand actions that were exactly matched in terms of their motion parameters. We show that varying critical stimulus parameters for both stimulus types leads to very similar modulations of the perception of causality. However, the additional form information about the hand shape and its relationship with the object supports more fine-grained distinctions for the naturalistic stimuli. Moreover, we show that a physiologically plausible model for the recognition of goal-directed hand actions reproduces the observed dependencies of causality perception on critical stimulus parameters. These results support the hypothesis that selectivity for abstract action stimuli might emerge from the same neural mechanisms that underlie the visual processing of natural goal-directed action stimuli. Furthermore, the model proposes specific detailed neural circuits underlying this visual function, which can be evaluated in future experiments.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8


  1. Allison, T., Puce, A., & McCarthy, G. (2000). Social perception from visual cues: Role of the STS region. Trends in Cognitive Sciences, 4(7), 267–278.

  2. Baker, C. L., Saxe, R., & Tenenbaum, J. B. (2009). Action understanding as inverse planning. Cognition, 113(3), 329–349.

  3. Barraclough, N. E., Keith, R. H., Xiao, D., Oram, M. W., & Perrett, D. I. (2009). Visual adaptation to goal-directed hand actions. Journal of Cognitive Neuroscience, 21(9), 1806–1820.

  4. Barrett, H. C., Todd, P. M., Miller, G. F., & Blythe, P. (2005). Accurate judgments of intention from motion alone: A cross-cultural study. Evolution and Human Behavior, 26, 313–331.

  5. Bassili, F. (1976). Temporal and spatial contingencies in the perception of social events. Journal of Personality and Social Psychology, 33(6), 680–685.

  6. Beardsley, S. A., & Vaina, L. M. (2001). A laterally interconnected neural architecture in MST accounts for psychophysical discrimination of complex motion patterns. Journal of Computational Neuroscience, 10(3), 255–280.

  7. Beasley, N. A. (1968). The extent of individual differences in the perception of causality. Canadian Journal of Psychology, 22(5), 399–407.

  8. Blakemore, S. J., & Decety, J. (2001). From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2(8), 561–567.

  9. Blythe, P. W., Todd, P. M., & Miller, G. F. (1999). How motion reveals intention: Categorizing social interactions. In G. Gigerenzer & P. M. Todd (Eds.), Simple heuristics that make us smart (pp. 257–285). Oxford: Oxford University Press.

  10. Bonaiuto, J., & Arbib, M. A. (2010). Extending the mirror neuron system model, II: What did I just do? A new role for mirror neurons. Biological Cybernetics, 102(4), 341–359.

  11. Brass, M., Schmitt, R. M., Spengler, S., & Gergely, G. (2007). Investigating action understanding: Inferential processes versus action simulation. Current Biology, 17(24), 2117–2121.

  12. Castelli, F., Happe, F., Frith, U., & Frith, C. (2000). Movement and mind: A functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage, 12(3), 314–325.

  13. Chersi, F. (2011). Neural mechanisms and models underlying joint action. Experimental Brain Research, 211(3–4), 643–653.

  14. Choi, H., & Scholl, B. J. (2006). Measuring causal perception: Connections to representational momentum? Acta psychologica (Amst), 123(1–2), 91–111.

  15. Dasser, V., Ulbaek, I., & Premack, D. (1989). The perception of intention. Science, 243(4889), 365–367.

  16. de Lange, F. P., Spronk, M., Willems, R. M., Toni, I., & Bekkering, H. (2008). Complementary systems for understanding action intentions. Current Biology, 18(6), 454–457.

  17. Deco, G., & Rolls, E. T. (2005). Attention, short-term memory, and action selection: A unifying theory. Progress in Neurobiology, 76(4), 236–256.

  18. Di Carlo, J. J., & Maunsell, J. H. R. (2003). Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object. Neurophysiology, 89, 3264–3278.

  19. Dittrich, W. H., & Lea, S. E. (1994). Visual perception of intentional motion. Perception, 23(3), 253–268.

  20. Fleischer, F., Casile, A., & Giese, M. A. (2009). Bio-inspired approach for the recognition of goal-directed hand actions. In X. Jiang & N. Petkov (Eds.), Conference on Computer Analysis of Images and Patterns (CAIP), LCNS (Vol. 5702, pp. 714–722).

  21. Fleischer, F., & Giese, M. A. (2010). Computational Mechanisms of the Visual Processing of Action Stimuli. In K. Johnson & M. Shiffrar (Eds.), Perception of the human body in motion: Findings, theory and practice (Vol. in press). New York: Oxford University Press.

  22. Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., & Rizzolatti, G. (2005). Parietal lobe: From action organization to intention understanding. Science, 29, 662–667.

  23. Fonlupt, P. (2003). Perception and judgement of physical causality involve different brain structures. Brain Research. Cognitive Brain Research, 17(2), 248–254.

  24. Frith, C. D., & Frith, U. (1999). Interacting minds—a biological basis. Science, 286(5445), 1692–1695.

  25. Frith, U., & Frith, C. D. (2003). Development and neurophysiology of mentalizing. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 358(1431), 459–473.

  26. Fugelsang, J. A., Roser, M. E., Corballis, P. M., Gazzaniga, M. S., & Dunbar, K. N. (2005). Brain mechanisms underlying perceptual causality. Brain Research. Cognitive Brain Research, 24(1), 41–47.

  27. Giese, M. A., & Poggio, T. (2003). Neural mechanisms for the recognition of biological movements. Nature Reviews Neuroscience, 4(3), 179–192.

  28. Hamilton, A. F., & Grafton, S. T. (2008). Action outcomes are represented in human inferior frontoparietal cortex. Cerebral Cortex, 18(5), 1160–1168.

  29. Heider, F., & Simmel, M. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57, 243–249.

  30. Jastorff, J., Clavagnier, S., Gergely, G., & Orban, G. A. (2011). Neural mechanisms of understanding rational actions: Middle temporal gyrus activation by contextual violation. Cerebral Cortex, 21(2), 318–329.

  31. Jellema, T., & Perrett, D. I. (2006). Neural representations of perceived bodily actions using a categorical frame of reference. Neuropsychologia, 44(9), 1535–1546.

  32. Kanizsa, G., & Vicario, G. (1968). The perception of intentional reaction. In G. Kanizsa & G. Vicario (Eds.), Experimental research on perception (pp. 71–126). Trieste: University of Trieste.

  33. Koenderink, J. J., van Doorn, A. J., & van de Grind, W. A. (1985). Spatial and temporal parameters of motion detection in the peripheral visual field. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 2(2), 252–259.

  34. Kourtzi, Z., & Connor, C. E. (2011). Neural representations for object perception: Structure, category, and adaptive coding. Annual Review of Neuroscience, 34, 45–67.

  35. Leslie, A. M., & Keeble, S. (1987). Do six-month-old infants perceive causality? Cognition, 25(3), 265–288.

  36. Martin, A., & Weisberg, J. (2003). Neural foundations for understanding social and mechanical concepts. Cognitive Neuropsychology, 20(3–6), 575–587.

  37. McAleer, P., & Pollick, F. E. (2008). Understanding intention from minimal displays of human activity. Behavior Research Methods, 40(3), 830–839.

  38. Michotte, A. (1946/1963). The Perception of Causality (Translated by T.R. Miles and E. Miles). London: Methuen: Basic Books.

  39. Nelissen, K., Borra, E., Gerbella, M., Rozzi, S., Luppino, G., Vanduffel, W., et al. (2011). Action observation circuits in the macaque monkey cortex. Journal of Neuroscience, 31(10), 3743–3756.

  40. Oakes, L. M., & Kannass, K. N. (1999). That’s the way the ball bounces: Infants’ and adults’ perception of spatial and temporal contiguity in collisions involving bouncing balls. Developmental Science, 2(1), 86–101.

  41. Ohnishi, T., Moriguchi, Y., Matsuda, H., Mori, T., Hirakata, M., Imabayashi, E., et al. (2004). The neural network for the mirror system and mentalizing in normally developed children: An fMRI study. NeuroReport, 15(9), 1483–1487.

  42. Op De Beeck, H., & Vogels, R. (2000). Spatial sensitivity of macaque inferior temporal neurons. Journal of Comparative Neurology, 426(4), 505–518.

  43. Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons and imitation: A computationally guided review. Neural Networks, 19(3), 254–271.

  44. Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S., Benson, P. J., Mistlin, A. J., Chitty, A. J., Hietanen, J. K., & Ortega, J. E. (1989). Frameworks of analysis for the neural representation of animate objects and actions. Journal of Experimental Biology, 146, 87–113.

  45. Pessoa, L., & Adolphs, R. (2010). Emotion processing and the amygdala: From a ‘low road’ to ‘many roads’ of evaluating biological significance. Nature Reviews Neuroscience, 11(11), 773–783.

  46. Petroni, A., Baguear, F., & Della-Maggiore, V. (2010). Motor resonance may originate from sensorimotor experience. Journal of Neurophysiology, 104(4), 1867–1871.

  47. Pouget, A., & Sejnowski, T. J. (1997). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9(2), 222–237.

  48. Prinz, W. (1997). Perception and Action Planning. European Journal of Cognitive Psychology, 9(2), 129–154.

  49. Reithler, J., van Mier, H. I., Peters, J. C., & Goebel, R. (2007). Nonvisual motor learning influences abstract action observation. Current Biology, 17(14), 1201–1207.

  50. Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11), 1019–1025.

  51. Rips, L. J. (2011). Split identity: Intransitive judgments of the identity of objects. Cognition, 119(3), 356–373.

  52. Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2(9), 661–670.

  53. Rizzolatti, G., & Sinigaglia, C. (2010). The functional role of the parietofrontal mirror circuit: Interpretations and misinterpretations. Nature Reviews Neuroscience, 11(4), 264–274.

  54. Rochat, P., Morgan, R., & Carpenter, M. (1997). Young infants’ sensitivity to movement information specifying social causality. Cognitive Development, 12, 537–561.

  55. Roser, M. E., Fugelsang, J. A., Dunbar, K. N., Corballis, P. M., & Gazzaniga, M. S. (2005). Dissociating processes supporting causal perception and causal inference in the brain. Neuropsychology, 19(5), 591–602.

  56. Salinas, E., & Abbott, L. F. (1995). Transfer of coded information from sensory to motor networks. Journal of Neuroscience, 15(10), 6461–6474.

  57. Saxe, R., & Carey, S. (2006). The perception of causality in infancy. Acta Psychologica (Amst), 123(1–2), 144–165.

  58. Saxe, R., Xiao, D. K., Kovacs, G., Perrett, D. I., & Kanwisher, N. (2004). A region of right posterior superior temporal sulcus responds to observed intentional actions. Neuropsychologia, 42(11), 1435–1446.

  59. Schlottmann, A., & Anderson, N. H. (1993). An information integration approach to phenomenal causality. Memory & Cognition, 21(6), 785–801.

  60. Schlottmann, A., Ray, E. D., Mitchell, A., & Demetriou, N. (2006). Perceived physical and social causality in animated motions: Spontaneous reports and ratings. Acta Psychologica (Amst), 123(1–2), 112–143.

  61. Schlottmann, A., & Shanks, D. R. (1992). Evidence for a distinction between judged and perceived causality. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 44(2), 321–342.

  62. Scholl, B. J., & Tremoulet, P. D. (2000). Perceptual causality and animacy. Trends in Cognitive Sciences, 4(8), 299–309.

  63. Schubotz, R. I., & von Cramon, D. Y. (2004). Sequences of abstract nonbiological stimuli share ventral premotor cortex with action observation and imagery. Journal of Neuroscience, 24(24), 5467–5474.

  64. Schultz, J., Imamizu, H., Kawato, M., & Frith, C. D. (2004). Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. Journal of Cognitive Neuroscience, 16(10), 1695–1705.

  65. Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007). Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 411–426.

  66. Smith, A. T., & Snowden, R. J. (Eds.). (1994). Visual detection of motion. London: Academic Press Limited.

  67. Straube, B., & Chatterjee, A. (2010). Space and time in perceptual causality. Frontiers in Human Neuroscience, 4, 28.

  68. Tessitore, G., Prevete, R., Catanzariti, E., & Tamburrini, G. (2010). From motor to sensory processing in mirror neuron computational modelling. Biological Cybernetics, 103(6), 471–485.

  69. Treue, S. (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24(5), 295–300.

  70. Treue, S., & Maunsell, J. H. R. (2006). Feature-based attention in visual cortex. Trends in Neurosciences, 29(6), 317–322.

  71. Ullman, S. (2007). Object recognition and segmentation by a fragment-based hierarchy. Trends in Cognitive Sciences, 11(2), 58–64.

  72. Van Overwalle, F., & Baetens, K. (2009). Understanding others’ actions and goals by mirror and mentalizing systems: A meta-analysis. Neuroimage, 48(3), 564–584.

  73. White, P. A., & Milne, A. (1997). Phenomenal causality: Impressions of pulling in the visual perception of objects in motion. American Journal of Psychology, 110(4), 573–602.

Download references


We thank H. Alhumsi for help with the data collection, D. Endres for stimulating discussions, M. Angelovska for help with the graphical illustrations, and K. Festl for help with the data analysis. This work was supported by the EC FP7 Projects TANGO (Grant FP7-249858-TP3) and AMARSi (Grant FP7-ICT-248311), the Deutsche Forschungsgemeinschaft Grant GI 305/4-1 and the Hermann and Lilly Schilling Foundation.

Author information

Correspondence to Martin A. Giese.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (AVI 60 kb)

Supplementary material 2 (AVI 74 kb)

Supplementary material 3 (AVI 66 kb)

Supplementary material 4 (AVI 60 kb)

Supplementary material 5 (AVI 66 kb)

Supplementary material 6 (AVI 70 kb)

Supplementary material 7 (AVI 81 kb)

Supplementary material 8 (AVI 61 kb)

Supplementary material 9 (AVI 60 kb)

Supplementary material 1 (AVI 60 kb)

Supplementary material 2 (AVI 74 kb)

Supplementary material 3 (AVI 66 kb)

Supplementary material 4 (AVI 60 kb)

Supplementary material 5 (AVI 66 kb)

Supplementary material 6 (AVI 70 kb)

Supplementary material 7 (AVI 81 kb)

Supplementary material 8 (AVI 61 kb)

Supplementary material 9 (AVI 60 kb)

Supplementary material 10 (AVI 42 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Fleischer, F., Christensen, A., Caggiano, V. et al. Neural theory for the perception of causal actions. Psychological Research 76, 476–493 (2012).

Download citation


  • Action Type
  • Stimulus Type
  • Hand Action
  • Action Stimulus
  • Abstract Stimulus