Brain Structure and Function

, Volume 224, Issue 9, pp 3291–3308 | Cite as

Predictive coding of action intentions in dorsal and ventral visual stream is based on visual anticipations, memory-based information and motor preparation

  • Simona MonacoEmail author
  • Giulia Malfatti
  • Alessandro Zendron
  • Elisa Pellencin
  • Luca Turella
Original Article


Predictions of upcoming movements are based on several types of neural signals that span the visual, somatosensory, motor and cognitive system. Thus far, pre-movement signals have been investigated while participants viewed the object to be acted upon. Here, we studied the contribution of information other than vision to the classification of preparatory signals for action, even in the absence of online visual information. We used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA) to test whether the neural signals evoked by visual, memory-based and somato-motor information can be reliably used to predict upcoming actions in areas of the dorsal and ventral visual stream during the preparatory phase preceding the action, while participants were lying still. Nineteen human participants (nine women) performed one of two actions towards an object with their eyes open or closed. Despite the well-known role of ventral stream areas in visual recognition tasks and the specialization of dorsal stream areas in somato-motor processes, we decoded action intention in areas of both streams based on visual, memory-based and somato-motor signals. Interestingly, we could reliably decode action intention in absence of visual information based on neural activity evoked when visual information was available and vice versa. Our results show a similar visual, memory and somato-motor representation of action planning in dorsal and ventral visual stream areas that allows predicting action intention across domains, regardless of the availability of visual information.


Functional magnetic resonance imaging (fMRI) Multivoxel pattern analysis (MVPA) Humans Actions Predictive coding Vision 



The authors would like to thank Pietro Chiesa for technical support, Jason Gallivan for providing the EBA and LO localizers, Angelika Lingnau for providing the MT localizer, and Egidio Malfatti for help with building the set-up for the LOtv localizer.


This project has received funding from the Ministero dell’istruzione, Universita’ e Ricerca under the Futuro in Ricerca 2013 grant, project RBFR132BKP to Luca Turella, and from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 703597 to Simona Monaco.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in this study were in accordance with the ethical standards of the Human Research Ethics Committee of the University of Trento (protocol 2016-021) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Supplementary material

429_2019_1970_MOESM1_ESM.docx (283 kb)
Supplementary material 1 (DOCX 282 kb)


  1. Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330PubMedGoogle Scholar
  2. Ariani G, Wurm MF, Lingnau A (2015) Decoding internally and externally driven movement plans. J Neurosci 35:14160–14171PubMedPubMedCentralGoogle Scholar
  3. Ariani G, Oosterhof NN, Lingnau A (2018) Time-resolved decoding of planned delayed and immediate prehension movements. Cortex 99:330–345PubMedGoogle Scholar
  4. Baizer JS, Ungerleider LG, Desimone R (1991) Organization of visual inputs to the inferior temporal and posterior parietal cortex in macaques. J Neurosci 11:168–190PubMedPubMedCentralGoogle Scholar
  5. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29(4):1165–1188Google Scholar
  6. Begliomini C, Caria A, Grodd W, Castiello U (2007a) Comparing natural and constrained movements: new insights into the visuomotor control of grasping. PLoS One 2:e1108. CrossRefPubMedPubMedCentralGoogle Scholar
  7. Begliomini C, Wall MB, Smith AT, Castiello U (2007b) Differential cortical activity for precision and whole-hand visually guided grasping in humans. Eur J Neurosci 25:1245–1252. CrossRefPubMedGoogle Scholar
  8. Binkofski F, Dohle C, Posse S, Stephan KM, Hefter H, Seitz RJ, Freund HJ (1998) Human anterior intraparietal area subserves prehension: a combined lesion and functional MRI activation study. Neurology 50:1253–1259PubMedGoogle Scholar
  9. Binkofski F, Kunesch E, Classen J, Seitz RJ, Freund HJ (2001) Tactile apraxia: unimodal apractic disorder of tactile object exploration associated with parietal lobe lesions. Brain 124:132–144PubMedGoogle Scholar
  10. Blakemore SJ, Wolpert D, Frith C (2000) Why can’t you tickle yourself? NeuroReport 11:R11–R16PubMedGoogle Scholar
  11. Borra E, Belmalih A, Calzavara R, Gerbella M, Murata A, Rozzi S, Luppino G (2008) Cortical connections of the macaque anterior intraparietal (AIP) area. Cereb Cortex 18:1094–1111PubMedGoogle Scholar
  12. Bracci S, Ietswaart M, Peelen MV, Cavina-Pratesi C (2010) Dissociable neural responses to hands and non-hand body parts in human left extrastriate visual cortex. J Neurophysiol 103(6):3389–3397PubMedPubMedCentralGoogle Scholar
  13. Bracci S, Cavina-Pratesi C, Ietswaart M, Caramazza A, Peelen MVMV (2012) Closely overlapping responses to tools and hands in left lateral occipitotemporal cortex. J Neurophysiol 107:1443–1456PubMedGoogle Scholar
  14. Breveglieri R, Kutz DF, Fattori P, Gamberini M, Galletti C (2002) Somatosensory cells in the parieto-occipital area V6A of the macaque. NeuroReport 13:2113–2116PubMedGoogle Scholar
  15. Breveglieri R, Bosco A, Galletti C, Passarelli L, Fattori P (2016) Neural activity in the medial parietal area V6A while grasping with or without visual feedback. Sci Rep 6:28893PubMedPubMedCentralGoogle Scholar
  16. Buneo CA, Andersen RA (2006) The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia 44:2594–2606PubMedGoogle Scholar
  17. Castiello U, Begliomini C (2008) The cortical control of visually guided grasping. Neurosci 14:157–170Google Scholar
  18. Catani M, Jones DK, Donato R, Ffytche DH (2003) Occipito-temporal connections in the human brain. Brain 126:2093–2107PubMedGoogle Scholar
  19. Cavina-Pratesi C, Monaco S, Fattori P, Galletti C, McAdam TDTDTDTD, Quinlan DJDJDJDJ, Goodale MAMA, Culham JCJCJC (2010) Functional magnetic resonance imaging reveals the neural substrates of arm transport and grip formation in reach-to-grasp actions in humans. J Neurosci 30:10306–10323PubMedPubMedCentralGoogle Scholar
  20. Cichy RM, Heinzle J, Haynes J-D (2012) Imagery and perception share cortical representations of content and location. Cereb Cortex 22:372–380PubMedGoogle Scholar
  21. Culham JC, Danckert SL, DeSouza JFX, Gati JS, Menon RS, Goodale MA (2003) Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas. Exp Brain Res 153:180–189PubMedGoogle Scholar
  22. Cohen NR, Cross ES, Tunik E, Grafton ST, Culham JC (2009) Ventral and dorsal stream contributions to the online control of immediate and delayed grasping: a TMS approach. Neuropsychologia 47:1553–1562PubMedGoogle Scholar
  23. Cole J (1995) Pride and a daily marathon. MIT Press, CambridgeGoogle Scholar
  24. Cole J (2016) Losing touch. Oxford University Press, OxfordGoogle Scholar
  25. Dijkerman HC, de Haan EHF (2007) Somatosensory processes subserving perception and action. Behav Brain Sci 30:189–201 (discussion 201–239) PubMedGoogle Scholar
  26. Downing PE, Jiang Y, Shuman M, Kanwisher N (2001) A cortical area selective for visual processing of the human body. Science (80–) 293:2470–2473Google Scholar
  27. Fattori P, Breveglieri R, Bosco A, Gamberini M, Galletti C (2015) Vision for prehension in the medial parietal cortex. Cereb Cortex 27:bhv302Google Scholar
  28. Fiehler K, Bannert MM, Bischoff M, Blecker C, Stark R, Vaitl D, Franz VH, Rösler F (2011) Working memory maintenance of grasp-target information in the human posterior parietal cortex. Neuroimage 54:2401–2411PubMedGoogle Scholar
  29. Fourkas A, Ionta S (2006) Influence of imagined posture and imagery modality on corticospinal excitability. Behav Brain Res 168:190–196PubMedGoogle Scholar
  30. Frey SH, Vinton D, Norlund R, Grafton ST (2005) Cortical topography of human anterior intraparietal cortex active during visually guided grasping. Cogn Brain Res 23:397–405. CrossRefGoogle Scholar
  31. Freud E, Macdonald SN, Chen J, Quinlan DJ, Goodale MA, Culham JC (2018) Getting a grip on reality: grasping movements directed to real objects and images rely on dissociable neural representations. Cortex 98:34–48PubMedGoogle Scholar
  32. Gallivan JP, Cavina-Pratesi C, Culham JC (2009) Is that within reach? fMRI reveals that the human superior parieto-occipital cortex encodes objects reachable by the hand. J Neurosci 29:4381–4391PubMedPubMedCentralGoogle Scholar
  33. Gallivan JP, Mclean DA, Valyear KF, Pettypiece CE, Culham JC (2011) Decoding action intentions from preparatory brain activity in human parieto-frontal networks. J Neurosci 31:9599–9610PubMedPubMedCentralGoogle Scholar
  34. Gallivan JP, Chapman CS, Mclean DA, Flanagan JR, Culham JC (2013) Activity patterns in the category-selective occipitotemporal cortex predict upcoming motor actions. Eur J Neurosci 38:2408–2424PubMedGoogle Scholar
  35. Goodale MA (2014) How (and why) the visual control of action differs from visual perception. Proc R Soc B Biol Sci 281:20140337Google Scholar
  36. Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15:20–25PubMedGoogle Scholar
  37. Goodale MA, Westwood DA (2004) An evolving view of duplex vision: separate but interacting cortical pathways for perception and action. Curr Opin Neurobiol 14:203–211PubMedGoogle Scholar
  38. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R (1999) Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 24:187–203PubMedGoogle Scholar
  39. Grill-Spector K, Kourtzi Z, Kanwisher N (2001) The lateral occipital complex and its role in object recognition. Vision Res 41:1409–1422. CrossRefPubMedGoogle Scholar
  40. Grol MJ, Majdandzic J, Stephan KE, Verhagen L, Dijkerman HC, Bekkering H, Verstraten FA, Toni I (2007) Parieto-frontal connectivity during visually guided grasping. J Neurosci 27:11877–11887PubMedPubMedCentralGoogle Scholar
  41. Hari R, Karhu J, Hämäläinen M, Knuutila J, Salonen O, Sams M, Vilkman V (1993) Functional organization of the human first and second somatosensory cortices: a neuromagnetic study. Eur J Neurosci 5:724–734PubMedGoogle Scholar
  42. Hermsdörfer J, Elias Z, Cole JD, Quaney BM, Nowak DA (2008) Preserved and impaired aspects of feed-forward grip force control after chronic somatosensory deafferentation. Neurorehabil Neural Repair 22:374–384PubMedGoogle Scholar
  43. Hutchison RM, Gallivan JP (2018) Functional coupling between frontoparietal and occipitotemporal pathways during action and perception. Cortex 98:8–27PubMedGoogle Scholar
  44. James TW, Culham JC, Humphrey GK, Milner AD, Goodale MA (2003) Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain 126:2463–2475PubMedGoogle Scholar
  45. Konen CS, Kastner S (2008) Two hierarchically organized neural systems for object information in human visual cortex. Nat Neurosci 11:224–231PubMedGoogle Scholar
  46. Kosslyn S, Thompson W (1995) Topographical representations of mental images in primary visual cortex. Nature 378:496–498PubMedGoogle Scholar
  47. Kourtzi Z, Kanwisher N (2000) Cortical regions involved in perceiving object shape. J Neurosci 20:3310–3318PubMedPubMedCentralGoogle Scholar
  48. Kourtzi Z, Kanwisher N (2001) Representation of perceived object shape by the human lateral occipital complex. Science (80–) 293:1506–1509Google Scholar
  49. Kourtzi Z, Bülthoff HH, Erb M, Grodd W (2002) Object-selective responses in the human motion area MT/MST. Nat Neurosci 5:17–18PubMedGoogle Scholar
  50. Kriegeskorte N, Lindquist MA, Nichols TE, Poldrack RA, Vul E (2010) Everything you never wanted to know about circular analysis, but were afraid to ask. J Cereb Blood Flow Metab 30:1551–1557PubMedPubMedCentralGoogle Scholar
  51. Lacquaniti F, Guigon E, Bianchi L, Ferraina S, Caminiti R (1995) Representing spatial information for limb movement: role of area 5 in the monkey. Cereb Cortex 5:391–409PubMedGoogle Scholar
  52. Lacquaniti F, Perani D, Guigon E, Bettinardi V, Carrozzo M, Grassi F, Rossetti Y, Fazio F (1997) Visuomotor transformations for reaching to memorized targets: a PET study. Neuroimage 5:129–146PubMedGoogle Scholar
  53. Luppino G, Murata A, Govoni P, Matelli M (1999) Largely segregated parietofrontal connections linking rostral intraparietal cortex (areas AIP and VIP) and the ventral premotor cortex (areas F5 and F4). Exp Brain Res 128:181–187PubMedGoogle Scholar
  54. Luppino G, Rozzi S, Calzavara R, Matelli M (2003) Prefrontal and agranular cingulate projections to the dorsal premotor areas F2 and F7 in the macaque monkey. Eur J Neurosci 17:559–578PubMedGoogle Scholar
  55. Marangon M, Kubiak A, Króliczak G (2016) Haptically guided grasping. fMRI shows right-hemisphere parietal stimulus encoding, and bilateral dorso-ventral parietal gradients of object- and action-related processing during grasp execution. Front Hum Neurosci 9:691PubMedPubMedCentralGoogle Scholar
  56. Mercier C, Aballea A, Vargas CD, Paillard J, Sirigu A (2008) Vision without proprioception modulates cortico-spinal excitability during hand motor imagery. Cereb Cortex 18:272–277PubMedGoogle Scholar
  57. Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9:1265–1279PubMedGoogle Scholar
  58. Milner AD, Goodale MA (1995) The visual brain in action, the visual brain in action. Oxford University Press, OxfordGoogle Scholar
  59. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci 92:8135–8139PubMedGoogle Scholar
  60. Monaco S, Cavina-Pratesi C, Sedda A, Fattori P, Galletti C, Culhaml JC, Culham JC (2011) Functional magnetic resonance adaptation reveals the involvement of the dorsomedial stream in hand orientation for grasping. J Neurophysiol 106:2248–2263PubMedGoogle Scholar
  61. Monaco S, Chen Y, Medendorp WPP, Crawford JDD, Fiehler K, Henriques DYP (2014) Functional magnetic resonance imaging adaptation reveals the cortical networks for processing grasp-relevant object properties. Cereb Cortex 24:1540–1554. CrossRefPubMedGoogle Scholar
  62. Monaco S, Gallivan JP, Figley TD, Singhal A, Culham JC (2017) Recruitment of foveal retinotopic cortex during haptic exploration of shapes and actions in the dark. J Neurosci 37(48):11572–11591PubMedPubMedCentralGoogle Scholar
  63. Mountcastle VB, Lynch JC, Georgopoulos AP, Sakata H, Acuna C (1975) Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. J Neurophysiol 38:871–908PubMedGoogle Scholar
  64. Murata A, Gallese V, Kaseda M, Sakata H (1996) Parietal neurons related to memory-guided hand manipulation. J Neurophysiol 75:2180–2186PubMedGoogle Scholar
  65. Murata A, Gallese V, Luppino G, Kaseda M, Sakata H (2000) Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol 83:2580–2601PubMedGoogle Scholar
  66. Ogawa S, Tank D, Menon RS, Ellermann JM, Kim SG, Merkle H, Ugurbil K (1992) Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci 89:5951–5955. CrossRefPubMedGoogle Scholar
  67. Oosterhof NN, Tipper SP, Downing PE (2012) Visuo-motor imagery of specific manual actions: a multi-variate pattern analysis fMRI study. Neuroimage 63:262–271PubMedGoogle Scholar
  68. Oosterhof NN, Connolly AC, Haxby JV (2016) CoSMoMVPA: multi-modal multivariate pattern analysis of neuroimaging data in Matlab/GNU Octave. Front Neuroinform. CrossRefPubMedPubMedCentralGoogle Scholar
  69. Passarelli L, Rosa MGP, Gamberini M, Bakola S, Burman KJ, Fattori P, Galletti C (2011) Cortical connections of area V6Av in the macaque: a visual-input node to the eye/hand coordination system. J Neurosci 31:1790–1801PubMedPubMedCentralGoogle Scholar
  70. Peelen MV, Wiggett AJ, Downing PE (2006) Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion. Neuron 49:815–822PubMedGoogle Scholar
  71. Pitzalis S, Sereno MI, Committeri G, Fattori P, Galati G, Tosoni A, Galletti C (2013) The human homologue of macaque area V6A. Neuroimage 82:517–530PubMedGoogle Scholar
  72. Raos V, Umilta MA, Gallese V, Fogassi L (2004) Functional properties of grasping-related neurons in the dorsal premotor area F2 of the macaque monkey. J Neurophysiol 92:1990–2002PubMedGoogle Scholar
  73. Raos V, Umiltá M-A, Murata A, Fogassi L, Gallese V (2005) Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. J Neurophysiol 95:709–729PubMedGoogle Scholar
  74. Reed CL, Klatzky RL, Halgren E (2005) What vs. where in touch: an fMRI study. Neuroimage 25:718–726PubMedGoogle Scholar
  75. Rossit S, McAdam T, Mclean DA, Goodale MA, Culham JC (2013) FMRI reveals a lower visual field preference for hand actions in human superior parieto-occipital cortex (SPOC) and precuneus. Cortex 49:2525–2541PubMedGoogle Scholar
  76. Sakata H, Takaoka Y, Kawarasaki A, Shibutani H (1973) Somatosensory properties of neurons in the superior parietal cortex (area 5) of the rhesus monkey. Brain Res 64:85–102PubMedGoogle Scholar
  77. Sakata H, Taira M, Murata A, Mine S (1995) Neural mechanisms of visual guidance of hand action in the parietal cortex of the monkey. Cereb Cortex 5:429–438PubMedGoogle Scholar
  78. Seitz RJ, Roland PE, Bohm C, Greitz T, Stone-Elander S (1991) Somatosensory discrimination of shape: tactile exploration and cerebral activation. Eur J Neurosci 3:481–492PubMedGoogle Scholar
  79. Singhal A, Monaco S, Kaufman LD, Culham JC, Jacobs C (2013) Human fMRI reveals that delayed action re-recruits visual perception. PLoS One 8:e73629PubMedPubMedCentralGoogle Scholar
  80. Styrkowiec PP, Nowik AM, Króliczak G (2019) The neural underpinnings of haptically guided functional grasping of tools: an fMRI study. Neuroimage 194:149–162PubMedGoogle Scholar
  81. Taira M, Mine S, Georgopoulos AP, Murata A, Sakata H (1990) Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Exp Brain Res 83:29–36PubMedGoogle Scholar
  82. Talairach J, Tournoux P (1988) Co-planar stereotaxic Atlas of the human brain. Theime, New YorkGoogle Scholar
  83. Tal Z, Geva R, Amedi A (2016) The origins of metamodality in visual object area LO: bodily topographical biases and increased functional connectivity to S1. Neuroimage 127:363–375PubMedPubMedCentralGoogle Scholar
  84. Tomassini V, Jbabdi S, Klein JC, Behrens TEJ, Pozzilli C, Matthews PM, Rushworth MFS, Johansen-Berg H (2007) Diffusion-weighted imaging tractography-based parcellation of the human lateral premotor cortex identifies dorsal and ventral subregions with anatomical and functional specializations. J Neurosci 27:10259–10269PubMedPubMedCentralGoogle Scholar
  85. Vargas CD, Olivier E, Craighero L, Fadiga L, Duhamel JR, Sirigu A (2004) The influence of hand posture on corticospinal excitability during motor imagery: a transcranial magnetic stimulation study. Cereb Cortex 14:1200–1206PubMedGoogle Scholar
  86. Verhagen L, Dijkerman HC, Grol MJ, Toni I (2008) Perceptuo-motor interactions during prehension movements. J Neurosci 28:4726–4735PubMedPubMedCentralGoogle Scholar
  87. Vul E, Kanwisher N, Kanwisher N (2010) Begging the question: the nonindependence error in fMRI data analysis. In: Hanson S, Bunzl M (eds) Foundational issues for human brain mapping. MIT Press, Cambridge, pp 71–91Google Scholar
  88. Winawer J, Huk AC, Boroditsky L (2010) A motion aftereffect from visual imagery of motion. Cognition 114:276–284PubMedGoogle Scholar
  89. Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience. Nat Neurosci 3:1212–1217PubMedGoogle Scholar
  90. Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Science 269:1880–1882PubMedGoogle Scholar
  91. Yousry TA, Schmid UD, Alkadhi H, Schmidt D, Peraud A, Buettner A, Winkler P (1997) Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. Brain 120(Pt 1):141–157PubMedGoogle Scholar
  92. Zeki S, Watson JDG, Lueck CJ, Friston KJ, Kennard C, Frackowiak RS (1991) A direct demonstration of functional specialization in human visual cortex. J Neurosci 17:641–649Google Scholar
  93. Zimmermann M, Verhagen L, de Lange FP, Toni I (2016) The extrastriate body area computes desired goal states during action planning. eNeuro. CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Center for Mind/Brain SciencesUniversity of TrentoMattarelloItaly
  2. 2.Department of Psychology and Cognitive ScienceUniversity of TrentoRoveretoItaly

Personalised recommendations