Predictive coding of action intentions in dorsal and ventral visual stream is based on visual anticipations, memory-based information and motor preparation
Predictions of upcoming movements are based on several types of neural signals that span the visual, somatosensory, motor and cognitive system. Thus far, pre-movement signals have been investigated while participants viewed the object to be acted upon. Here, we studied the contribution of information other than vision to the classification of preparatory signals for action, even in the absence of online visual information. We used functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA) to test whether the neural signals evoked by visual, memory-based and somato-motor information can be reliably used to predict upcoming actions in areas of the dorsal and ventral visual stream during the preparatory phase preceding the action, while participants were lying still. Nineteen human participants (nine women) performed one of two actions towards an object with their eyes open or closed. Despite the well-known role of ventral stream areas in visual recognition tasks and the specialization of dorsal stream areas in somato-motor processes, we decoded action intention in areas of both streams based on visual, memory-based and somato-motor signals. Interestingly, we could reliably decode action intention in absence of visual information based on neural activity evoked when visual information was available and vice versa. Our results show a similar visual, memory and somato-motor representation of action planning in dorsal and ventral visual stream areas that allows predicting action intention across domains, regardless of the availability of visual information.
KeywordsFunctional magnetic resonance imaging (fMRI) Multivoxel pattern analysis (MVPA) Humans Actions Predictive coding Vision
The authors would like to thank Pietro Chiesa for technical support, Jason Gallivan for providing the EBA and LO localizers, Angelika Lingnau for providing the MT localizer, and Egidio Malfatti for help with building the set-up for the LOtv localizer.
This project has received funding from the Ministero dell’istruzione, Universita’ e Ricerca under the Futuro in Ricerca 2013 grant, project RBFR132BKP to Luca Turella, and from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 703597 to Simona Monaco.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in this study were in accordance with the ethical standards of the Human Research Ethics Committee of the University of Trento (protocol 2016-021) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
- Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29(4):1165–1188Google Scholar
- Castiello U, Begliomini C (2008) The cortical control of visually guided grasping. Neurosci 14:157–170Google Scholar
- Cavina-Pratesi C, Monaco S, Fattori P, Galletti C, McAdam TDTDTDTD, Quinlan DJDJDJDJ, Goodale MAMA, Culham JCJCJC (2010) Functional magnetic resonance imaging reveals the neural substrates of arm transport and grip formation in reach-to-grasp actions in humans. J Neurosci 30:10306–10323PubMedPubMedCentralGoogle Scholar
- Cole J (1995) Pride and a daily marathon. MIT Press, CambridgeGoogle Scholar
- Cole J (2016) Losing touch. Oxford University Press, OxfordGoogle Scholar
- Downing PE, Jiang Y, Shuman M, Kanwisher N (2001) A cortical area selective for visual processing of the human body. Science (80–) 293:2470–2473Google Scholar
- Fattori P, Breveglieri R, Bosco A, Gamberini M, Galletti C (2015) Vision for prehension in the medial parietal cortex. Cereb Cortex 27:bhv302Google Scholar
- Goodale MA (2014) How (and why) the visual control of action differs from visual perception. Proc R Soc B Biol Sci 281:20140337Google Scholar
- Kourtzi Z, Kanwisher N (2001) Representation of perceived object shape by the human lateral occipital complex. Science (80–) 293:1506–1509Google Scholar
- Milner AD, Goodale MA (1995) The visual brain in action, the visual brain in action. Oxford University Press, OxfordGoogle Scholar
- Monaco S, Chen Y, Medendorp WPP, Crawford JDD, Fiehler K, Henriques DYP (2014) Functional magnetic resonance imaging adaptation reveals the cortical networks for processing grasp-relevant object properties. Cereb Cortex 24:1540–1554. https://doi.org/10.1093/cercor/bht006 CrossRefPubMedGoogle Scholar
- Ogawa S, Tank D, Menon RS, Ellermann JM, Kim SG, Merkle H, Ugurbil K (1992) Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci 89:5951–5955. https://doi.org/10.1016/S0006-3495(93)81441-3 CrossRefPubMedGoogle Scholar
- Talairach J, Tournoux P (1988) Co-planar stereotaxic Atlas of the human brain. Theime, New YorkGoogle Scholar
- Tomassini V, Jbabdi S, Klein JC, Behrens TEJ, Pozzilli C, Matthews PM, Rushworth MFS, Johansen-Berg H (2007) Diffusion-weighted imaging tractography-based parcellation of the human lateral premotor cortex identifies dorsal and ventral subregions with anatomical and functional specializations. J Neurosci 27:10259–10269PubMedPubMedCentralGoogle Scholar
- Vul E, Kanwisher N, Kanwisher N (2010) Begging the question: the nonindependence error in fMRI data analysis. In: Hanson S, Bunzl M (eds) Foundational issues for human brain mapping. MIT Press, Cambridge, pp 71–91Google Scholar
- Zeki S, Watson JDG, Lueck CJ, Friston KJ, Kennard C, Frackowiak RS (1991) A direct demonstration of functional specialization in human visual cortex. J Neurosci 17:641–649Google Scholar