Skip to main content
SpringerLink
Log in

Effective Neural Representations for Brain-Mediated Human-Robot Interactions

  • Chapter
  • First Online:
Neuro-Robotics

Part of the book series: Trends in Augmentation of Human Performance ((TAHP,volume 2))

Abstract

Physical interactions between robots and humans are an integral part of many neurorobotic, neural prosthetic and rehabilitation robotics applications. It is generally acknowledged that such interactions can be enhanced by providing robots with advance knowledge of the intentions of human agents, e.g. their desired motor plans and goals in a given context. One potential source of these intentions are decoded neural signals obtained from the cerebral cortex, but precisely which cortical representations are most beneficial for facilitating effective human-robot interactions is unclear. Here we review the neural representations of movement plans in the cortex and discuss the potential utility of these representations for jointly performed motor actions, particularly manipulation tasks involving the hand and arm. Emphasis is placed on the coordinate frames used by different cortical areas to encode sensory- and motor-related variables. It is argued that relative coding of sensorimotor variables, a concept that has also recently been applied to robotic planning and control algorithms, might be particularly useful for facilitating joint actions of the hand and arm. More generally, discussion of the various neural representations will provide critical insight into how biological agents might better interact with robotic agents for the development of next-generation neural prosthetic systems and rehabilitation robots.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJC, Velliste M, Boninger ML, Schwartz AB (2013) High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381(9866):557–564. doi:10.1016/s0140-6736(12)61816-9

    PubMed Central  PubMed  Google Scholar 

  2. Fukuda O, Tsuji T, Kaneko M, Otsuka A (2003) A human-assisting manipulator teleoperated by EMG signals and arm motions. IEEE Trans Robot Autom 19(2):210–222. doi:10.1109/tra.2003.808873

    Google Scholar 

  3. Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, Haddadin S, Liu J, Cash SS, van der Smagt P, Donoghue JP (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398):372–375. doi:10.1038/nature11076

    CAS  PubMed Central  PubMed  Google Scholar 

  4. Artemiadis P, Kyriakopoulos K (2010) EMG-based control of a robot arm using low-dimensional embeddings. IEEE Trans Robot Autom 26(2):393–398

    Google Scholar 

  5. Andersen RA, Buneo CA (2002) Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25:189–220

    CAS  PubMed  Google Scholar 

  6. Hocherman S, Wise SP (1991) Effects of hand movement path on motor cortical activity in awake, behaving rhesus monkeys. Exp Brain Res 83(2):285–302

    CAS  PubMed  Google Scholar 

  7. Torres EB, Quian Quiroga R, Cui H, Buneo CA (2013) Neural correlates of learning and trajectory planning in the posterior parietal cortex. Front Integr Neurosci 7:1–20

    Google Scholar 

  8. Xiao J, Padoa-Schioppa C, Bizzi E (2006) Neuronal correlates of movement dynamics in the dorsal and ventral premotor area in the monkey. Exp Brain Res 168(1–2):106–119. doi:10.1007/s00221-005-0074-2

    PubMed  Google Scholar 

  9. Padoa-Schioppa C, Li CSR, Bizzi E (2002) Neuronal correlates of kinematics-to-dynamics transformation in the supplementary motor area. Neuron 36(4):751–765

    CAS  PubMed  Google Scholar 

  10. Padoa-Schioppa C, Li CSR, Bizzi E (2004) Neuronal activity in the supplementary motor area of monkeys adapting to a new dynamic environment. J Neurophysiol 91(1):449–473. doi:10.1152/jn.00876.2002

    PubMed  Google Scholar 

  11. Soechting JF, Flanders M (1992) Moving in 3-dimensional space – frames of reference, vectors, and coordinate systems. Annu Rev Neurosci 15:167–191. doi:10.1146/annurev.neuro.15.1.167

    CAS  PubMed  Google Scholar 

  12. Salinas E, Sejnowski TJ (2001) Gain modulation in the central nervous system: where behavior, neurophysiology, and computation meet. Neuroscientist 7(5):430–440

    CAS  PubMed Central  PubMed  Google Scholar 

  13. Recanzone GH, Merzenich MM, Jenkins WM (1992) Frequency discrimination training engaging a restricted skin surface results in an emergence of a cutaneous response zone in cortical area 3a. J Neurophysiol 67:1057–1070

    CAS  PubMed  Google Scholar 

  14. Sebanz N, Bekkering H, Knoblich G (2006) Joint action: bodies and minds moving together. Trends Cogn Sci 10(2):70–76. doi:10.1016/j.tics.2005.12.009

    PubMed  Google Scholar 

  15. Tummolini L, Castelfranchi C (2006) Introduction to the special issue on cognition, joint action and collective intentionality. Cogn Syst Res 7(2–3):97–100. doi:10.1016/j.cogsys.2006.01.003

    Google Scholar 

  16. Sebanz N, Knoblich G (2009) Prediction in joint action: what, when, and where. Top Cogn Sci 1(2):353–367. doi:10.1111/j.1756-8765.2009.01024.x

    Google Scholar 

  17. Tomasello M, Carpenter M, Call J, Behne T, Moll H (2005) Understanding and sharing intentions: the origins of cultural cognition. Behav Brain Sci 28(5):675–691. doi:10.1017/s0140525x05000129

    PubMed  Google Scholar 

  18. Obhi SS, Sebanz N (2011) Moving together: toward understanding the mechanisms of joint action. Exp Brain Res 211(3–4):329–336. doi:10.1007/s00221-011-2721-0

    PubMed  Google Scholar 

  19. Rizzolatti G, Sinigaglia C (2010) The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations. Nat Rev Neurosci 11(4):264–274. doi:10.1038/nrn2805

    CAS  PubMed  Google Scholar 

  20. Becchio C, Manera V, Sartori L, Cavallo A, Castiello U (2012) Grasping intentions: from thought experiments to empirical evidence. Front Hum Neurosci 6:117. doi:10.3389/fnhum.2012.00117

    PubMed Central  PubMed  Google Scholar 

  21. Sartori L, Becchio C, Castiello U (2011) Cues to intention: the role of movement information. Cognition 119(2):242–252. doi:10.1016/j.cognition.2011.01.014

    PubMed  Google Scholar 

  22. Richardson MJ, Marsh KL, Isenhower RW, Goodman JRL, Schmidt RC (2007) Rocking together: dynamics of intentional and unintentional interpersonal coordination. Hum Mov Sci 26(6):867–891. doi:10.1016/j.humov.2007.07.002

    PubMed  Google Scholar 

  23. Pecenka N, Keller PE (2009) Auditory pitch imagery and its relationship to musical synchronization. In: Dalla Bella S, Kraus N, Overy K, Pantev C, Snyder JS, Tervaniemi M, Tillman B, Schlaug G (eds) Neurosciences and music III: disorders and plasticity, vol 1169, Annals of the New York academy of sciences. Blackwell, Boston, pp 282–286. doi:10.1111/j.1749-6632.2009.04785.x

    Google Scholar 

  24. Braun DA, Ortega PA, Wolpert DM (2009) Nash equilibria in multi-agent motor interactions. Plos Comput Biol 5(8):e1000468. doi:10.1371/journal.pcbi.1000468

    PubMed Central  PubMed  Google Scholar 

  25. Braun DA, Ortega PA, Wolpert DM (2011) Motor coordination: when two have to act as one. Exp Brain Res 211(3–4):631–641. doi:10.1007/s00221-011-2642-y

    PubMed Central  PubMed  Google Scholar 

  26. Nash JF (1950) Equilibrium points in N-person games. Proc Natl Acad Sci U S A 36(1):48–49. doi:10.1073/pnas.36.1.48

    CAS  PubMed Central  PubMed  Google Scholar 

  27. Sebanz N, Knoblich G, Prinz W (2003) Representing others’ actions: just like one’s own? Cognition 88(3):B11–B21. doi:10.1016/s0010-0277(03)00043-x

    PubMed  Google Scholar 

  28. Hommel B (2011) The Simon effect as tool and heuristic. Acta Psychol 136(2):189–202. doi:10.1016/j.actpsy.2010.04.011

    Google Scholar 

  29. Welsh TN (2009) When 1 + 1 = 1: the unification of independent actors revealed through joint Simon effects in crossed and uncrossed effector conditions. Hum Mov Sci 28(6):726–737. doi:10.1016/j.humov.2009.07.006

    PubMed  Google Scholar 

  30. Dolk T, Liepelt R, Prinz W, Fiehler K (2013) Visual experience determines the use of external reference frames in joint action control. PLoS One 8(3):e59008. doi:10.1371/journal.pone.0059008

    CAS  PubMed Central  PubMed  Google Scholar 

  31. Liepelt R, Wenke D, Fischer R (2013) Effects of feature integration in a hands-crossed version of the Social Simon paradigm. Psychol Res-Psychologische Forschung 77(2):240–248. doi:10.1007/s00426-012-0425-0

    Google Scholar 

  32. Tsai CC, Kuo WJ, Hung DL, Tzeng OJL (2008) Action co-representation is tuned to other humans. J Cogn Neurosci 20(11):2015–2024. doi:10.1162/jocn.2008.20144

    PubMed  Google Scholar 

  33. Adorno BV, Bo APL, Fraisse P, Poignet P (2011) Towards a cooperative framework for interactive manipulation involving a human and a humanoid. In: Robotics and automation (ICRA), 2011 IEEE international conference on, 9–13 May 2011. IEEE, Piscataway, pp 3777–3783. doi:10.1109/icra.2011.5979787

    Google Scholar 

  34. Shibata S, Tanaka K, Shimizu A (1995) Experimental analysis of handing over. In: IEEE international workshop on robot and human communication. IEEE, Piscataway, pp 53–58

    Google Scholar 

  35. Kajikawa S, Ishikawa E (2000) Trajectory planning for hand-over between human and robot. In: Proceedings of the 2000 IEEE international workshop on robot and human interactive communication, Osaka, Japan. IEEE, Piscataway, pp 281–287

    Google Scholar 

  36. Chan W, Parker C, Van der Loos H, Croft E (2012) Grip forces and load forces in handovers: Implications for designing human-robot handover controllers. In: Human robot interaction 2012, Boston, pp 9–16

    Google Scholar 

  37. Huber M, Rickert M, Knoll A, Brandt T, Glasauer S (2008) Human-robot interaction in handing-over tasks. In: Proceedings of the 17th IEEE international symposium on robot and human interactive communication, Munich, Germany. IEEE, Piscataway, pp 107–112

    Google Scholar 

  38. Glasauer S, Huber M, Knoll A, Brandt T (2010) Interacting in time and space: investigating human-human and human-robot joint action. In: Proceedings of the 19th IEEE international symposium on robot and human interactive communication, Viareggio, Italy. IEEE, Piscataway, pp 252–257

    Google Scholar 

  39. Huber M, Knoll A, Brandt T, Glasauer S (2009) Handing over a cube spatial features of physical joint-action. In: Strupp M, Buttner U, Cohen B (eds) Basic and clinical aspects of vertigo and dizziness, vol 1164, Annals of the New York academy of sciences. Blackwell, Boston, pp 380–382. doi:10.1111/j.1749-6632.2008.03743.x

    Google Scholar 

  40. Huber M, Kupferberg A, Lenz C, Knoll A, Brandt T, Glasauer S (2013) Spatiotemporal movement planning and rapid adaptation for manual interaction. PLoS One 8(5):e64982. doi:10.1371/journal.pone.0064982

    CAS  PubMed Central  PubMed  Google Scholar 

  41. Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye-hand coordination. J Neurophysiol 92(1):10–19

    CAS  PubMed  Google Scholar 

  42. McIntyre J, Stratta F, Lacquaniti F (1997) Viewer-centered frame of reference for pointing to memorized targets in three-dimensional space. J Neurophysiol 78(3):1601–1618

    CAS  PubMed  Google Scholar 

  43. Shadmehr R, Wise SP (2005) The computational neurobiology of reaching and pointing. The MIT Press, Cambridge, MA

    Google Scholar 

  44. Pouget A, Deneve S, Duhamel JR (2002) A computational perspective on the neural basis of multisensory spatial representations. Nat Rev Neurosci 3(9):741–747

    CAS  PubMed  Google Scholar 

  45. Flanders M, Helms-Tillery SI, Soechting JF (1992) Early stages in a sensorimotor transformation. Behav Brain Sci 15:309–362

    Google Scholar 

  46. Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD (1998) Gaze-centered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18(4):1583–1594

    CAS  PubMed  Google Scholar 

  47. Khan AZ, Crawford JD, Blohm G, Urquizar C, Rossetti Y, Pisella L (2007) Influence of initial hand and target position on reach errors in optic ataxic and normal subjects. J Vis 7(5):8.1–8.16. doi:10.1167/7.5.8

    Google Scholar 

  48. McGuire LMM, Sabes PN (2009) Sensory transformations and the use of multiple reference frames for reach planning. Nat Neurosci 12(8):1056–1061. doi:10.1038/nn.2357

    CAS  PubMed Central  PubMed  Google Scholar 

  49. McIntyre J, Stratta F, Lacquaniti F (1998) Short-term memory for reaching to visual targets: psychophysical evidence for body-centered reference frames. J Neurosci 18(20):8423–8435

    CAS  PubMed  Google Scholar 

  50. Pouget A, Ducom JC, Torri J, Bavelier D (2002) Multisensory spatial representations in eye-centered coordinates for reaching. Cognition 83(1):B1–B11

    PubMed  Google Scholar 

  51. Vetter P, Goodbody SJ, Wolpert DM (1999) Evidence for an eye-centered spherical representation of the visuomotor map. J Neurophysiol 81(2):935–939

    CAS  PubMed  Google Scholar 

  52. Tuan MT, Soueres P, Taix M, Girard B (2009) Eye-centered vs. body-centered reaching control: a robotics insight into the neuroscience debate. Paper presented at the IEEE international conference on robotics and biomimetics, Guilin, China

    Google Scholar 

  53. Carrozzo M, McIntyre J, Zago M, Lacquaniti F (1999) Viewer-centered and body-centered frames of reference in direct visuomotor transformations. Exp Brain Res 129:201–210

    CAS  PubMed  Google Scholar 

  54. Engel KC, Flanders M, Soechting JF (2002) Oculocentric frames of reference for limb movement. Archives Italiennes De Biologie 140(3):211–219

    CAS  PubMed  Google Scholar 

  55. Heuer H, Sangals J (1998) Task-dependent mixtures of coordinate systems in visuomotor transformations. Exp Brain Res 119(2):224–236

    CAS  PubMed  Google Scholar 

  56. Sober SJ, Sabes PN (2005) Flexible strategies for sensory integration during motor planning. Nat Neurosci 8(4):490–497

    CAS  PubMed Central  PubMed  Google Scholar 

  57. Buneo CA, Andersen RA (2006) The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia 44(13):2594–2606. doi:10.1016/j.neuropsychologia.2005.10.011

    PubMed  Google Scholar 

  58. Buneo CA, Soechting JF (2009) Motor psychophysics. In: Squire LR (ed) Encyclopedia of neuroscience, vol 5. Elsevier, Oxford, pp 1041–1045

    Google Scholar 

  59. Sabes PN (2011) Sensory integration for reaching: models of optimality in the context of behavior and the underlying neural circuits. In: Green AM, Chapman CE, Kalaska JF, Lepore F (eds) Enhancing performance for action and perception: multisensory integration, neuroplasticity and neuroprosthetics, Part I, vol 191, Progress in brain research. Elsevier, Amsterdam, pp 195–209. doi:10.1016/b978-0-444-53752-2.00004-7

    Google Scholar 

  60. Paulignan Y, Frak VG, Toni I, Jeannerod M (1997) Influence of object position and size on human prehension movements. Exp Brain Res 114(2):226–234. doi:10.1007/pl00005631

    CAS  PubMed  Google Scholar 

  61. Santello M, Flanders M, Soechting JF (2002) Patterns of hand motion during grasping and the influence of sensory guidance. J Neurosci 22(4):1426–1435

    CAS  PubMed  Google Scholar 

  62. Winges SA, Weber DJ, Santello M (2003) The role of vision on hand preshaping during reach to grasp. Exp Brain Res 152(4):489–498. doi:10.1007/s00221-003-1571-9

    PubMed  Google Scholar 

  63. Perenin MT, Vighetto A (1988) Optic ataxia: a specific disruption in visuomotor mechanisms. I. Different aspects of the deficit in reaching for objects. Brain 111(Pt 3):643–674

    PubMed  Google Scholar 

  64. Goodale MA, Milner AD, Jakobson LS, Carey DP (1991) A neurological dissociation between perceiving objects and grasping them. Nature 349(6305):154–156

    CAS  PubMed  Google Scholar 

  65. Darling WG, Miller GF (1993) Transformations between visual and kinesthetic coordinate systems in reaches to remembered object locations and orientations. Exp Brain Res 93(3):534–547

    CAS  PubMed  Google Scholar 

  66. Dyde RT, Milner AD (2002) Two illusions of perceived orientation: one fools all of the people some of the time; the other fools all of the people all of the time. Exp Brain Res 144(4):518–527. doi:10.1007/s00221-002-1065-1

    PubMed  Google Scholar 

  67. Fan J, He JP, Tillery SIH (2006) Control of hand orientation and arm movement during reach and grasp. Exp Brain Res 171(3):283–296. doi:10.1007/s00221-005-0277-6

    PubMed  Google Scholar 

  68. Gentilucci M, Daprati E, Gangitano M, Saetti MC, Toni I (1996) On orienting the hand to reach and grasp an object. Neuroreport 7(2):589–592. doi:10.1097/00001756-199601310-00051

    CAS  PubMed  Google Scholar 

  69. Marotta JJ, Medendorp WP, Crawford JD (2003) Kinematic rules for upper and lower arm contributions to grasp orientation. J Neurophysiol 90(6):3816–3827. doi:10.1152/jn.00418.2003

    CAS  PubMed  Google Scholar 

  70. Mamassian P (1997) Prehension of objects oriented in three-dimensional space. Exp Brain Res 114(2):235–245. doi:10.1007/pl00005632

    CAS  PubMed  Google Scholar 

  71. Roby-Brami A, Bennis N, Mokhtari M, Baraduc P (2000) Hand orientation for grasping depends on the direction of the reaching movement. Brain Res 869(1–2):121–129. doi:10.1016/s0006-8993(00)02378-7

    CAS  PubMed  Google Scholar 

  72. Soechting JF, Flanders M (1993) Parallel, independent channels for location and orientation in sensorimotor transformations for reaching and grasping. J Neurophysiol 70(3):1137–1150

    CAS  PubMed  Google Scholar 

  73. Torres EB, Zipser D (2002) Reaching to grasp with a multi-jointed arm. I. Computational model. J Neurophysiol 88(5):2355–2367

    PubMed  Google Scholar 

  74. Torres EB, Zipser D (2004) Simultaneous control of hand displacements and rotations in orientation-matching experiments. J Appl Physiol 96(5):1978–1987. doi:10.1152/japplphysiol.00872.2003

    PubMed  Google Scholar 

  75. Tunik E, Frey SH, Grafton ST (2005) Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp. Nat Neurosci 8(4):505–511. doi:10.1038/nn1430

    CAS  PubMed  Google Scholar 

  76. van Doorn H, van der Kamp J, Savelsbergh GJP (2005) Catching and matching bars with different orientations. Acta Psychol 120(3):288–306. doi:10.1016/j.actpsy.2005.05.005

    Google Scholar 

  77. Gosselin-Kessiby N, Kalaska JF, Messier J (2009) Evidence for a proprioception-based rapid on-line error correction mechanism for hand orientation during reaching movements in blind subjects. J Neurosci 29(11):3485–3496. doi:10.1523/jneurosci.2374-08.2009

    CAS  PubMed  Google Scholar 

  78. Gosselin-Kessiby N, Messier J, Kalaska JF (2008) Evidence for automatic on-line adjustments of hand orientation during natural reaching movements to stationary targets. J Neurophysiol 99(4):1653–1671. doi:10.1152/jn.00980.2007

    CAS  PubMed  Google Scholar 

  79. Desmurget M, Prablanc C (1997) Postural control of three-dimensional prehension movements. J Neurophysiol 77(1):452–464

    CAS  PubMed  Google Scholar 

  80. Strabala K, Lee MK, Dragan A, Forlizzi J, Srinivasa SS, Cakmak M, Micelli V (2013) Towards seamless human-robot handovers. J Hum Robot Interact 1(1):1–23

    Google Scholar 

  81. Cakmak M, Srinivasa SS, Lee MK, Kiesler S, Forlizzi J (2011) Using spatial and temporal contrast for fluent robot-human handovers. In: Proceedings of the 6th international conference on human-robot interaction, Lausanne, Switzerland. ACM Press, New York, pp 489–496

    Google Scholar 

  82. Nakata T, Sato T, Mori T (1998) Expression of emotion and intention by robot body movement. In: Proceedings of the international conference on intelligent autonomous systems. IOS press, Amsterdam

    Google Scholar 

  83. Schulte J, Rosenberg C, Thrun S (1999) Spontaneous, short-term interaction with mobile robots in public places. In: Proceedings of ICRA. IEEE, Piscataway

    Google Scholar 

  84. Jee E, Jeong Y, Kim CH, Kobayashi H (2010) Sound design for emotion and intention expression of socially interactive robots. Intell Serv Robot 3(3):199–206

    Google Scholar 

  85. Grigore EC, Eder K, Pipe AG, Melhuish C, Leonards U (2013) Joint action understanding improves robot-to-human object handover. In: Intelligent robots and systems 2013. IEEE, Piscataway, pp 1–8

    Google Scholar 

  86. Hogan N, Flash T (1987) Moving gracefully – quantitative theories of motor coordination. Trends Neurosci 10(4):170–174

    Google Scholar 

  87. Koay K, Sisbot E, Syrdal D, Walters M, Dautenhahn K, Alami R (2007) Exploratory study of a robot approaching a person in the context of handing over an object. In: Multidisciplinary collaboration for socially assistive robotics, Stanford, CA, pp 18–24

    Google Scholar 

  88. Bekkering H, de Bruijn ERA, Cuijpers RH, Newman-Norlund R, van Schie HT, Meulenbroek R (2009) Joint action: neurocognitive mechanisms supporting human interaction. Top Cogn Sci 1(2):340–352. doi:10.1111/j.1756-8765.2009.01023.x

    Google Scholar 

  89. Oztop E, Wolpert D, Kawato M (2005) Mental state inference using visual control parameters. Cogn Brain Res 22(2):129–151. doi:10.1016/j.cogbrainres.2004.08.004

    Google Scholar 

  90. Fabbri-Destro M, Rizzolatti G (2008) Mirror neurons and mirror systems in monkeys and humans. Physiology 23(3):171–179. doi:10.1152/physiol.00004.2008

    PubMed  Google Scholar 

  91. Jastorff J, Begliomini C, Fabbri-Destro M, Rizzolatti G, Orban GA (2010) Coding observed motor acts: different organizational principles in the parietal and premotor cortex of humans. J Neurophysiol 104(1):128–140. doi:10.1152/jn.00254.2010

    PubMed  Google Scholar 

  92. Fernandez-Ruiz J, Goltz HC, DeSouza JFX, Vilis T, Crawford JD (2007) Human parietal “reach region” primarily encodes intrinsic visual direction, not extrinsic movement direction, in a visual-motor dissociation task. Cereb Cortex 17(10):2283–2292. doi:10.1093/cercor/bhl137

    PubMed  Google Scholar 

  93. Vesia M, Yan XG, Henriques DY, Sergio LE, Crawford JD (2008) Transcranial magnetic stimulation over human dorsal-lateral posterior parietal cortex disrupts integration of hand position signals into the reach plan. J Neurophysiol 100(4):2005–2014. doi:10.1152/jn.90519.2008

    PubMed  Google Scholar 

  94. Beurze SM, Toni I, Pisella L, Medendorp WP (2010) Reference frames for reach planning in human parietofrontal cortex. J Neurophysiol 104(3):1736–1745. doi:10.1152/jn.01044.2009

    CAS  PubMed  Google Scholar 

  95. Beurze SM, Van Pelt S, Medendorp WP (2006) Behavioral reference frames for planning human reaching movements. J Neurophysiol 96(1):352–362. doi:10.1152/jn.01362.2005

    PubMed  Google Scholar 

  96. Batista AP, Buneo CA, Snyder LH, Andersen RA (1999) Reach plans in eye-centered coordinates. Science 285:257–260

    CAS  PubMed  Google Scholar 

  97. Buneo CA, Jarvis MR, Batista AP, Andersen RA (2002) Direct visuomotor transformations for reaching. Nature 416(6881):632–636. doi:10.1038/416632a

    CAS  PubMed  Google Scholar 

  98. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433

    CAS  PubMed  Google Scholar 

  99. Azanon E, Longo MR, Soto-Faraco S, Haggard P (2010) The posterior parietal cortex remaps touch into external space. Curr Biol 20(14):1304–1309. doi:10.1016/j.cub.2010.05.063

    CAS  PubMed  Google Scholar 

  100. Tillery SIH, Flanders M, Soechting JF (1991) A coordinate system for the synthesis of visual and kinesthetic information. J Neurosci 11(3):770–778

    CAS  PubMed  Google Scholar 

  101. Tillery SIH, Flanders M, Soechting JF (1994) Errors in kinesthetic transformations for hand apposition. Neuroreport 6(1):177–181

    CAS  PubMed  Google Scholar 

  102. Tillery SIH, Soechting JF, Ebner TJ (1996) Somatosensory cortical activity in relation to arm posture: nonuniform spatial tuning. J Neurophysiol 76(4):2423–2438

    CAS  PubMed  Google Scholar 

  103. Buneo CA, Andersen RA (2002) effects of gaze angle and vision of the hand on reach-related activity in the posterior parietal cortex. In: Program no 62.3. 2002 Abstract viewer/itinerary planner. Washington, DC

    Google Scholar 

  104. Buneo CA, Apker GA, Shi Y (2011) Probing neural correlates of cue integration. In: Trommershauser J, Kording K, Landy MS (eds) Sensory cue integration. Oxford University Press, New York/Oxford, pp 317–332

    Google Scholar 

  105. Gu Y, Angelaki DE, DeAngelis GC (2008) Neural correlates of multisensory cue integration in macaque MSTd. Nat Neurosci 11(10):1201–1210

    CAS  PubMed Central  PubMed  Google Scholar 

  106. Kokal I, Gazzola V, Keysers C (2009) Acting together in and beyond the mirror neuron system. Neuroimage 47(4):2046–2056. doi:10.1016/j.neuroimage.2009.06.010

    PubMed  Google Scholar 

  107. Kokal I, Keysers C (2010) Granger causality mapping during joint actions reveals evidence for forward models that could overcome sensory-motor delays. PLoS One 5(10):e13507. doi:10.1371/journal.pone.0013507

    PubMed Central  PubMed  Google Scholar 

  108. Newman-Norlund RD, Bosga J, Meulenbroek RGJ, Bekkering H (2008) Anatomical substrates of cooperative joint-action in a continuous motor task: virtual lifting and balancing. Neuroimage 41(1):169–177. doi:10.1016/j.neuroimage.2008.02.026

    PubMed  Google Scholar 

  109. Fluet MC, Baumann MA, Scherberger H (2010) Context-specific grasp movement representation in macaque ventral premotor cortex. J Neurosci 30(45):15175–15184. doi:10.1523/jneurosci.3343-10.2010

    CAS  PubMed  Google Scholar 

  110. Baumann MA, Fluet MC, Scherberger H (2009) Context-specific grasp movement representation in the macaque anterior intraparietal area. J Neurosci 29(20):6436–6448. doi:10.1523/jneurosci.5479-08.2009

    CAS  PubMed  Google Scholar 

  111. Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G (2001) I know what you are doing: a neurophysiological study. Neuron 31(1):155–165. doi:10.1016/s0896-6273(01)00337-3

    CAS  PubMed  Google Scholar 

  112. Fujii N, Hihara S, Iriki A (2008) Social cognition in premotor and parietal cortex. Soc Neurosci 3(3–4):250–260. doi:10.1080/17470910701434610

    PubMed  Google Scholar 

  113. Kaas JH, Gharbawie OA, Stepniewska I (2011) The organization and evolution of dorsal stream multisensory motor pathways in primates. Front Neuroanat 5:34. doi:10.3389/fnana.2011.00034

    PubMed Central  PubMed  Google Scholar 

  114. Andersen RA, Cui H (2009) Intention, action planning, and decision making in parietal-frontal circuits. Neuron 63(5):568–583. doi:10.1016/j.neuron.2009.08.028

    CAS  PubMed  Google Scholar 

  115. Buneo CA (2010) Neural representations of intended movement. In: Koob G, Le Moal M, Thompson R (eds) Encyclopedia of behavioral neuroscience, vol 2. Elsevier, Oxford, pp 356–361

    Google Scholar 

  116. Andersen RA, Buneo CA (2002) Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25:189–220. doi:10.1146/annurev.neuro.25.112701.142922

    CAS  PubMed  Google Scholar 

  117. Raos V, Umilta MA, Gallese V, Fogassi L (2004) Functional properties of grasping-related neurons in the dorsal premotor area F2 of the macaque monkey. J Neurophysiol 92(4):1990–2002. doi:10.1152/jn.00154.2004

    PubMed  Google Scholar 

  118. Fattori P, Kutz DF, Breveglieri R, Marzocchi N, Galletti C (2005) Spatial tuning of reaching activity in the medial parieto-occipital cortex (area V6A) of macaque monkey. Eur J Neurosci 22(4):956–972. doi:10.1111/j.1460-9568.2005.04288.x

    PubMed  Google Scholar 

  119. Hikosaka O, Wurtz RH (1983) Visual and oculomotor functions of monkey substantia nigra pars reticulata. III. Memory-contingent visual and saccade responses. J Neurophysiol 49(5):1268–1284

    CAS  PubMed  Google Scholar 

  120. Kurata K, Wise SP (1988) Premotor cortex of rhesus-monkeys: set-related activity during 2 conditional motor tasks. Exp Brain Res 69(2):327–343

    CAS  PubMed  Google Scholar 

  121. Shen LM, Alexander GE (1997) Neural correlates of a spatial sensory-to-motor transformation in primary motor cortex. J Neurophysiol 77(3):1171–1194

    CAS  PubMed  Google Scholar 

  122. Moran DW, Schwartz AB (1999) Motor cortical representation of speed and direction during reaching. J Neurophysiol 82(5):2676–2692

    CAS  PubMed  Google Scholar 

  123. Crammond DJ, Kalaska JF (2000) Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity. J Neurophysiol 84(2):986–1005

    CAS  PubMed  Google Scholar 

  124. Messier J, Kalaska JF (2000) Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task. J Neurophysiol 84(1):152–165

    CAS  PubMed  Google Scholar 

  125. Churchland MM, Afshar A, Shenoy KV (2006) A central source of movement variability. Neuron 52(6):1085–1096. doi:10.1016/j.neuron.2006.10.034

    CAS  PubMed Central  PubMed  Google Scholar 

  126. Li CSR, Padoa-Schioppa C, Bizzi E (2001) Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30(2):593–607

    CAS  PubMed  Google Scholar 

  127. Colby CL, Goldberg ME (1999) Space and attention in parietal cortex. Annu Rev Neurosci 22:319–349

    CAS  PubMed  Google Scholar 

  128. Snyder LH, Batista AP, Andersen RA (1997) Coding of intention in the posterior parietal cortex. Nature 386(6621):167–170

    CAS  PubMed  Google Scholar 

  129. Battaglia-Mayer A, Ferraina S, Mitsuda T, Marconi B, Genovesio A, Onorati P, Lacquaniti F, Caminiti R (2000) Early coding of reaching in the parietooccipital cortex. J Neurophysiol 83(4):2374–2391

    CAS  PubMed  Google Scholar 

  130. Quiroga RQ, Snyder LH, Batista AP, Cui H, Andersen RA (2006) Movement intention is better predicted than attention in the posterior parietal cortex. J Neurosci 26(13):3615–3620

    CAS  Google Scholar 

  131. Battaglia-Mayer A, Mascaro M, Caminiti R (2007) Temporal evolution and strength of neural activity in parietal cortex during eye and hand movements. Cereb Cortex 17(6):1350–1363

    PubMed  Google Scholar 

  132. Cui H, Andersen RA (2007) Posterior parietal cortex encodes autonomously selected motor plans. Neuron 56(3):552–559. doi:10.1016/j.neuron.2007.09.031

    CAS  PubMed Central  PubMed  Google Scholar 

  133. Buneo CA, Andersen RA (2012) Integration of target and hand position signals in the posterior parietal cortex: effects of workspace and hand vision. J Neurophysiol 108(1):187–199. doi:10.1152/jn.00137.2011

    PubMed Central  PubMed  Google Scholar 

  134. Beurze SM, de Lange FP, Toni I, Medendorp WP (2007) Integration of target and effector information in the human brain during reach planning. J Neurophysiol 97(1):188–199

    CAS  PubMed  Google Scholar 

  135. Chang SWC, Papadimitriou C, Snyder LH (2009) Using a compound gain field to compute a reach plan. Neuron 64(5):744–755. doi:10.1016/j.neuron.2009.11.005

    CAS  PubMed Central  PubMed  Google Scholar 

  136. Bremner LR, Andersen RA (2012) Coding of the reach vector in parietal area 5d. Neuron 75(2):342–351. doi:10.1016/j.neuron.2012.03.041

    CAS  PubMed Central  PubMed  Google Scholar 

  137. McGuire LMM, Sabes PN (2011) Heterogeneous representations in the superior parietal lobule are common across reaches to visual and proprioceptive targets. J Neurosci 31(18):6661–6673. doi:10.1523/jneurosci.2921-10.2011

    CAS  PubMed Central  PubMed  Google Scholar 

  138. Chang SWC, Snyder LH (2010) Idiosyncratic and systematic aspects of spatial representations in the macaque parietal cortex. Proc Natl Acad Sci U S A 107(17):7951–7956. doi:10.1073/pnas.0913209107

    CAS  PubMed Central  PubMed  Google Scholar 

  139. Battaglia-Mayer A, Ferraina S, Genovesio A, Marconi B, Squatrito S, Molinari M, Lacquaniti F, Caminiti R (2001) Eye-hand coordination during reaching. II. An analysis of the relationships between visuomanual signals in parietal cortex and parieto-frontal association projections. Cereb Cortex 11(6):528–544

    CAS  PubMed  Google Scholar 

  140. Russo GS, Backus DA, Ye SP, Crutcher MD (2002) Neural activity in monkey dorsal and ventral cingulate motor areas: comparison with the supplementary motor area. J Neurophysiol 88(5):2612–2629

    PubMed  Google Scholar 

  141. Buneo CA, Batista AP, Jarvis MR, Andersen RA (2008) Time-invariant reference frames for parietal reach activity. Exp Brain Res 188(1):77–89. doi:10.1007/s00221-008-1340-x

    PubMed  Google Scholar 

  142. Scott SH, Sergio LE, Kalaska JF (1997) Reaching movements with similar hand paths but different arm orientations. 2. Activity of individual cells in dorsal premotor cortex and parietal area 5. J Neurophysiol 78(5):2413–2426

    CAS  PubMed  Google Scholar 

  143. Hamel-Paquet C, Sergio LE, Kalaska JF (2006) Parietal area 5 activity does not reflect the differential time-course of motor output kinetics during arm-reaching and isometric-force tasks. J Neurophysiol 95(6):3353–3370. doi:10.1152/jn.00789.2005

    PubMed  Google Scholar 

  144. Kalaska JF, Cohen DAD, Prudhomme M, Hyde ML (1990) Parietal area-5 neuronal-activity encodes movement kinematics, not movement dynamics. Exp Brain Res 80(2):351–364

    CAS  PubMed  Google Scholar 

  145. Batista AP, Santhanam G, Yu BM, Ryu SI, Afshar A, Shenoy KV (2007) Reference frames for reach planning in macaque dorsal premotor cortex. J Neurophysiol 98(2):966–983

    PubMed  Google Scholar 

  146. Pesaran B, Nelson MJ, Andersen RA (2006) Dorsal premotor neurons encode the relative position of the hand, eye, and goal during reach planning. Neuron 51(1):125–134

    CAS  PubMed Central  PubMed  Google Scholar 

  147. Hayworth KJ, Lescroart MD, Biederman I (2011) Neural encoding of relative position. J Exp Psychol-Hum Percept Perform 37(4):1032–1050. doi:10.1037/a0022338

    PubMed  Google Scholar 

  148. Desai JP, Ostrowski JP, Kumar V (2001) Modeling and control of formations of nonholonomic mobile robots. IEEE Trans Robot Autom 17(6):905–908. doi:10.1109/70.976023

    Google Scholar 

  149. Dissanayake M, Newman P, Clark S, Durrant-Whyte HF, Csorba M (2001) A solution to the simultaneous localization and map building (SLAM) problem. IEEE Trans Robot Autom 17(3):229–241. doi:10.1109/70.938381

    Google Scholar 

  150. Taylor CJ, Spletzer J (2007) A bounded uncertainty approach to cooperative localization using relative bearing constraints. In: Proceedings of the 2007 IEEE/RSJ international conference on intelligent robots and systems, San Diego, CA. IEEE, Piscataway

    Google Scholar 

  151. Santello M, Baud-Bovy G, Jorntell H (2013) Neural bases of hand synergies. Front Comput Neurosci 7:23. doi:10.3389/fncom.2013.00023

    PubMed Central  PubMed  Google Scholar 

  152. Georgopoulos AP, Pellizzer G, Poliakov AV, Schieber MH (1999) Neural coding of finger and wrist movements. J Comput Neurosci 6(3):279–288. doi:10.1023/a:1008810007672

    CAS  PubMed  Google Scholar 

  153. Selen LPJ, Medendorp WP (2011) Saccadic updating of object orientation for grasping movements. Vis Res 51(8):898–907. doi:10.1016/j.visres.2011.01.004

    CAS  PubMed  Google Scholar 

  154. Fattori P, Breveglieri R, Marzocchi N, Filippini D, Bosco A, Galletti C (2009) Hand orientation during reach-to-grasp movements modulates neuronal activity in the medial posterior parietal area V6A. J Neurosci 29(6):1928–1936. doi:10.1523/jneurosci.4998-08.2009

    PubMed  Google Scholar 

  155. Theys T, Srivastava S, van Loon J, Goffin J, Janssen P (2012) Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area. J Neurophysiol 107(3):995–1008. doi:10.1152/jn.00248.2011

    PubMed  Google Scholar 

  156. Kakei S, Hoffman DS, Strick PL (2001) Direction of action is represented in the ventral premotor cortex. Nat Neurosci 4(10):1020–1025

    CAS  PubMed  Google Scholar 

  157. Batista AP, Yu BM, Santhanam G, Ryu SI, Afshar A, Shenoy KV (2008) Cortical neural prosthesis performance improves when eye position is monitored. IEEE Trans Neural Syst Rehabil Eng 16(1):24–31

    PubMed  Google Scholar 

  158. Pouget A, Dayan P, Zemel R (2000) Information processing with population codes. Nat Rev Neurosci 1(2):125–132. doi:10.1038/35039062

    CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Biological and Health Systems Engineering, Arizona State University, Tempe, AZ, 85287, USA

    Christopher A. Buneo, Stephen Helms Tillery & Marco Santello

  2. School for Engineering of Matter, Transport, and Energy, Arizona State University, Tempe, AZ, 85287, USA

    Veronica J. Santos & Panagiotis Artemiadis

Authors
  1. Christopher A. Buneo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Helms Tillery
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Santello
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Veronica J. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Panagiotis Artemiadis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher A. Buneo .

Editor information

Editors and Affiliations

  1. School for Engineering of Matter, Transport and Energy, Arizona State University, Tempe, Arizona, USA

    Panagiotis Artemiadis

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Buneo, C.A., Tillery, S.H., Santello, M., Santos, V.J., Artemiadis, P. (2014). Effective Neural Representations for Brain-Mediated Human-Robot Interactions. In: Artemiadis, P. (eds) Neuro-Robotics. Trends in Augmentation of Human Performance, vol 2. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-8932-5_8

Download citation

Publish with us

Policies and ethics

Search

Navigation