Skip to main content
SpringerLink
Log in

Embodied Mental Imagery in Cognitive Robots

  • Chapter
Springer Handbook of Model-Based Science

Part of the book series: Springer Handbooks ((SHB))

Abstract

This chapter is focused on discussing the concept of mental imagery as a fundamental cognitive capability to enhance the performance of cognitive robots . Indeed, the emphasis will be on the embodied imagery mechanisms applied to build artificial cognitive models of motor imagery and mental simulation to control complex behaviors of humanoid platforms, which represent the artificial body.

With the aim of providing a panorama of the research activity on the topic, first we give an introduction on the neuroscientific and psychological background of mental imagery in order to help the reader to contextualize the multidisciplinary environment in which we operate. Then, we review the work done in the field of artificial cognitive systems and robotics to mimic the process behind the human ability of creating mental images of events and experiences, and to use this process as a cognitive mechanism to improve the behavior of complex robots. Finally, we report the detail of three recent empirical studies in which mental imagery approaches were modelled trough artificial neural networks (GlossaryTerm

ANN

s) to enable a cognitive robot with some human-like capabilities. These empirical studies exemplify how the proprioceptive information can be used by mental imagery models to enhance the performance of the robot, giving evidence of the embodied cognition theories in the context of artificial cognitive systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 269.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 349.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

ANN:

artificial neural network

BPTT:

backpropagation through time

DoF:

degree of freedom

DRNN:

dual recurrent neural network

FFNN:

feedforward neural network

MSE:

mean squared error

RBC:

reference benchmark controller

RNN:

recurrent neural network

TRoPICAL:

two route, prefrontal instruction, competition of affordances, language simulation

References

  1. A.M. Turing: Computing machinery and intelligence, Mind 59, 433–460 (1950)

    Article  MathSciNet  Google Scholar 

  2. G. Metta, L. Natale, F. Nori, G. Sandini, D. Vernon, L. Fadiga, C. von Hofsten, K. Rosander, M. Lopes, J. Santos-Victor, A. Bernardino, L. Montesano: The iCub humanoid robot: An open-systems platform for research in cognitive development, Neural Netw. 23, 1125–1134 (2010)

    Article  Google Scholar 

  3. D. Gouaillier, V. Hugel, P. Blazevic, C. Kilner, J. Monceaux, P. Lafourcade, B. Marnier, J. Serre, B. Maisonnier: Mechatronic design of NAO humanoid, Proc. IEEE Int. Conf. Robot. Autom. (2009) pp. 764–774

    Google Scholar 

  4. Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, K. Fujimura: The intelligent ASIMO: System overview and integration, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Vol. 3 (2002) pp. 2478–2483

    Chapter  Google Scholar 

  5. R. Pfeifer, J. Bongard, S. Grand: How The Body Shapes The Way We Think: A New View of Intelligence (MIT Press, Cambridge 2007)

    Google Scholar 

  6. S.M. Kosslyn: Image and Brain: The Resolution of The Imagery Debate (MIT Press, Cambridge 1996)

    Google Scholar 

  7. S. Gardini, C. Cornoldi, R. De Beni, A. Venneri: Cognitive and neuronal processes involved in sequential generation of general and specific mental images, Psychol. Res. 73, 633–643 (2009)

    Article  Google Scholar 

  8. S.D. Slotnick, W.L. Thompson, S.M. Kosslyn: Visual memory and visual mental imagery recruit common control and sensory regions of the brain, Cogn. Neurosci. 3, 14–20 (2012)

    Article  Google Scholar 

  9. A.Z.J. Zeman, S. Della Sala, L.A. Torrens, V.E. Gountouna, D.J. McGonigle, R.H. Logie: Loss of imagery phenomenology with intact visuo-spatial task performance: A case of blind imagination, Neuropsychologia 48, 145–155 (2010)

    Article  Google Scholar 

  10. M. Kozhevnikov: Cognitive styles in the context of modern psychology: Toward an integrated framework of cognitive style, Psychol. Bull. 133, 464–481 (2007)

    Article  Google Scholar 

  11. M. Kozhevnikov, S. Kosslyn, J. Shephard: Spatial versus object visualizers: A new characterization of visual cognitive style, Mem. Cogn. 33, 710–726 (2005)

    Article  Google Scholar 

  12. M. Wilson: Six views of embodied cognition, Psychon. Bull. Rev. 9(4), 625–636 (2002)

    Article  Google Scholar 

  13. A. Clark, D. Chalmers: The extended mind, Analysis 58(1), 10–23 (1998)

    Article  Google Scholar 

  14. L. Munari: How the body shapes the way we think – A new view of intelligence, J. Med. Pers. 7, 110–111 (2009)

    Article  Google Scholar 

  15. A. Ishai, L.G. Ungerleider, G.V. Haxby: Distributed neural systems for the generation of visual images, Neuron 28(3), 379–390 (2000)

    Article  Google Scholar 

  16. M. Jeannerod: The representing brain. Neural correlates of motor intention and imagery, Behav. Brain Sci. 17(2), 187–245 (1994)

    Article  Google Scholar 

  17. J. Decety, M. Jeannerod, C. Prablanc: The timing of mentally represented actions, Behav. Brain Res. 34(1/2), 35–42 (1989)

    Article  Google Scholar 

  18. M. Jeannerod, J. Decety: Mental motor imagery: A window into the representational stages of action, Curr. Opin. Neurobiol. 5(6), 727–732 (1995)

    Article  Google Scholar 

  19. G. Hesslow: The current status of the simulation theory of cognition, Brain Res. 1428, 71–79 (2012)

    Article  Google Scholar 

  20. D.M. Wolpert: Computational approaches to motor control, Trends Cogn. Sci. 1(6), 209–216 (1997)

    Article  Google Scholar 

  21. X. Skoura, A. Vinter, C. Papaxanthis: Mentally simulated motor actions in children, Dev. Neuropsychol. 34(3), 356–367 (2009)

    Article  Google Scholar 

  22. B. Steenbergen, M. van Nimwegen, C. Crajé: Solving a mental rotation task in congenital hemiparesis: Motor imagery versus visual imagery, Neuropsychologia 45(14), 3324–3328 (2007)

    Article  Google Scholar 

  23. M. Jeannerod: Neural simulation of action: A unifying mechanism for motor cognition, Neuroimage 14, S103–S109 (2001)

    Article  Google Scholar 

  24. J. Munzert, B. Lorey, K. Zentgraf: Cognitive motor processes: The role of motor imagery in the study of motor representations, Brain Res. Rev. 60(2), 306–326 (2009)

    Article  Google Scholar 

  25. R. Ramsey, J. Cumming, D. Eastough, M.G. Edwards: Incongruent imagery interferes with action initiation, Brain Cogn. 74(3), 249–254 (2010)

    Article  Google Scholar 

  26. K.D. Markman, W.M. Klein, J.A. Suhr: Handbook of Imagination and Mental Simulation (Psychology, New York 2009)

    Google Scholar 

  27. G. Yue, K.J. Cole: Strength increases from the motor program: Comparison of training with maximal voluntary and imagined muscle contractions, J. Neurophysiol. 67(5), 1114–1123 (1992)

    Google Scholar 

  28. T. Mulder, S. Zijlstra, W. Zijlstra, J. Hochstenbach: The role of motor imagery in learning a totally novel movement, Exp. Brain Res. 154(2), 211–217 (2004)

    Article  Google Scholar 

  29. L. Yágüez, D. Nagel, H. Hoffman, A.G. Canavan, E. Wist, V. Hömberg: A mental route to motor learning: Improving trajectorial kinematics through imagery training, Behav. Brain Res. 90(1), 95–106 (1998)

    Article  Google Scholar 

  30. M. Takahashi, S. Hayashi, Z. Ni, S. Yahagi, M. Favilla, T. Kasai: Physical practice induces excitability changes in human hand motor area during motor imagery, Exp. Brain Res. 163(1), 132–136 (2005)

    Article  Google Scholar 

  31. P.L. Jackson, M.F. Lafleur, F. Malouin, C. Richards, J. Doyon: Potential role of mental practice using motor imagery in neurologic rehabilitation, Arch. Phys. Med. Rehabil. 8, 1133–1141 (2001)

    Article  Google Scholar 

  32. J.A. Verbunt, H.A. Seelen, F.P. Ramos, B.H. Michielsen, W.L. Wetzelaer, M. Moennekens: Mental practice-based rehabilitation training to improve arm function and daily activity performance in stroke patients: A randomized clinical trial, BMC Neurol. 8(C), 7 (2008)

    Article  Google Scholar 

  33. S.J. Page, P. Levine, A. Leonard: Mental practice in chronic stroke: Results of a randomized, placebo-controlled trial, Stroke A J. Cereb. Circ. 38(4), 1293–1297 (2007)

    Article  Google Scholar 

  34. D.M. Nilsen, G. Gillen, A.M. Gordon: Use of mental practice to improve upper-limb recovery after stroke: A systematic review, Am. J. Occup. Ther. 64(5), 695–708 (2010)

    Article  Google Scholar 

  35. A.A. Sheikh, E.R. Korn: Imagery in Sports and Physical Performance (Baywood, Amityville 1994)

    Google Scholar 

  36. B.S. Rushall, L.G. Lippman: The role of imagery in physical performance, Int. J. Sport Psychol. 29(1), 57–72 (1998)

    Google Scholar 

  37. T. Morris, M. Spittle, A.P. Watt: Imagery in Sport (Human Kinetics, Champaign 2005)

    Google Scholar 

  38. R. Weinberg: Does imagery work? Effects on performance and mental skills, J. Imag. Res. Sport Phys. Act. 3(1), 1–22 (2008), http://www.degruyter.com/view/j/jirspa.2008.3.1/jirspa.2008.3.1.1025/jirspa.2008.3.1.1025.xml

    Google Scholar 

  39. S.A. Hamilton, W.J. Fremouw: Cognitive-behavioral training for college basketball free-throw performance, Cogn. Ther. Res. 9(4), 479–483 (1985)

    Article  Google Scholar 

  40. D. Smith, P. Holmes, L. Whitemore, D. Collins, T. Devonport: The effect of theoretically-based imagery scripts on field hockey performance, J. Sport Behav. 24(4), 408–419 (2001)

    Google Scholar 

  41. A. Guillot, E. Nadrowska, C. Collet: Using motor imagery to learn tactical movements in basketball, J. Sport Behav. 32(2), 27–29 (2009)

    Google Scholar 

  42. G. Wei, J. Luo: Sport expert’s motor imagery: Functional imaging of professional motor skills and simple motor skills, Brain Res. 1341, 52–62 (2010)

    Article  Google Scholar 

  43. M. Asada, K. Hosoda, Y. Kuniyoshi, H. Ishiguro, T. Inui, Y. Yoshikawa, M. Ogino, C. YoshidaL: Cognitive developmental robotics: A survey, Auton. Ment. Dev. IEEE Trans. 1(1), 12–34 (2009)

    Article  Google Scholar 

  44. A. Di Nuovo, V.M.D. De La Cruz Marocco: Special issue on artificial mental imagery in cognitive systems and robotics, Adapt. Behav. 21(4), 217–221 (2013)

    Article  Google Scholar 

  45. J. Tani: Model-based learning for mobile robot navigation from the dynamical systems perspective, IEEE Trans. Syst. Man. Cybern. 26(3), 421–436 (1996)

    Article  Google Scholar 

  46. D. Roy, K.-Y. Hsiao, N. Mavridis: Mental imagery for a conversational robot, Syst. Man. Cybern. Part B Cybern. IEEE Trans. 34(3), 1374–1383 (2004)

    Article  Google Scholar 

  47. T. Ziemke, D.-A. Jirenhed, G. Hesslow: Internal simulation of perception: A minimal neuro-robotic model, Neurocomputing 68(0), 85–104 (2005)

    Article  Google Scholar 

  48. R. Nishimoto, J. Tani: Development of hierarchical structures for actions and motor imagery: A constructivist view from synthetic neuro-robotics study, Psychol. Res. PRPF 73(4), 545–558 (2009)

    Article  Google Scholar 

  49. O. Gigliotta, G. Pezzulo, S. Nolfi: Evolution of a predictive internal model in an embodied and situated agent, Theory Biosci. 130(4), 259–276 (2011)

    Article  Google Scholar 

  50. H. Svensson, S. Thill, T. Ziemke: Dreaming of electric sheep? Exploring the functions of dream-like mechanisms in the development of mental imagery simulations, Adapt. Behav. 21(4), 222–238 (2013)

    Article  Google Scholar 

  51. A. Kaiser, W. Schenck, R. Möller: Solving the correspondence problem in stereo vision by internal simulation, Adapt. Behav. 21(4), 239–250 (2013)

    Article  Google Scholar 

  52. F. Chersi, F. Donnarumma, G. Pezzulo: Mental imagery in the navigation domain: A computational model of sensory-motor simulation mechanisms, Adapt. Behav. 21(4), 251–262 (2013)

    Article  Google Scholar 

  53. H. Iizuka, H. Ando, T. Maeda: Extended homeostatic adaptation model with metabolic causation in plasticity mechanism – toward constructing a dynamic neural network model for mental imagery, Adapt. Behav. 21(4), 263–273 (2013)

    Article  Google Scholar 

  54. S. Lallee, P.F. Dominey: Multi-modal convergence maps: From body schema and self-representation to mental imagery, Adapt. Behav. 21(4), 274–285 (2013)

    Article  Google Scholar 

  55. K. Seepanomwan, D. Caligiore, G. Baldassarre, A. Cangelosi: Modelling mental rotation in cognitive robots, Adapt. Behav. 21(4), 299–312 (2013)

    Article  Google Scholar 

  56. D. Lewkowicz, Y. Delevoye-Turrell, D. Bailly, P. Andry, P. Gaussier: Reading motor intention through mental imagery, Adapt. Behav. 21, 315–327 (2013)

    Article  Google Scholar 

  57. W. Gaona, E. Escobar, J. Hermosillo, B. Lara: Anticipation by multi-modal association through an artificial mental imagery process, Conn. Sci. 27, 68–88 (2015)

    Article  Google Scholar 

  58. A. Di Nuovo, D. Marocco, S. Di Nuovo, A. Cangelosi: Autonomous learning in humanoid robotics through mental imagery, Neural Netw. 41, 147–155 (2013)

    Article  Google Scholar 

  59. A. Di Nuovo, V.M. De La Cruz, S. Di Nuovo, D. Marocco, A. Cangelosi: Mental practice and verbal instructions execution: A cognitive robotics study, Proc. Int. Jt. Conf. Neural Netw. (IJCNN) (2012) pp. 2771–2776

    Google Scholar 

  60. A. Di Nuovo, D. Marocco, S. Di Nuovo, A. Cangelosi: A neural network model for spatial mental imagery investigation: A study with the humanoid robot platform iCub, Proc. Int. Jt. Conf. Neural Netw. (IJCNN) (2011) pp. 2199–2204

    Google Scholar 

  61. V. Tikhanoff, A. Cangelosi, P. Fitzpatrick, G. Metta, L. Natale, F. Nori: An open-source simulator for cognitive robotics research: The prototype of the iCub humanoid robot simulator, Proc. 8th Workshop Perform. Metr. Intell. Syst. (PerMIS) (2008) pp. 57–61

    Chapter  Google Scholar 

  62. R. Bartlett: Introduction to Sports Biomechanics: Analysing Human Movement Patterns (Routledge, London 2007)

    Google Scholar 

  63. E.P. Zehr, D.G. Sale: Ballistic movement: Muscle activation and neuromuscular adaptation, Can. J. Appl. Physiol. 19(4), 363–378 (1994)

    Article  Google Scholar 

  64. M.M. Botvinick, D.C. Plaut: Short-term memory for serial order: A recurrent neural network model, Psychol. Rev. 113(2), 201–233 (2006)

    Article  Google Scholar 

  65. M.I. Jordan: Attractor dynamics and parallelism in a connectionist sequential machine, Proc. 8th Annu. Conf. Cogn. Sci. Soc. (1986) pp. 531–546

    Google Scholar 

  66. J.L. Elman: Finding structure in time, Cogn. Sci. 14(2), 179–211 (1990)

    Article  Google Scholar 

  67. J. Decety: The neurophysiological basis of motor imagery, Behav. Brain Res. 77, 45–52 (1996)

    Article  Google Scholar 

  68. D.E. Rumelhart, J.L. McClelland: Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT Press, Cambridge 1986)

    Google Scholar 

  69. P.J. Werbos: Consistency of HDP applied to a simple reinforcement learning problem, Neural Netw. 3(2), 179–189 (1990)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by UK EPSRC Project BABEL and the European Commission FP7 Projects: POETICON++ (ICT-288382) within the Cognitive Systems, Interaction, Robotics unit (FP7 ICT Challenge 2), ROBOT-ERA (ICT-288899) within the ICT for Health, Ageing Well, Inclusion and Governance unit (FP7 ICT Challenge 5).

Author information

Authors and Affiliations

  1. Faculty of Engineering and Architecture, University of Enna “Kore”, Cittadella Universitaria, 94100, Enna, Italy

    Alessandro Di Nuovo

  2. Electronics and Mathematics, Plymouth University, School of Computing, Drake Circus, PL4 8AA, Plymouth, UK

    Davide Marocco

  3. Department of Education, University of Catania, 4 via Biblioteca, 95124, Catania, Italy

    Santo Di Nuovo

  4. Centre for Robotics and Neural Systems, Plymouth University, Drake Circus, PL4 8AA, Plymouth, UK

    Angelo Cangelosi

Authors
  1. Alessandro Di Nuovo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Davide Marocco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Santo Di Nuovo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angelo Cangelosi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Di Nuovo .

Editor information

Editors and Affiliations

  1. Department of Humanities, University of Pavia, Piazza Botta 6, 27100, Pavia, Italy

    Lorenzo Magnani  & Tommaso Bertolotti  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Di Nuovo, A., Marocco, D., Di Nuovo, S., Cangelosi, A. (2017). Embodied Mental Imagery in Cognitive Robots. In: Magnani, L., Bertolotti, T. (eds) Springer Handbook of Model-Based Science. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-30526-4_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30526-4_28

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-30525-7

  • Online ISBN: 978-3-319-30526-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation