Knowledge as Embodied Performance

  • Satinder P. Gill
Part of the Human-Computer Interaction Series book series (HCIS)


This chapter presents an analysis of knowledge as a process of tacit knowing embodied in dialogue rather than as a process of information transfer embodied in signal processing (Shannon and Weaver 1949). Within dialogue we acquire knowledge (experiential knowledge, tacit knowing, practical knowledge) through our embodied performance of ’how ’ and ’what’ we know about the communication situation we are in. This embodied performance includes utterances, gestures, movement and modulation of body and voice, as well as mediating artefacts, such as interactive technologies. This chapter explores the implications of this perspective of knowledge for the conception and design of interactive technologies that seek to facilitate joint action and joint attention.


Joint Attention Interactive Technology Engagement Space Landscape Architect Conversational Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alibali, M.W. & Heath, D.C. (1999), Effects of visibility between speaker and listener on gesture production: some gestures are meant to be seen. Source of reference was this pre-published paper. (Later publication of this article: Alibali, M. W., Heath, D. C., & Myers, H. J. (2001) in Journal of Memory and Language, Vol. 44:169-188.)Google Scholar
  2. Allen, J. and Core, M. (1997). DAMSL: Dialog Act Markup in Several Layers. Unpublished manuscript.Google Scholar
  3. Allwood, J., Nivre, J., & Ahlsen, E. (1991). On the Semantics and Pragmatics of Linguistic Feedback. Technical Report No. 64. Gothenburg Papers on Theoretical Linguistics.Google Scholar
  4. Anderson, M.L. (2003) Embodied cognition: A field guide. Artificial Intelligence 149(1): 91-130.CrossRefGoogle Scholar
  5. Arom, S. (1991). African Polyphony and Polyrhythm: musical structure and methodology. Cambridge: Cambridge University Press.Google Scholar
  6. Bateson, G. (1955). “The Message. ’This is the Play.”’ In Schaffner, B. (ed.) Group Processes. Vol.II. New York: Macy.Google Scholar
  7. Bavelas, J.B. (1994). Gestures as Part of Speech: Methodological Implications. Research on Language and Social Interaction. 27(3): 201-221.CrossRefGoogle Scholar
  8. Bavelas, J.B., Chovil, N., Coates, L., Rose, L. (1995). Gestures Specialized for Dialogue. In PSPB, 21(4): 394-405.Google Scholar
  9. Bekerian, D.A. and Dennett, J.L. (1990). ‘Spoken and Written Recall of Visual Narratives’, Applied Cognitive Psychology, 175-187Google Scholar
  10. Birdwhistle, R.L. (1970). Kinesics and Context. University of PennsylvaniaGoogle Scholar
  11. Blacking, J. (1976) How Musical is Man? London: FaberGoogle Scholar
  12. Carletta, J., Isard, A., Isard S., Doherty-Sneddon, G, & Anderson, A. (1997). The Reliability of a Dialogue Structure Coding System. Association for Computational Linguistics. 23(1):13-31.Google Scholar
  13. Cassell, J., Sullivan,J., Prevost, S., Churchill, E. (2000) Embodied Conversational Agents. MIT Press.Google Scholar
  14. Clark, H.H. and Schaefer, E.F. (1989). Contributing to Discourse. Cognitive Science. 13:259-294.CrossRefGoogle Scholar
  15. Clark, H.H. (1996). Using Language. Cambridge: Cambridge University Press.Google Scholar
  16. Clayton, M., Sager, R., and Will, U. (2005). In Time with the Music: The Concept of Entrainment and its Significance for Ethnomusicology. In European Meetings in Ethnomusicology II, ESEM CounterPoint Vol.1.Google Scholar
  17. Condon, W. S., & Ogston, W. D. (1966). Sound Film Analysis of Normal and Pathological Behavior Patterns. Journal of Nervous and Mental Disease, 143:338-347.CrossRefGoogle Scholar
  18. Cross, I. (2007 in press). The Evolutionary Nature of Musical Meaning. In Musica Scientiae Google Scholar
  19. Di Paola, S. (17th April, 2001) Gesture and Narrative Creation in Avatar-Based 3D Virtual Communities). Invited paper at Gesture and Dialogue Seminar, CSLI, Stanford University. Scholar
  20. Ekman, P. and Friesen, W.V. (1969). The Repertoire of Nonverbal Behaviour: Categories, Origins, Usage, and Coding. In Semiotica. 1:49-98.Google Scholar
  21. Ekman, P. and Friesen, W.V. (1972). Hand Movements. In Journal of Communication. Vol. 22: 353-374.CrossRefGoogle Scholar
  22. Eilan, N., Hoerl, C., McCormack, T., Roessler, J. (2005). Joint Attention: Communication and Other Minds. Issues in Philosophy and Psychology. Oxford: Oxford University Press.Google Scholar
  23. Engle, R.A. 1998. Not Channels But Composite Signals: Speech, Gesture, Diagrams and Object Demonstrations Are Integrated in Multimodal Explanations. In Proceedings of the Twentieth Annual Conference of the Cognitive Science Society, pp. 321-327.Google Scholar
  24. Ferrari P.F., Fogassi L., Gallese V., Rizzolatti G. (2003) Mirror neurons responding to the observation of ingestive and communicative mouth actions in the monkey ventral premotor cortex. European Journal of Neuroscience, 17 (8):1703-1714.CrossRefGoogle Scholar
  25. Franco, F. (1997). The Development of Meaning in Infancy: Early Communication and Social Understanding. In S. Hala (Ed.) The Development of Social Cognition. Hove: Psychology Press.Google Scholar
  26. Fridlund, A.J. 1991. Sociality of Solitary Smiling: Potentiation by an Implicit Audience. In Journal of Personality and Social Psychology. 60.2: 229-240.CrossRefGoogle Scholar
  27. Gill, J.H. (2000). The Tacit Mode. Michael Polanyi’s Postmodern Philosophy. New York: SUNY PressGoogle Scholar
  28. Gill, K.S. (2007) (in this volume). Rethinking the Interaction Architecture. In SP Gill (Ed.) Cognition, Communication and Interaction. London: Springer.Google Scholar
  29. Gill, S.P. (1995) Dialogue and Tacit Knowledge for Knowledge Transfer. PhD Thesis, University of Cambridge.Google Scholar
  30. Gill, S.P. (1996) Aesthetic Design: Dialogue and Learning. A Case Study of Landscape Architecture. AI & Society, Vol. 9(3).Google Scholar
  31. Gill, S.P. (1998). Body Language: The unspoken dialogue of Bodies in Rhythm. In Proceedings of the ESSLI Workshop on Mutual Knowledge, Common Ground and Public Information.Google Scholar
  32. Gill, S.P., Kawamori, M., Katagiri, Y., Shimojima, A. (2000). The Role of Body Moves in Dialogue. In RASK, Vol.12: 89-114.Google Scholar
  33. Gill, S.P. (2002). The Parallel Coordinated Move: Case of a Conceptual Drawing Task. Published Working Paper: CKIR, Helsinki. ISBN.Google Scholar
  34. Gill, S.P., and Kawamori, M. (2002). Coordination of Gestures in a Non Face-to-Face Setting. In M. Rector, I. Poggi & N. Trigo (Eds.) Gestures, Meaning and Use. Porto: Fundacao Fernando Pessoa.Google Scholar
  35. Gill, S.P. and Borchers, J.O. (2004). Knowledge in Co-Action: Social Intelligence in Collaborative Design Activity. AI & Society, 17(3).Google Scholar
  36. Gill, S.P. (2004). Body Moves and Tacit Knowing. In Gorayska, B. and Mey, J.L. (Eds.) Cognition and Technology. John Benjamin.Google Scholar
  37. Gill, S.P. (2005) Pulse Periodicity in Paralinguistic Coordination. Presented at International Conference, Interacting Bodies, Lyon, 2005.Google Scholar
  38. Good, D.A. (1996). Pragmatics and Presence. AI & Society Journal. Vol.10 (3&4):309-14.CrossRefGoogle Scholar
  39. Goodwin, C. (1997). The Blackness of Black: Colour Categories as Situated Practice. In Lauren B. Resnick, Roger Saljo, Clotilde Pontecorvo, & Barbara Burge (Eds.), Discourse, Tools and Reasoning: Essays on Situated Cognition (pp.111-140). Berlin, Heidelberg, New York: Springer.Google Scholar
  40. Goranzon, B. (1993). The Practical Intellect: Computers and Skills (Artificial Intelligence and Society). Springer-Verlag.Google Scholar
  41. Guberina, P. (1985). The role of the body in learning foreign languages. Revue de Phonetique Applique: 37-50.Google Scholar
  42. Guimbretiere, F., Stone, M., Winograd, T. (2001). Stick it on the Wall: A Metaphor for Interaction with Large Displays. Submitted to Computer Graphics (SIGGRAPH 2001 Proceedings).Google Scholar
  43. Himberg, T. (2006). Co-operative tapping and collective time-keeping - differences of timing accuracy in duet performance with human or computer partner. Presentation at 9th International Conference on Music Perception and Cognition, Bologna, Italy.Google Scholar
  44. Hutchins, E. (1995) Cognition in the Wild. MIT Press: MasachusettsGoogle Scholar
  45. Ikuta, K. (1990) The Role of ‘Craft Language’ in Learning Waza. AI & Society, Vol. 4(2): 127-147.CrossRefGoogle Scholar
  46. Ishii, H. and Arita, K., (1991). ClearFace: Translucent Multiuser Interface for TeamWorkStation, In Proceedings of Second European Conference on Computer-Supported Cooperative Work (ECSCW ’91), Amsterdam, 25-27 September. pp. 163-174.Google Scholar
  47. Ishii, H. and Kobayashi, M. (1992). ClearBoard: A Seamless Media for Shared Drawing and Conversation with Eye-Contact. In Proceedings of Conference on Human Factors in Computing Systems (CHI ’92), ACM SIGCHI, Monterey, pp. 525-532.Google Scholar
  48. Ishii, H., Kobayashi, M. and Arita, K. (1994). Iterative Design of Seamless Collaboration Media,” Communications of the ACM (CACM), Special Issue on Internet Technology, ACM, Vol. 37, No. 8.pp. 83-97.Google Scholar
  49. Jeannerod, Marc. 1997. The Cognitive Neuroscience of Action. Oxford: Blackwell.Google Scholar
  50. Johannessen, K.S. (1988) Rule Following and Tacit Knowledge. AI & Society, Vol. 2(4): 287-303.CrossRefGoogle Scholar
  51. Kawamori, M., Shimazu, A., Kogure, K. (1994). Roles of Interjectory Utterances in Spoken Discourse. Proceedings of the International Conference on Spoken Language Processing.Google Scholar
  52. Kawamori, M., Kawabata, T., Shimazu, A. (1998). Discourse Markers in Spontaneous Dialogue: A corpus based study of Japanese and English. In Proceedings of 17th International Conference on Computational Linguistics (COLING-ACL98).Google Scholar
  53. Kendon, A. (1970). Movement Coordination in Social Interaction: Some examples described. In Acta Psychologia, 32:100-125.Google Scholar
  54. Kendon, A. (1972) Some Relationships Between Body Motion and Speech: An analysis of an example. In A. Seigman and B. Pope (eds.) Studies in Dyadic Communication. Elmsford, NY: Pergamon Press.Google Scholar
  55. Kendon, A. (1990) Conducting Interaction. Cambridge: Cambridge University PressGoogle Scholar
  56. Kendon, A. (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press.Google Scholar
  57. Kita, S. (1993). Language and thought interface: A study of spontaneous gestures and Japanese mimetics. Ph.D. thesis. Chicago, Illinois: University of Chicago.Google Scholar
  58. von Krogh, G., Ichijo, K., Nonaka, I. (2000). Enabling Knowledge Creation: How to Unlock the Mystery of Tacit Knowledge and Release the Power of Innovation. OUP: OxfordGoogle Scholar
  59. Large, E. and Jones, M.R. (1999) The Dynamics of Attending: How People Track Time-Varying Events. Pyschological Review. Vol. 106(1): 119-159CrossRefGoogle Scholar
  60. Maturana, H. and Varela, F. (1980). Autopoiesis and Cognition: The Realisation of the Living. Dordecht: D. Reidel Publishing Co.Google Scholar
  61. McNeill, D. (1992). Hand and Mind. Chicago: University of Chicago Press.Google Scholar
  62. McNeill, W. H. (1995). Keeping together in time. London: Harvard University PressGoogle Scholar
  63. Mey, J. (2001). Pragmatics. An Introduction. Oxford: Blackwell.Google Scholar
  64. Miall, D. S., & Dissanayake, E. (2003). “The Poetics of Babytalk.” Human Nature 14: 337-364.CrossRefGoogle Scholar
  65. McNeill, D., Cassell, J., McCullough, K-E. (1994). Communicative Effects of Speech-Mismatched Gestures. Research on Language and Social Interaction.Google Scholar
  66. Polanyi, M. (1964). Personal Knowledge: Towards a post critical philosophy. N.Y.: Harper and Row.Google Scholar
  67. Polanyi, M. (1966). The Tacit Dimension. Doubleday. Reprinted version, 1983, Gloucester, Mass.: Peter Smith.Google Scholar
  68. Pylyshyn, Z.W. (1984) Computation and Cognition. Toward a Foundation for Cognitive Science. Cambridge: CUP.Google Scholar
  69. Reiner, M. and Gilbert, J. (2004) The Symbiotic Roles of Empirical Experimentation and Thought Experimentation in the Learning of Physics. International Journal of Science Education, 26:1819-1834.CrossRefGoogle Scholar
  70. Rizzolatti G., Fogassi L., Gallese V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2: 661-670.CrossRefGoogle Scholar
  71. Saha, B. and Gangopadhyay, S. (2006) Building a Pedagogy around Action and Emotion: Experiences of Blind Opera of Kolkata. AI & Society, Vol. 21: 57-71.CrossRefGoogle Scholar
  72. Scheflen, A.E. (1974). How Behaviour Means. Exploring the contexts of speech and meaning: Kinesics, posture, interaction, setting, and culture. New York: Anchor Press/Doubleday.Google Scholar
  73. Schiffrin, D. (1987). Discourse Markers. Cambridge: Cambridge University Press.Google Scholar
  74. Sha, X.W. (2002) Resistance Is Fertile: Gesture and Agency in the Field of Responsive Media, in Makeover: Writing the Body into the Posthuman Technoscape, Configurations, 10(3): 439-472.CrossRefGoogle Scholar
  75. Sha, X-W and Gill, S.P. (2005). Gesture and Response in Field-Based Performance. In the ACM Proceedings of Creativity and Cognition 2005, Goldsmiths College, LondonGoogle Scholar
  76. Sha, X.W. (2005) “The TGarden Performance Research Project.” In Modern Drama, Special Issue: Technology. 48(3):585-608.MathSciNetGoogle Scholar
  77. Shannon, C. and Weaver, W. (1949). The mathematical theory of communication. Urbana, Il: University of Illinois Press.Google Scholar
  78. Shimojima, A., Katagiri, Y., Koiso, H. (1997). Scorekeeping for Conversation-Construction. In Proceedings of the Munich Workshop on Semantics and a Pragmatics of DialogueGoogle Scholar
  79. Simon, H. (Ed.) (1982) Models of bounded rationality. Behavioral economics and business organization, 2: 424-443. Cambridge, MA: MIT Press.Google Scholar
  80. Simon, H. (1969). The sciences of the artificial (1st ed). Cambridge, MA: MIT Press.Google Scholar
  81. Simon, H. (1983). Reason in human affairs. Stanford, CA: Stanford University PressGoogle Scholar
  82. Sperber, D. and Wilson, D. (1986). Relevance: communication and cognition. Oxford: Blackwell.Google Scholar
  83. Stobart, H. and Cross, I. (2000) The Andean Anacrusis? rhythmic structure and perception in Easter songs of Northern Potosí, Bolivia. British Journal of Ethnomusicology, 9(2): 63-94.CrossRefGoogle Scholar
  84. Tolbert, E. (2001) Music and Meaning: An Evolutionary Story. In Psychology of Music, Vol. 29, No. 1, 84-94.CrossRefGoogle Scholar
  85. Tilghman, B.R. (1988). Seeing and Seeing-As. AI & Society Journal, Vol.2(4): 303-319 London: Springer.Google Scholar
  86. Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28: 675-691.CrossRefGoogle Scholar
  87. Trevarthen C., Aitken K.J. (1994): Brain development, infant communication, and empathy disorders: Intrinsic factors in child mental health. In: Development and Psychopathology, 6: 597-633. (zit. nach Trevarthen, 1996)Google Scholar
  88. Trevarthen, C. (2005). “Disorganized rhythm and synchrony: Early signs of autism and Rett syndrome". In Brain & Development, 27: S25-S34.CrossRefGoogle Scholar
  89. Trevarthen, C. (2005). “First things first: infants make good use of the sympathetic rhythm of imitation, without reason or language". In Journal of Child Psychotherapy, 31(1):91-113.CrossRefGoogle Scholar
  90. Weizenbaum, J. (1966) ELIZA–A Computer Program For the Study of Natural Language Communication Between Man and MachineCommunications of the ACM. Volume 9(1) (January 1966): 36-35.Google Scholar
  91. Weizenbaum, J. (1976). Computer Power and Human Reason. San Francisco: WH Freeman.Google Scholar
  92. Wittgenstein, L. (1953). Philosophical Investigations.Google Scholar

Copyright information

© Springer-Verlag London Limited 2008

Authors and Affiliations

  • Satinder P. Gill
    • 1
  1. 1.Middlesex University, School of ComputingLondonUK

Personalised recommendations