Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

How to point and to interpret pointing gestures? Instructions can reduce pointer–observer misunderstandings

Abstract

In everyday communication, people often point. However, a pointing act is often misinterpreted as indicating a different spatial referent position than intended by the pointer. It has been suggested that this happens because pointers put the tip of the index finger close to the line joining the eye to the referent. However, the person interpreting the pointing act extrapolates the vector defined by the arm and index finger. As this line crosses the eye-referent line, it suggests a different referent position than the one that was meant. In this paper, we test this hypothesis by manipulating the geometry underlying the production and interpretation of pointing gestures. In Experiment 1, we compared naïve pointer-observed dyads with dyads in which the discrepancy between the vectors defining the production and interpretation of pointing acts has been reduced. As predicted, this reduced pointer–observer misunderstandings compared to the naïve control group. In Experiment 2, we tested whether pointers elevate their arms steeper than necessary to orient it toward the referent, because they visually steer their index finger tips onto the referents in their visual field. Misunderstandings between pointers and observers were smaller when pointers pointed without visual feedback. In sum, the results support the hypothesis that misunderstandings between (naïve) pointers and observers result from different spatial rules describing the production and interpretation of pointing gestures. Furthermore, we suggest that instructions that reduce the discrepancy between these spatial rules can improve communicating with pointing gestures.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    We report Greenhouse Geisser corrected p values but uncorrected dfs throughout the article.

  2. 2.

    One observer of a naive dyad and one observer of a head-on-shoulder dyad claimed to base referent estimates on the eye-finger line. The respective naïve dyad performed actually worse than the average naïve dyads. The errors of the respective head-on-shoulder dyad were about half the size of the errors of the remaining head-on-shoulder dyads.

References

  1. Alibali, M. W. (2005). Gesture in spatial cognition: expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5(4), 307–331. doi:10.1207/s15427633scc0504_2.

  2. Alibali, M. W., Kita, S., & Young, A. J. (2000). Gesture and the process of speech production: we think, therefore we gesture. Language and Cognitive Processes, 15(6), 593–613. doi:10.1080/016909600750040571.

  3. Alibali, M. W., Spencer, R. C., Knox, L., & Kita, S. (2011). Spontaneous gestures influence strategy choices in problem solving. Psychological Science, 22(9), 1138–1144. doi:10.1177/0956797611417722.

  4. Armstrong, L., & Marks, L. E. (1997). Differential effects of stimulus context on perceived length: implications for the horizontal-vertical illusion. Perception and Psychophysics, 59(8), 1200–1213. doi:10.3758/BF03214208.

  5. Bangerter, A. (2004). Using pointing and describing to achieve joint focus of attention in dialogue. Psychological Science, 15(6), 415–419. doi:10.1111/j.0956-7976.2004.00694.x.

  6. Bangerter, A., & Oppenheimer, D. M. (2006). Accuracy in detecting referents of pointing gestures unaccompanied by language. Gesture, 6(1), 85–102. doi:10.1075/gest.6.1.05ban.

  7. Behne, T., Liszkowski, U., Carpenter, M., & Tomasello, M. (2012). Twelve-month-olds’ comprehension and production of pointing. British Journal of Developmental Psychology, 30(3), 359–375. doi:10.1111/j.2044-835X.2011.02043.x.

  8. Biederman, I., & Shiffrar, M. M. (1987). Sexing day-old chicks: A case study and expert systems analysis of a difficult perceptual-learning task. Journal of Experimental Psychology. Learning, Memory, and Cognition, 13(4), 640–645. doi:10.1037/0278-7393.13.4.640.

  9. Butterworth, G. (2003). Pointing is the royal road to language for babies. In S. Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 9–33). Mahwah, NJ: Lawrence Erlbaum Associates.

  10. Butterworth, G., & Itakura, S. (2000). How the eyes, head and hand serve definite reference. British Journal of Developmental Psychology, 18(1), 25–50. doi:10.1348/026151000165553.

  11. Carpenter, M., Nagell, K., Tomasello, M., Butterworth, G. & Moore, C. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monographs of the Society for Research in Child Development, 63(4), i–vi, 1–143. doi:10.2307/1166214.

  12. Cleret de Langavant, L., Remy, P., Trinkler, I., McIntyre, J., Dupoux, E., Berthoz, A., & Bachoud-Lévi, A.-C. (2011). Behavioral and neural correlates of communication via pointing. PLoS ONE, 6(3), e17719. doi:10.1371/journal.pone.0017719.

  13. Coren, S. (1993). The lateral preference inventory for measurement of handedness, footedness, eyedness, and earedness: norms for young adults. Bulletin of the Psychonomic Society, 31(1), 1–3.

  14. Elliott, D., Dutoy, C., Andrew, M., Burkitt, J. J., Grierson, L. E., Lyons, J. L., … Bennett, S. J. (2014). The influence of visual feedback and prior knowledge about feedback on vertical aiming strategies. Journal of Motor Behavior, 46(6), 433–443. doi:10.1080/00222895.2014.933767.

  15. Herbort, O., & Kunde, W. (2016). Spatial (mis-)interpretation of pointing gestures to distal referents. Journal of Experimental Psychology: Human Perception and Performance, 42(1), 78–89. doi:10.1037/xhp0000126.

  16. Hermsdörfer, J., Li, Y., Randerath, J., Goldenberg, G., & Johannsen, L. (2012). Tool use without a tool: kinematic characteristics of pantomiming as compared to actual use and the effect of brain damage. Experimental Brain Research, 218(2), 201–214. doi:10.1007/s00221-012-3021-z.

  17. Hollingworth, H. L. (1910). The central tendency of judgment. The Journal of Philosophy, Psychology and Scientific Methods, 7(17), 461–469. doi:10.2307/2012819.

  18. Leavens, D. A., & Hopkins, W. D. (1999). The whole-hand point: the structure and function of pointing from a comparative perspective. Journal of Comparative Psychology, 113(4), 417–425. doi:10.1037/0735-7036.113.4.417.

  19. Lücking, A., Pfeiffer, T., & Rieser, H. (2015). Pointing and reference reconsidered. Journal of Pragmatics, 77, 56–79. doi:10.1016/j.pragma.2014.12.013.

  20. Mazzoni, P., & Krakauer, J. W. (2006). An implicit plan overrides an explicit strategy during visuomotor adaptation. The Journal of Neuroscience, 26(14), 3642–3645. doi:10.1523/JNEUROSCI.5317-05.2006.

  21. Pechmann, T., & Deutsch, W. (1982). The development of verbal and nonverbal devices for reference. Journal of Experimental Child Psychology, 34(2), 330–341. doi:10.1016/0022-0965(82)90050-9.

  22. Peeters, D., Chu, M., Holler, J., Hagoort, P., & Özyürek, A. (2015). Electrophysiological and kinematic correlates of communicative intent in the planning and production of pointing gestures and speech. Journal of Cognitive Neuroscience, 27(12), 2352–2368. doi:10.1162/jocn_a_00865.

  23. Poulter, D., Jackson, R., Wann, J., & Berry, D. (2005). The effect of learning condition on perceptual anticipation, awareness, and visual search. Human Movement Science, 24(3), 345–361. doi:10.1016/j.humov.2005.06.005.

  24. Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7(4), 226–231. doi:10.1111/j.1467-9280.1996.tb00364.x.

  25. Roth, W.-M. (2001). Gestures: their role in teaching and learning. Review of Educational Research, 71(3), 365–392. doi:10.3102/00346543071003365.

  26. Sanchez, D. J., & Reber, P. J. (2013). Explicit pre-training instruction does not improve implicit perceptual-motor sequence learning. Cognition, 126(3), 341–351. doi:10.1016/j.cognition.2012.11.006.

  27. Taylor, J. L., & McCloskey, D. (1988). Pointing. Behavioural Brain Research, 29(1–2), 1–5. doi:10.1016/0166-4328(88)90046-0.

  28. Wnuczko, M., & Kennedy, J. M. (2011). Pivots for pointing: visually-monitored pointing has higher arm elevations than pointing blindfolded. Journal of Experimental Psychology: Human Perception and Performance, 37(5), 1485–1491. doi:10.1037/a0024232.

  29. Zhen, W., & Gros-Louis, J. (2015). Caregivers provide more labeling responses to infants’ pointing than to infants’ object-directed vocalizations. Journal of Child Language, 42(3), 538–561. doi:10.1017/S0305000914000221.

Download references

Acknowledgements

We thank Georg Schüssler for technical support, Hanna Mathew and Felicitas Muth for assistance with the data collection.

Author information

Correspondence to Oliver Herbort.

Ethics declarations

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 41 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Herbort, O., Kunde, W. How to point and to interpret pointing gestures? Instructions can reduce pointer–observer misunderstandings. Psychological Research 82, 395–406 (2018). https://doi.org/10.1007/s00426-016-0824-8

Download citation