TouchCounts and Gesture Design

  • Nicholas JackiwEmail author
  • Nathalie Sinclair
Part of the Human–Computer Interaction Series book series (HCIS)


TouchCounts is an iPad app for early number learning that illustrates how the cognitive facets can be mapped on to many of the affordances of the touchscreen tablet environment. Looking at the design of the mathematical gestures that learners use in the software’s interface to count and perform basic arithmetic operations, we find that theories of embodied cognition and gesture analysis (which focus on only sometimes deliberate hand and body motions as supporting acts of communication and reasoning) usefully expand formulations of gestures common in the software design discourse (which constitute gesture as highly deliberate, and unambiguous physical imperatives articulated within a specific software application’s command syntax). We develop this potential contribution in the context of educational software design, where the application purpose is to introduce users to novel conceptual domains, rather than to extend competence or enable expertise within already familiar ones.


  1. 1.
    Carlson, R.A., Avraamides, M.N., Cary, M., Strasberg, S.: What do the hands externalize in simple arithmetic? J. Exp. Psychol.: Learn. Mem. Cognit. 33(4), 747 (2007)Google Scholar
  2. 2.
    Cook, S.W., Mitchell, Z., Goldin-Meadow, S.: Gesturing makes learning last. Cognition 106(2), 1047–1058 (2008)CrossRefGoogle Scholar
  3. 3.
    Dubé, A.K., McEwen, R.N.: Do gestures matter? The implications of using touchscreen devices in mathematics instruction. Learn. Instruct. 40, 89–98 (2015)Google Scholar
  4. 4.
    Jackiw, N.: Touch and multitouch in dynamic geometry: sketchpad explorer and “digital” mathematics. In: Proceedings of the 11th International Conference on Technology in Mathematics Teaching, pp. 149–155 (2013)Google Scholar
  5. 5.
    McNeill, D.: Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press, Chicago/London (1992)Google Scholar
  6. 6.
    Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: International Gesture Workshop, pp. 409–420. Springer (2003)Google Scholar
  7. 7.
    Norman, D.A.: Natural user interfaces are not natural. Interactions 17(3), 6–10 (2010)CrossRefGoogle Scholar
  8. 8.
    Papert, S.: Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc. (1980)Google Scholar
  9. 9.
    Peirce, C.S., Welby, V.: Semiotic and Significs the Correspondence Between Charles S. Peirce and Lady Victoria Welby. Indiana University Press, Bloomington (1977)Google Scholar
  10. 10.
    Penner-Wilger, M., Anderson, M.L.: The relation between finger gnosis and mathematical ability: why redeployment of neural circuits best explains the finding. Front. Psychol. 4, 877 (2013)CrossRefGoogle Scholar
  11. 11.
    Sinclair, N., Freitas, E.D.: The haptic nature of gesture: rethinking gesture with new multitouch digital technologies. Gesture 14(3), 351–374 (2014)CrossRefGoogle Scholar
  12. 12.
    Sinclair, N., Jackiw, N.: Touchcounts [software application for the ipad]. Tangible Learning Project, SFU (2011)Google Scholar
  13. 13.
    Sinclair, N., Pimm, D.: Whatever be their number: counting on the visible, the audible, and the tangible. In: Meletiou-Mavrotheris, M., Mavrou, K., Paparistodemou, E. (eds.) Integrating Touch-Enabled and Mobile Devices into Contemporary Mathematics Education, p. 49. Information Science Reference, Hershey (2015)Google Scholar
  14. 14.
    Wigdor, D., Wixon, D.: Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Elsevier, Burlington (2011)Google Scholar
  15. 15.
    Wigdor, D., Fletcher, J., Morrison, G.: Designing user interfaces for multi-touch and gesture devices. In: CHI’09 Extended Abstracts on Human Factors in Computing Systems, pp. 2755–2758. ACM (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.SRI InternationalMenlo ParkUSA
  2. 2.Simon Fraser UniversityBurnabyCanada

Personalised recommendations