Prediction of Multi-touch Gestures during Input

  • Michael Schmidt
  • Gerhard Weber
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8511)


In the work at hand, a method is presented that can predict gestures during input. The scheme is based on the specification of prominent points defining subgestures within templates. Classification of a partial input is only against a small set of subgestures pre-selected by nearest neighbor searches regarding these prominent points. The gesture prediction is invariant against variations in scale, rotation, translation and speed of an input and handles single-touch, single-stroke and (sequential) multi-touch gestures. We provide thorough investigations of the classifiers performance on tests with two medium sized gesture sets. Results are promising and feasible for a wide range of applications. Even common direct manipulation operations can be reliably detected.


gestures multi-touch prediction classification template-based 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Appert, C., Bau, O.: Scale detection for a priori gesture recognition. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2010, pp. 879–882. ACM, New York (2010)Google Scholar
  2. 2.
    Bau, O., Mackay, W.E.: Octopocus: A dynamic guide for learning gesture-based command sets. In: UIST 2008: Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, pp. 37–46. ACM, New York (2008)Google Scholar
  3. 3.
    Bennett, M., McCarthy, K., O’Modhrain, S., Smyth, B.: Simpleflow: Enhancing gestural interaction with gesture prediction, abbreviation and autocompletion. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011, Part I. LNCS, vol. 6946, pp. 591–608. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Bentley, J.L.: K-d trees for semidynamic point sets. In: Proceedings of the Sixth Annual Symposium on Computational Geometry, SCG 1990, pp. 187–197. ACM, New York (1990)Google Scholar
  5. 5.
    Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization 10(2), 112–122 (1973)CrossRefGoogle Scholar
  6. 6.
    Deng, J.W., Tsui, H.T.: An hmm-based approach for gesture segmentation and recognition. In: Proceedings of the 15th International Conference on Pattern Recognition, vol. 3, pp. 679–682 (2000)Google Scholar
  7. 7.
    Freeman, D., Benko, H., Morris, M.R., Wigdor, D.: Shadowguides: Visualizations for in-situ learning of multi-touch and whole-hand gestures. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS 2009, pp. 165–172. ACM, New York (2009)Google Scholar
  8. 8.
    Giorgino, T.: Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package. Journal of Statistical Software 31(7), 1–24 (2009)Google Scholar
  9. 9.
    Henry, T.R., Hudson, S.E., Newell, G.L.: Integrating gesture and snapping into a user interface toolkit. In: Proceedings of the 3rd Annual ACM SIGGRAPH Symposium on User Interface Software and Technology, UIST 1990, pp. 112–122. ACM, New York (1990)Google Scholar
  10. 10.
    Kawashima, M., Shimada, A., Nagahara, H., Taniguchi, R.-I.: Adaptive template method for early recognition of gestures. In: 2011 17th Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), pp. 1–6 (2011)Google Scholar
  11. 11.
    Mori, A., Uchida, S., Kurazume, R., Taniguchi, R., Hasegawa, T., Sakoe, H.: Early recognition and prediction of gestures. In: Proceedings of the 18th International Conference on Pattern Recognition, ICPR 2006, vol. 3, pp. 560–563. IEEE Computer Society, Washington, DC (2006)Google Scholar
  12. 12.
    Ouyang, T.Y., Davis, R.: A visual approach to sketched symbol recognition. In: Proceedings of the 21st International Jont Conference on Artifical intelligence, IJCAI 2009, pp. 1463–1468. Morgan Kaufmann Publishers Inc., San Francisco (2009)Google Scholar
  13. 13.
    Ramer, U.: An iterative procedure for the polygonal approximation of plane curves. Computer Graphics and Image Processing 1(3), 244–256 (1972)CrossRefGoogle Scholar
  14. 14.
    Rubine, D.: The Automatic Recognition of Gestures. PhD thesis, Carnegie Mellon University (1991)Google Scholar
  15. 15.
    Schmidt, M., Fibich, A., Weber, G.: Mtis: A multi-touch text input system. In: Streitz, N., Stephanidis, C. (eds.) DAPI 2013. LNCS, vol. 8028, pp. 62–71. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Schmidt, M., Weber, G.: Multitouch Haptic Interaction. In: Stephanidis, C. (ed.) UAHCI 2009, Part II. LNCS, vol. 5615, pp. 574–582. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    Schmidt, M., Weber, G.: Template based classification of multi-touch gestures. Pattern Recognition 46(9), 2487–2496 (2013)CrossRefGoogle Scholar
  18. 18.
    Shneiderman, B.: Direct manipulation: A step beyond programming languages. Computer 16(8), 57–69 (1983)CrossRefGoogle Scholar
  19. 19.
    Tirkaz, C., Yanikoglu, B., Sezgin, T.M.: Sketched symbol recognition with auto-completion. Pattern Recognition 45(11), 3926–3937 (2012)CrossRefGoogle Scholar
  20. 20.
    Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In: UIST 2007: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, pp. 159–168. ACM, New York (2007)Google Scholar
  21. 21.
    Yang, J., Xu, Y.: Hidden markov model for gesture recognition. Technical Report CMU-RI-TR-94-10, Robotics Institute, Pittsburgh, PA (May 1994)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Michael Schmidt
    • 1
  • Gerhard Weber
    • 1
  1. 1.Institute of Applied Science, Human-Computer InteractionDresden University of TechnologyDresdenGermany

Personalised recommendations