MakeIt: Integrate User Interaction Times in the Design Process of Mobile Applications

  • Paul Holleis
  • Albrecht Schmidt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5013)


Besides key presses and text input, modern mobile devices support advanced interactions like taking pictures, gesturing, reading NFC-tags, as well as supporting physiological and environmental sensors. Implementing applications that benefit of this variety of interactions is still difficult. Support for developers and interaction designers remains basic and tools and frameworks are rare. This paper presents a prototyping environment that allows quickly and easily creating fully functional, high-fidelity prototypes deployable on the actual devices. With this work, we target the gap between paper prototyping and integrated development environments. Additionally, new interaction techniques can be significantly faster or slower to use than conventional mobile user interfaces. Hence it is essential to assess the impact of interface design decisions on interaction time. Additionally, the presented tool supports implicit and explicit user performance evaluations during all phases of prototyping. This approach builds on the original as well as extensions of the Keystroke-Level Model (KLM) which allows estimating interaction times in early phases of the development with a simulated prototype. An underlying state graph structure enables automatic checks of the application logic. This tool helps user interface designers and developers to create efficient and consistent novel applications.


Mobile Phone Mobile Application State Graph Interaction Technique Task Completion Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Davies, N., Landay, J., Hudson, S., Schmidt, A.: Rapid Prototyping in Ubiquitous Computing. IEEE Pervasive Computing 4(4), 15–17 (2005)CrossRefGoogle Scholar
  2. 2.
    Rukzio, E., Leichtenstern, K., Callaghan, V., Holleis, P., Schmidt, A.: An Experimental Comparison of Physical Mobile Interaction Techniques: Touching, Pointing and Scanning. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 87–104. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Rohs, M.: Marker-Based Interaction Techniques for Camera-Phones. In: MU3I (2005)Google Scholar
  4. 4.
    Nicolai, T., Kenn, H.: Towards Detecting Social Situations with Bluetooth. In: Adjunct Proceedings Ubicomp 2006 (2006)Google Scholar
  5. 5.
    Hull, R., Clayton, B., Melamed, T.: Rapid Authoring of Mediascapes. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 125–142. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Nuria, O., Flores-Mangas, F.: MPTrain: A Mobile Music and Physiology Based Personal Trainer. In: MobileHCI 2006 (2006)Google Scholar
  7. 7.
    Thimbleby, H., Gow, J.: Applying Graph Theory to Interaction Design. In: DSVIS 2007 (2007)Google Scholar
  8. 8.
    Holleis, P., Huhtala, J., Häkkilä, J.: Studying Applications for Touch-Enabled Mobile Phone Keypads. In: TEI 2008, pp. 15–18 (2008)Google Scholar
  9. 9.
    Holleis, P., Otto, F., Hussmann, H., Schmidt, A.: Keystroke-level Model for Advanced Mobile Phone Interaction. In: CHI 2007, pp. 1505–1514 (2007)Google Scholar
  10. 10.
    Rukzio, E., Wetzstein, S., Schmidt, A.: A Framework for Mobile Interactions with the Physical World. In: WPMC 2005 (2005)Google Scholar
  11. 11.
    Greenhalgh, Izadi, H.J.S., Mathrick, J., Taylor, I.: ECT: A Toolkit to Support Rapid Construction of Ubicomp Environments. In: UbiSys 2004 (2004)Google Scholar
  12. 12.
    Hartmann, B., Klemmer, S.R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., Gee, J.: Reflective Physical Prototyping Through Integrated Design, Test, and Analysis. In: UIST 2006 (2006)Google Scholar
  13. 13.
    Greenberg, S., Fitchett, C.: Phidgets: Easy Development of Physical Interfaces Through Physical Widgets. In: UIST 2001, pp. 209–218 (2001)Google Scholar
  14. 14.
    Ballagas, R., Ringel, M., Stone, M., Borchers, J.: iStuff: a Physical User Interface Toolkit for Ubiquitous Computing Environments. In: CHI 2003, pp. 537–544 (2003)Google Scholar
  15. 15.
    Villar, N., Gellersen, H.: A Malleable Control Structure for Softwired User Interfaces. In: TEI 2007 (2007)Google Scholar
  16. 16.
    Hull, R., Clayton, B., Melamed, T.: Rapid Authoring of Mediascapes. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 125–142. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Sohn, T., Dey, A.: iCAP: Rapid Prototyping of Context-Aware Applications. In: CHI 2004 (2004)Google Scholar
  18. 18.
    Li, Y., Hong, J., Landay, J.: Topiary: A Tool for Prototyping Location-Enhanced Applications. In: UIST 2004 (2004)Google Scholar
  19. 19.
    Newman, M.W., Lin, J., Hong, J.I., Landay, J.A.: DENIM: An Informal Web Site Design Tool Inspired by Observations of Practice. Human-Computer Int. 18(3), 259–324 (2003)CrossRefGoogle Scholar
  20. 20.
    Jameson, A., Krüger, A.: Preface to the Special Issue on User Modeling in Ubiquitous Computing. User Modeling and User-Adapted Interaction 15(3-4), 193–195 (2005)CrossRefGoogle Scholar
  21. 21.
    Card, S.K., Newell, A., Moran, T.P.: The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, Inc., Mahwah (1983)Google Scholar
  22. 22.
    Card, S.K., Moran, T.P., Newell, A.: The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM 23(7), 396–410 (1980)CrossRefGoogle Scholar
  23. 23.
    Hinckley, K., Guimbretière, F., Baudisch, P., Sarin, R., Agrawala, M., Cutrell, E.: The Springboard: Multiple Modes in one Spring-loaded Control. In: CHI 2006, pp. 181–190 (2006)Google Scholar
  24. 24.
    John, B.E., Vera, A.H.: A GOMS Analysis of a Graphic Machine-paced, Highly Interactive Task. In: CHI 1992, pp. 251–258 (1992)Google Scholar
  25. 25.
    Kieras, D.: Using the Keystroke-Level Model to Estimate Execution Times. The University of Michigan, Unpublished Report (1993),
  26. 26.
    Bälter, O.: Keystroke Level Analysis of Email Message Organization. In: CHI 2000 (2000)Google Scholar
  27. 27.
    Teo, L., John, B.E.: Comparisons of Keystroke-Level Model Predictions to Observed Data. In: Extended Abstracts CHI 2006, pp. 1421–1426 (2006)Google Scholar
  28. 28.
    Koester, H.H., Levine, S.P.: Validation of a Keystroke-Level Model for a Text Entry System Used by People with Disabilities. In: Assets 1994, pp. 115–122 (1994)Google Scholar
  29. 29.
    Manes, D., Green, P., Hunter, D.: Prediction of Destination Entry and Retrieval Times Using Keystroke-Level Models. UMTRI-96-37. University of Michigan (1996) Google Scholar
  30. 30.
    John, B.E., Salvucci, D.D.: Multi-Purpose Prototypes for Assessing User Interfaces in Pervasive Computing Systems. IEEE Pervasive Computing 4(4), 27–34 (2005)CrossRefGoogle Scholar
  31. 31.
    Klemmer, S.R., Sinha, A.K., Chen, J., Landay, J.A., et al.: SUEDE: A Wizard of Oz Prototyping Tool for Speech User Interfaces. In: CHI Letters UIST 2000, vol. 2(2), pp. 1–10 (2000)Google Scholar
  32. 32.
    Hong, J.I., Heer, J., Waterson, S., Landay, J.A.: WebQuilt: A Proxy-based Approach to Remote Web Usability Testing. ACM Trans. Inf. Syst. 19(3), 263–385 (2001)CrossRefGoogle Scholar
  33. 33.
    Gow, J., Thimbleby, H.: MAUI: An Interface Design Tool Based On Matrix Algebra. In: CADUI 2004, pp. 81–94 (2004)Google Scholar
  34. 34.
    Rekimoto, J., Schwesig, C.: PreSenseII: Bi-directional Touch and Pressure Sensing Interactions with Tactile Feedback. In: Extended Abstracts CHI 2006, pp. 1253–1258 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Paul Holleis
    • 1
  • Albrecht Schmidt
    • 1
  1. 1.Pervasive Computing and User Interface EngineeringUniversity of Duisburg-EssenGermany

Personalised recommendations