Skip to main content
SpringerLink
Log in

Field Experiments in HCI: Promises and Challenges

  • Chapter
  • First Online:
Future Interaction Design II

Abstract

Experimental methods have been under criticism since the advent of mobile and ubiquitous technologies, due to clear limitations in their suitability for studies in the field. However, the laboratory paradigm cannot be directly transferred to field conditions because of its strict notions of experimentation. This chapter examines the theory of quasi-experimentation as an alternative conceptualization of causality, control, and validity. Several threats to experimental validity in field experiments in HCI are discussed. These concerns must be addressed at all levels of experimentation, from the design and execution of a field experiment to analysis of data. Noteworthy also are new technical solutions that have enabled high-fidelity data collection and that generally support endeavors in ensuring validity. If field experimentation is to become the de facto standard of research in human–computer interaction, the methodological core and technical tools must be developed in concert.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In our studies, mirroring statistics gathered in Finland, only 45–75% (average by subject, 15 subjects, 3,969 total call attempts) of calls reached the intended receiver (Oulasvirta, Petit, Raento, & Tiitta, 2007).

  2. 2.

    This section is partly based on the manuscript Raento, M., Oulasvirta, A, & Eagle, N. (in press).

  3. 3.

    The study reported here has been designed and conducted with Sara Estlander and Antti Nurminen (Oulasvirta, Estlander, & Nurminen, in press).

References

  • Applied Science Laboratories. (n.d.). Head Mounted Optics. Retrieved July 30, 2007, from http://www.a-s-l.com/prod-head.htm

  • Card, S., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Cook, T., & Campbell, D. (1979). Quasi-experimentation: Design & analysis issues for field settings. New York: Houghton Mifflin.

    Google Scholar 

  • Darken, R., & Sibert, J. (1996). Wayfinding strategies and behaviors in large virtual worlds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’96, pp. 142–149). New York: ACM Press.

    Google Scholar 

  • Eagle, N., & Pentland, A. (2006). Reality mining: Sensing complex social systems. Personal and Ubiquitous Computing, 10, 255–268.

    Article  Google Scholar 

  • Engelbart, D. C., & English, W. K. (1988). A research center for augmenting human intellect. In I. Greif (Ed.), Computer-supported cooperative work: A book of readings (pp. 81–106). San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381–391.

    Article  Google Scholar 

  • Galison, P. (1987). How experiments end. Chicago: University of Chicago Press.

    Google Scholar 

  • Hacking, I. (1983). Representing and intervening: Introductory topics in the philosophy of natural science. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Ho, J., & Intille, S. (2005). Using context-aware computing to reduce the perceived burden of interruptions from mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05, pp. 909–918). New York: ACM Press.

    Google Scholar 

  • Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.

    Google Scholar 

  • Kant, I. (1999). Critique of pure reason. Cambridge, UK: Cambridge University Press. (Original work published in 1781).

    Google Scholar 

  • Kjeldskov J., & Graham, C. (2003). A review of mobile HCI research methods. In Proceedings of Mobile HCI 2003 (MobileHCI ’03, pp. 317–335). London, UK: Springer-Verlag.

    Google Scholar 

  • Kuutti, K. (1996). Activity theory as a potential framework for human-computer interaction research. In B. Nardi (Ed.), Context and consciousness: Activity theory and human-computer interaction (pp. 17–44). Cambridge, MA: MIT Press.

    Google Scholar 

  • Lyons, K., & Starner, T. (2001). Mobile capture for wearable computer usability testing. In Proceedings of IEEE International Symposium on Wearable Computing (ISWC 2001, pp. 69–76). Los Alamitos, CA: IEEE Computer Society.

    Google Scholar 

  • Nagel, E. (1979). The structure of science: Problems in the logic of scientific explanation. Indianapolis, IN: Hackett Publishing.

    Google Scholar 

  • Nielsen, J. (1993). Usability engineering. London, UK: Academic Press.

    MATH  Google Scholar 

  • Nielsen, J. (1995). Usability inspection methods. New York: ACM Press.

    Google Scholar 

  • Nurminen, A., & Oulasvirta, A. (2008). Designing interactions for navigation in 3D mobile maps. In L. Meng, A. Zipf, & S. Winter (Eds.), Map-based mobile services: Design, interaction and usability. Lecture Notes in Geoinformation and Cartography, (pp. 198–224). Springer: London.

    Google Scholar 

  • Oulasvirta, A., Estlander, S., & Nurminen, A. (2009). Embodied interaction with a 3D versus 2D mobile map. Personal and Ubiquitous Computing, 13(4).

    Google Scholar 

  • Oulasvirta, A., Nurminen, A., & Nivala, A. (2007). Interacting with 3D and 2D mobile maps: An exploratory study (Tech. Rep. No. 2007-1). Helsinki, Finland: Helsinki Institute for Information Technology.

    Google Scholar 

  • Oulasvirta, A., Petit, R., Raento, M., & Tiitta, S. (2007). Interpreting and acting on mobile awareness cues. Human-Computer Interaction, 22, 97–135.

    Google Scholar 

  • Oulasvirta, A., Raento, M., & Tiitta, S. (2005). ContextContacts: Re-designing SmartPhone’s contact book to support mobile awareness and collaboration. In Proceedings of Mobile HCI 2003 (MobileHCI ’03, pp. 167–174). New York: ACM Press.

    Google Scholar 

  • Oulasvirta, A., Tamminen, S., Roto, V., & Kuorelahti, J. (2005). Interaction in 4-second bursts: The fragmented nature of attentional resources. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05, pp. 919–928). New York: ACM Press.

    Google Scholar 

  • Pawson, R., & Tilley, N. (1997). Realistic evaluation. London, UK: Sage Publishing.

    Google Scholar 

  • Raento, M., Oulasvirta, A, & Eagle, N. (in press). Smartphone: An emerging tool for social scientists. Sociological Methods and Research.

    Google Scholar 

  • Raento, M., Oulasvirta, A., Petit, R., & Toivonen, H. (2005). ContextPhone: A prototyping platform for context-aware mobile applications. IEEE Pervasive Computing, 4, 51–59.

    Article  Google Scholar 

  • Reichl, P., Froehlich, P., Baillie, L., Schatz, R., & Dantcheva, A. (2007). The LiLiPUT prototype: A wearable environment for user tests of mobile telecommunication applications. In Extended Abtracts of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07; pp. 1833–1838). New York: ACM Press.

    Chapter  Google Scholar 

  • Roto, V., Oulasvirta, A., Haikarainen, T., Kuorelahti, J., Lehmuskallio, H., & Nyyssonen, T. (2004). Examining mobile phone use in the wild with quasi-experimentation (Tech. Rep. No. 2004-1). Helsinki, Finland: Helsinki Institute for Information Technology.

    Google Scholar 

  • Schusteritsch, R., Wei, C. Y., & LaRosa, M. (2007). Towards the perfect infrastructure for usability testing on mobile devices. In Extended Abstracts of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07, pp. 1839–1844). New York: ACM Press.

    Google Scholar 

  • Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs. Boston, MA: Houghton Mifflin.

    Google Scholar 

  • Suchman, L. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge, UK: Cambridge University Press.

    Google Scholar 

Download references

Acknowledgments

The author wishes to express gratitude to all collaborators, particularly Mika Raento, Sara Estlander, and Tuomo Nyyssönen. Parts of the text on background logging are based on an manuscript written with Mika Raento and Nathan Eagle. This research has been funded jointly by the FP6 EU project PASION (FP6-2004-IST-4-27654) and the Academy of Finland project ContextCues. The camera system described in Fig. 5 has been developed in the PASION project.

Author information

Authors and Affiliations

  1. Helsinki Institute for Information Technology [HIIT], Helsinki University of Technology and University of Helsinki, Helsinki, Finland

    Antti Oulasvirta

Authors
  1. Antti Oulasvirta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antti Oulasvirta .

Editor information

Editors and Affiliations

    Rights and permissions

    Reprints and permissions

    Copyright information

    © 2009 Springer-Verlag London Limited

    About this chapter

    Cite this chapter

    Oulasvirta, A. (2009). Field Experiments in HCI: Promises and Challenges. In: Isomäki, H., Saariluoma, P. (eds) Future Interaction Design II. Springer, London. https://doi.org/10.1007/978-1-84800-385-9_5

    Download citation

    • DOI: https://doi.org/10.1007/978-1-84800-385-9_5

    • Published:

    • Publisher Name: Springer, London

    • Print ISBN: 978-1-84800-300-2

    • Online ISBN: 978-1-84800-385-9

    • eBook Packages: Computer ScienceComputer Science (R0)

    Publish with us

    Policies and ethics

    Search

    Navigation