Abstract
Evaluation is an essential part of development. There are several good reasons for carrying out user testing in particular. A successful evaluation requires careful planning. Here we describe the issues that you need to take into account and discuss several effective methods that can be used to collect data. User testing reduces the risk that you will deliver a system to your users that is unusable and is therefore ineffective. We also touch briefly on the need make sure that any evaluation that you carry out is conducted according to appropriate ethical guidelines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Berg, M. (1997). Rationalizing medical work: Decision support techniques and medical practices. Cambridge, MA: MIT Press.
Brooke, J. (1996). SUS: A ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability evaluation in industry (pp. 189–194). London: Taylor & Francis.
Brown, C. M. L. (1988). Human-computer interface design guidelines. Norwood, NJ: Ablex.
Calfee, R. C. (1985). Experimental methods in psychology. New York, NY: Holt, Rinehart and Winston.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.
Card, S. K., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum.
Carroll, J. M. (2000). Making use: Scenario-based design of human-computer interactions. Cambridge, MA: MIT Press.
Carroll, J. M., & Rosson, M. B. (1992). Getting around the task-artifact cycle: How to make claims and design by scenario. ACM Transactions on Information Systems, 10, 181–212.
Churchill, E. F., Nelson, L., Denoue, L., Murphy, P., & Helfman, J. I. (2003). The Plasma poster network: Social hypermedia on public display. In K. O’Hara, M. Perry, E. Churchill, & D. Russell (Eds.), Social and interactional aspects of shared display technologies. London: Kluwer Academic Publishers.
Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64.
Ericsson, K. A., & Simon, H. A. (1980). Protocol analysis: Verbal reports as data. Psychological Review, 87, 215–251.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (2nd ed.). Cambridge, MA: MIT Press.
Feurzeig, W., & Ritter, F. (1988). Understanding reflective problem solving. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned. Hillsdale, NJ: Erlbaum.
Goodman, E., Kuniavsky, M., & Moed, A. (2012). Observing the user experience: A practitioner’s guide to user research (2nd ed.). Waltham, MA: Morgan Kaufmann.
Hart, S. G., & Staveland, L. E. (1988). Development of the NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139–185). Amsterdam: North Holland.
Hock, R. R. (2002). Forty studies that changed psychology. Upper Saddle River, NJ: Prentice Hall.
Howell, D. C. (2010). Statistical methods for psychology (7th ed.). Belmont, CA: Wadsworth.
Kukreja, U., Stevenson, W. E., & Ritter, F. E. (2006). RUI—recording user input from interfaces under Windows and Mac OS X. Behavior Research Methods, 38(4), 656–659.
LaPiere, R. T. (1934). Attitude versus action. Social Forces, 13, 230–237.
Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research methods in human-computer interaction. New York, NY: Wiley.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2013). Qualitative data analysis: A methods sourcebook. Thousand Oaks, CA: Sage.
Molich, R., & Nielsen, J. (1990). Improving a human-computer dialogue. Communications of the ACM, 33(3), 338–348.
Monk, A., Wright, P., Haber, J., & Davenport, L. (1993). Apendix 1—cooperative evaluation: A run-time guide. In Improving your human-computer interface: A practical technique. New York: Prentice-Hall.
Monk, A. F. (1998). Lightweight techniques to encourage innovative user interface design. In L. Wood (Ed.), User interface design: Bridging the gap between user requirements and design (pp. 109–129). Boca Raton, FL: CRC Press.
Mosier, J. N., & Smith, S. L. (1986). Application of guidelines for designing user interface software. Behaviour and Information Technology, 5, 39–46.
Müller, M. J., Haslwanter, J. H., & Dayton, T. (1997). Participatory practices in the software lifecycle. In M. G. Helander, T. K. Landauer, & P. V. Prabhu (Eds.), Handbook of human-computer interaction (2nd ed., pp. 255–297). Amsterdam, NL: Elsevier.
NASA. (1987). NASA Task Load Index (TLX) V 1.0. Users Manual. Retrieved 10 March 2014, from http://humansystems.arc.nasa.gov/groups/TLX/downloads/TLX_comp_manual.pdf.
Navalpakkam, V., & Churchill, E. F. (in press). Eyetracking: A brief introduction. In J. S. Olson & W. Kellogg (Eds.), Ways of knowing, HCI methods. Heidelberg, Germany: Springer.
Nielsen, J. (1993). Usability engineering. Chestnut Hill, MA: AP Professional Press.
Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of CHI 90 (pp. 249–256). New York: ACM.
Nielsen, J., & Pernice, K. (2010). Eyetracking web usability. Berkeley, CA: New Riders.
Ray, W. J. (2008). Methods toward a science of behavior and experience (9th ed.). Belmont, CA: Wadsworth Publishing.
Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places. New York: NY Cambridge University Press.
Ritter, F. E., Kim, J. W., & Morgan, J. H. (2009). Running behavioral experiments with human participants: A practical guide (Tech. Report No. 2009-1). Applied Cognitive Science Lab: The Pennsylvania State University, College of Information Sciences and Technology.
Ritter, F. E., Kim, J. W., Morgan, J. H., & Carlson, R. A. (2013). Running behavioral studies with human participants: A practical guide. Thousand Oaks, CA: Sage.
Smallman, H. S., & St. John, M. (2005). Naïve realism: Misplaced faith in the utility of realistic displays. Ergonomics in Design, 13(Summer), 6–13.
Todd, Z. (Ed.). (2004). Mixing methods in psychology: The integration of qualitative and quantitative methods in theory and practice. Abingdon, UK: Psychology Press.
Tullis, T., & Albert, B. (2008). Measuring the user experience. Burlington, MA: Morgan Kaufmann.
Whiteside, J., Jones, S., Levy, P. S., & Wixon, D. (1985). User performance with command, menu, and iconic interfaces. In Proceedings of CHI’85 Human Factors in Computing Systems (185–191). New York: ACM.
Woods, D. D., & Dekker, S. W. A. (2000). Anticipating the effects of technological change: A new era of dynamics for human factors. Theoretical Issues in Ergonomic Science, 1(3), 272–282.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2014 Springer-Verlag London
About this chapter
Cite this chapter
Ritter, F.E., Baxter, G.D., Churchill, E.F. (2014). Methodology III: Empirical Evaluation . In: Foundations for Designing User-Centered Systems. Springer, London. https://doi.org/10.1007/978-1-4471-5134-0_13
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5134-0_13
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-5133-3
Online ISBN: 978-1-4471-5134-0
eBook Packages: Computer ScienceComputer Science (R0)