Measuring the Response Bias Induced by an Experience and Application Research Center
In recent years we have observed the rise of Experience and Application Research centers (EARC). These EARCs simulate realistic environments and are used for the empirical evaluation of interactive systems in a controlled setting. Such laboratory environments are intended to facilitate data collection without influencing the data itself. Accumulated experience in the use of EARCs has raised concerns that test participants could be impressed by the environments and have raised expectations for advanced systems they expect to encounter; this brings about the danger of systematic bias in subjective report data collected with EARCs. To evaluate the impact of an EARC as an instrument, a controlled experiment with 40 test participants was conducted. This experiment involved the replication of a traditional usability test in both the EARC and a traditional laboratory environment. The results of this study provide evidence regarding the validity and reliability of EARCs as instruments for evaluating interactive systems.
KeywordsMood State Usability Test Consumer Satisfaction Mental Workload Task Load
Unable to display preview. Download preview PDF.
- 1.Aiello, A.: Scaling the heights of consumer satisfaction: An evaluation of alternate measures. In: Day, R.L. (ed.) Consumer Satisfaction, Dissatisfaction and Complaining Behavior. School of Business, Indiana University, Bloomington (1977)Google Scholar
- 3.Carroll, J.: Human – Computer Interaction: Psychology as a science of design. Annual Review of Psychology, 48–61 (1997)Google Scholar
- 5.Donovan, R.J., Rossiter, J.R.: Store atmosphere: An experimental psychology approach. Journal of Retailing 58, 34–57 (1982)Google Scholar
- 7.Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 239–250. North-Holland Press, Amsterdam (1988)Google Scholar
- 9.Kotler, P.: Atmospherics as a marketing tool. Journal of Retailing 49, 48–64 (1974)Google Scholar
- 10.Linda, G., Oliver, R.L.: Multiple brand analysis of expectation and disconfirmation effects on satisfaction. Paper presented at the 87th Annual Convention of the American Psychological Association (1979)Google Scholar
- 12.Mehrabian, A., Russell, J.A.: An approach to environmental psychology. MIT Press, Cambridge (1974)Google Scholar
- 13.Meiselman, H.L.: The contextual basis for food acceptance, food choice and food intake: The food, the situation and the individual. In: Meiselman, H.L., MacFie, H.J.H. (eds.) Food choice acceptance and consumption, pp. 139–263. Blackie Academic and Professional, Glasgow (1996)Google Scholar
- 14.Nieuwenhuizen, K.: First Investigation of the Potential Halo Effect of High-End Usability Laboratories on User Evaluation Questionnaires, Stan Ackermans Institute, USI programme, graduation thesis, Eindhoven University of Technology, The Netherlands (2006)Google Scholar
- 18.Russel, S., Cousins, S.B.: IBM Almaden’s User Sciences & Experience Research Lab. In: Proceedings of the ACM Conference on Human Factors of Computer Systems, Vienna, Austria, April 24-29, pp. 1079–1080 (2004)Google Scholar
- 19.Swan, J.E., Trawick, I.F.: Consumer satisfaction with a retail store related to the fulfillment of expectations on an initial shopping trip. In: Day, R.L. (ed.) Consumer Satisfaction, Dissatisfaction and Complaining Behavior. School of Business, Indiana University, Bloomington (1980)Google Scholar