Advertisement

Measuring the Response Bias Induced by an Experience and Application Research Center

  • Boris de Ruyter
  • Rick van Geel
  • Panos Markopoulos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5859)

Abstract

In recent years we have observed the rise of Experience and Application Research centers (EARC). These EARCs simulate realistic environments and are used for the empirical evaluation of interactive systems in a controlled setting. Such laboratory environments are intended to facilitate data collection without influencing the data itself. Accumulated experience in the use of EARCs has raised concerns that test participants could be impressed by the environments and have raised expectations for advanced systems they expect to encounter; this brings about the danger of systematic bias in subjective report data collected with EARCs. To evaluate the impact of an EARC as an instrument, a controlled experiment with 40 test participants was conducted. This experiment involved the replication of a traditional usability test in both the EARC and a traditional laboratory environment. The results of this study provide evidence regarding the validity and reliability of EARCs as instruments for evaluating interactive systems.

Keywords

Mood State Usability Test Consumer Satisfaction Mental Workload Task Load 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aiello, A.: Scaling the heights of consumer satisfaction: An evaluation of alternate measures. In: Day, R.L. (ed.) Consumer Satisfaction, Dissatisfaction and Complaining Behavior. School of Business, Indiana University, Bloomington (1977)Google Scholar
  2. 2.
    Bell, R., Meiselman, H.L., Pierson, B.J., Reeve, W.G.: Effects of adding an Italian theme to a restaurant on perceived ethnicity, acceptability, and selection of foods. Appetite 22, 11–24 (1994)CrossRefGoogle Scholar
  3. 3.
    Carroll, J.: Human – Computer Interaction: Psychology as a science of design. Annual Review of Psychology, 48–61 (1997)Google Scholar
  4. 4.
    Churchill Jr., G.A., Surprenant, C.: An investigation into the determinants of customer satisfaction. Journal of Marketing Research 19, 491–504 (1982)CrossRefGoogle Scholar
  5. 5.
    Donovan, R.J., Rossiter, J.R.: Store atmosphere: An experimental psychology approach. Journal of Retailing 58, 34–57 (1982)Google Scholar
  6. 6.
    Foxall, G.R.: The emotional texture of consumer environments: A systematic approach to atmospherics. Journal of Economic Psychology 18, 505–523 (1997)CrossRefGoogle Scholar
  7. 7.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 239–250. North-Holland Press, Amsterdam (1988)Google Scholar
  8. 8.
    Kirakowski, J., Corbett, M.: SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology 24(3), 210 (1993)CrossRefGoogle Scholar
  9. 9.
    Kotler, P.: Atmospherics as a marketing tool. Journal of Retailing 49, 48–64 (1974)Google Scholar
  10. 10.
    Linda, G., Oliver, R.L.: Multiple brand analysis of expectation and disconfirmation effects on satisfaction. Paper presented at the 87th Annual Convention of the American Psychological Association (1979)Google Scholar
  11. 11.
    Mayer, J.D., Gaschke, Y.N.: The experience and meta-experience of Mood. Journal of Personality and Social Psychology 55(1), 102–111 (1988)CrossRefGoogle Scholar
  12. 12.
    Mehrabian, A., Russell, J.A.: An approach to environmental psychology. MIT Press, Cambridge (1974)Google Scholar
  13. 13.
    Meiselman, H.L.: The contextual basis for food acceptance, food choice and food intake: The food, the situation and the individual. In: Meiselman, H.L., MacFie, H.J.H. (eds.) Food choice acceptance and consumption, pp. 139–263. Blackie Academic and Professional, Glasgow (1996)Google Scholar
  14. 14.
    Nieuwenhuizen, K.: First Investigation of the Potential Halo Effect of High-End Usability Laboratories on User Evaluation Questionnaires, Stan Ackermans Institute, USI programme, graduation thesis, Eindhoven University of Technology, The Netherlands (2006)Google Scholar
  15. 15.
    Oliver, R.L.: Effect of expectation and disconfirmation on postexposure product evaluations: An alternative interpretation. Journal of Applied Psychology 62, 480–486 (1977)CrossRefGoogle Scholar
  16. 16.
    Petit, C., Sieffermann, J.M.: Testing consumer preferences for iced-coffee: Does the drinking environment have any influence? Food Quality and Preference 18, 161–172 (2007)CrossRefGoogle Scholar
  17. 17.
    Rubio, S., Díaz, E., Martín, J., Puente, J.M.: Evaluation of subjective mental workload: A comparison of SWAT, NASA-TLX, and workload profile methods. Applied Psychology: An international review 53(1), 61–86 (2004)CrossRefGoogle Scholar
  18. 18.
    Russel, S., Cousins, S.B.: IBM Almaden’s User Sciences & Experience Research Lab. In: Proceedings of the ACM Conference on Human Factors of Computer Systems, Vienna, Austria, April 24-29, pp. 1079–1080 (2004)Google Scholar
  19. 19.
    Swan, J.E., Trawick, I.F.: Consumer satisfaction with a retail store related to the fulfillment of expectations on an initial shopping trip. In: Day, R.L. (ed.) Consumer Satisfaction, Dissatisfaction and Complaining Behavior. School of Business, Indiana University, Bloomington (1980)Google Scholar
  20. 20.
    Westbrook, R.A.: Intrapersonal affective influences upon consumer satisfaction. Journal of Consumer Research 7, 49–54 (1980)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Boris de Ruyter
    • 1
  • Rick van Geel
    • 2
  • Panos Markopoulos
    • 3
  1. 1.Philips Research Europe, HTC34EindhovenThe Netherlands
  2. 2.University of TilburgThe Netherlands
  3. 3.Eindhoven UniversityEindhovenThe Netherlands

Personalised recommendations