Advertisement

UX Metrics: Deriving Country-Specific Usage Patterns of a Website Plug-In from Web Analytics

  • Florian LachnerEmail author
  • Florian Fincke
  • Andreas Butz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10515)

Abstract

Metrics for User Experience (UX) often involve traditional usability aspects, such as task success, but also mental aspects, such as interpretation and meaning. The actual experience of a user also highly depends on personal characteristics, such as the social and cultural background. In this paper, we investigate the relation between users’ country of origin and their interaction patterns with an e-commerce website plug-in. We used a quantitative web analytics approach based on six UX-related metrics to evaluate the applicability of a quantitative UX evaluation approach in an international context. In a 34 day study we analyzed the usage patterns of 5.843 French, 2.760 German, and 5.548 Italian website visitors and found that they show significantly different patterns. This indicates that website metrics are a suitable means for cost-effective UX analysis on a large scale, which can provide valuable starting points for a further in-depth analysis.

Keywords

User experience Cross-cultural design User tracking Data logging Interfaces Globalization Localization 

References

  1. 1.
    Ahtinen, A., Ramiah, S., Blom, J., Isomursu, M.: Design of mobile wellness applications: identifying cross-cultural factors. In: Proceedings of the 20th Australasian Conference on Computer-Human Interaction (OZCHI), pp. 164–171. ACM Press (2008)Google Scholar
  2. 2.
    Al-Shamaileh, O., Sutcliffe, A.: Investigating a multi-faceted view of user experience. In: Proceedings of the 24th Australian Computer-Human Interaction Conference (OZCHI), pp. 9–18. ACM Press (2012)Google Scholar
  3. 3.
    Andreasen, M., Nielsen, H., Schrøder, S., Stage, J.: What happened to remote usability testing? An empirical study of three methods. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1405–1414, ACM Press (2007)Google Scholar
  4. 4.
    Aykin, N.: Overview: where to start and what to consider. In: Aykin, N. (ed.) Usability and Internationalization of Information Technology, pp. 3–20. Lawrence Erlbaum Associates Inc., New Jersey (2005)Google Scholar
  5. 5.
    Barber, W., Badre, A.: Culturability: The Merging of Culture and Usability (1998). http://research.microsoft.com/en-us/um/people/marycz/hfweb98/barber/
  6. 6.
    Bargas-Avila, J., Hornbæk, K.: Old wine in new bottles or novel challenges? A critical analysis of empirical studies of user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 2689–2698. ACM Press (2011)Google Scholar
  7. 7.
    Battarbee, K., Koskinen, I.: Co-experience: user experience as interaction. CoDesign 1(1), 5–18 (2005)CrossRefGoogle Scholar
  8. 8.
    Beaton, J., Kumar, R.: Indian cultural effects on user research methodologies. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pp. 4267–4271. ACM Press (2010)Google Scholar
  9. 9.
    Bjørneseth, F., Dunlop, M., Strand, J.: Dynamic positioning systems: usability and interaction styles. In: Proceedings of the 5th Nordic Conference on Human-Computer Interaction (NordiCHI), pp. 43–52. ACM Press (2008)Google Scholar
  10. 10.
    Chang, T.-H., Yeh, T., Miller, R.: GUI testing using computer vision. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1535–1544. ACM Press (2010)Google Scholar
  11. 11.
    Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Academic Press, New York (1998)zbMATHGoogle Scholar
  12. 12.
    Dunlop, M., Hamilton, I., Komninos, A., Nicol, E.: Shake ‘N’ Tap: a gesture enhanced keyboard for older adults. In: Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services (MobileHCI), pp. 525–530. ACM Press (2014)Google Scholar
  13. 13.
    Dunlop, M., Mcgregor, B., Elliot, M.: Using smartphones in cities to crowdsource dangerous road sections and give effective in-car warnings. In: Proceedings of the SEACHI 2016 on Smart Cities for Better Living with HCI and UX (SEACHI), pp. 14–18. ACM Press (2016)Google Scholar
  14. 14.
    Frandsen-Thorlacius, O., Hornbæk, K., Hertzum, M., Clemmensen, T.: Non-universal usability? A survey of how usability is understood by Chinese and Danish users. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 41–50. ACM Press (2009)Google Scholar
  15. 15.
    Gerea, C., Herskovic, V.: Measuring user experience in Latin America: an exploratory survey. In: Proceedings of the Latin American Conference on Human Computer Interaction (CLIHC). ACM Press (2015)Google Scholar
  16. 16.
    Gordon, M., Ouyang, T., Zhai, S.: WatchWriter. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 3817–3821. ACM Press (2016)Google Scholar
  17. 17.
    Gorman, T., Rose, E., Yaaqoubi, J., Bayor, A., Kolko, B.: Adapting usability testing for oral, rural users. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1437–1440. ACM Press (2011)Google Scholar
  18. 18.
    Grubbs, F.: Procedures for detecting outlying observations in samples. Technometrics 11(1), 1–21 (1974)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)CrossRefGoogle Scholar
  20. 20.
    Hassenzahl, M.: User experience (UX): towards an experiential perspective on product quality. In: Proceedings of the 20th Conference on l’Interaction Homme-Machine (IHM), pp. 11–15. ACM Press (2008)Google Scholar
  21. 21.
    He, Y., Zhao, C., Hinds, P.: Understanding information sharing from a cross-cultural perspective. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pp. 3823–3828. ACM Press (2010)Google Scholar
  22. 22.
    Hofstede, G., Hofstede, G.J., Minkov, M.: Cultures and Organizations. Software of the Mind. McGraw-Hill, New York (2010)Google Scholar
  23. 23.
    Hoßfeld, T., Keimel, C., Hirth, M., Gardlo, B., Habigt, J., Diepold, K., Tran-Gia, P.: Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Trans. Multimedia 16(2), 541–588 (2014)CrossRefGoogle Scholar
  24. 24.
    ISO DIS. 2009. 9241–210. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. International Standardization Organization (ISO). Switzerland (2009)Google Scholar
  25. 25.
    Jewell, C., Salvetti, F.: Towards a combined method of web usability testing: an assessment of the complementary advantages of lab testing, pre-session assignments, and online usability services. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pp. 1865–1870. ACM Press (2012)Google Scholar
  26. 26.
    Jordan, P.: Human factors for pleasure in product use. Appl. Ergon. 29(1), 25–33 (1998)CrossRefGoogle Scholar
  27. 27.
    Karapanos, E., Zimmerman, J., Forlizzi, J., Martens, J.: User experience over time: an initial framework. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 729–738. ACM Press (2009)Google Scholar
  28. 28.
    Kayes, I., Kourtellis, N., Quercia, D., Iamnitchi, A., Bonchi, F.: Cultures in community question answering. In: Proceedings of the 26th ACM Conference on Hypertext & Social Media, pp. 175–184. ACM Press (2015)Google Scholar
  29. 29.
    Komninos, A., Nicol, E., Dunlop, M.: Designed with older adults to SupportBetter error correction in SmartPhone text entry. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI), pp. 797–802. ACM Press (2015)Google Scholar
  30. 30.
    Lachner, F., Nägelein, P., Kowalski, R., Spann, M., Butz, A.: Quantified UX: towards a common organizational understanding of user experience. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI). ACM Press (2016)Google Scholar
  31. 31.
    Lallemand, C., Gronier, G., Koenig, V.: User experience: a concept without consensus? Exploring practitioners’ perspectives through an international survey. Comput. Human Behav. 43, 35–48 (2015)CrossRefGoogle Scholar
  32. 32.
    Law, E., Roto, V., Hassenzahl, M., Vermeeren, A., Kort, J.: Understanding, scoping and defining user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 719–728. ACM Press (2009)Google Scholar
  33. 33.
    Law, E., Van Schaik, P., Roto, V.: Attitudes towards user experience (UX) measurement. Int. J. Hum Comput Stud. 72(6), 526–541 (2014)CrossRefGoogle Scholar
  34. 34.
    Lindley, S., Wallace, J.: Placing in age: transitioning to a new home in later life. Trans. Comput.-Hum. Interact. 22(4), 1–40 (2015)CrossRefGoogle Scholar
  35. 35.
    Lindley, S.: Making time. In: 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW), pp. 1442–1452. ACM Press (2015)Google Scholar
  36. 36.
    Malinen, S., Nurkka, P.: The role of community in exercise: cross-cultural study of online exercise diary users. In: Proceedings of the 6th International Conference on Communities and Technologies (C&T), pp. 55–63. ACM Press (2013)Google Scholar
  37. 37.
    Marcus, A., Gould, E.: Crosscurrents: cultural dimensions and global web user-interface design. Interactions 7(4), 32–46 (2000)CrossRefGoogle Scholar
  38. 38.
    Mekler, E., Hornbæk, K.: Momentary pleasure or lasting meaning ? Distinguishing eudaimonic and hedonic user experiences. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 4509–4520. ACM Press (2016)Google Scholar
  39. 39.
    Nielsen, J.: Designing for international use (panel). In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 291–294. ACM Press (1990)Google Scholar
  40. 40.
    Norman, D.: Emotional design. Ubiquity 45(4), 1–1 (2004)CrossRefGoogle Scholar
  41. 41.
    Obrist, M., Roto, V., Väänänen-Vainio-Mattila, K.: User experience evaluation - do you know which method to use?. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 2763–2766. ACM Press (2009)Google Scholar
  42. 42.
    Obrist, M., Wurhofer, D., Gärtner, M., Förster, F., Tscheligi, M.: Exploring children’s 3DTV experience. In: Proceedings of the 10th European Conference on Interactive TV and Video (EuroiTV), pp. 125–134. ACM Press (2012)Google Scholar
  43. 43.
    Oliveira, N.: Culture-aware Q&A environments. In: Proceedings of DC CSCW Companion, pp. 101–104. ACM Press (2015)Google Scholar
  44. 44.
    Oliveira, N., Andrade, N., Reinecke, K.: Participation differences in Q&A sites across countries: opportunities for cultural adaptation. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI). ACM Press (2016)Google Scholar
  45. 45.
    Rea, L.M., Parker, R.A.: Designing and Conducting Survey Research: A Comprehensive Guide. Wiley, Hoboken (2014)Google Scholar
  46. 46.
    Reinecke, K., Bernstein, A.: Improving performance, perceived usability, and aesthetics with culturally adaptive user interfaces. Trans. Comput.-Hum. Interact. 18(2), 1–29 (2011)CrossRefGoogle Scholar
  47. 47.
    Reinecke, K., Bernstein, A.: Predicting user interface preferences of culturally ambiguous users. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA), pp. 3261–3266. ACM Press (2008)Google Scholar
  48. 48.
    Reinecke, K., Gajos, K.: Quantifying visual preferences around the world. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 11–20. ACM Press (2014)Google Scholar
  49. 49.
    Reyal, S., Zhai, S., Kristensson, P.: Performance and user experience of touchscreen and gesture keyboards in a lab setting and in the wild. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 679–688. ACM Press (2015)Google Scholar
  50. 50.
    Rodden, K., Hutchinson, H., Fu, X.: Measuring the user experience on a large scale: user-centered metrics for web applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 2395–2398. ACM Press (2010)Google Scholar
  51. 51.
    Rödel, C., Stadler, S., Meschtscherjakov, A., Tscheligi, M.: Towards autonomous cars: the effect of autonomy levels on acceptance and user experience. In: Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI), pp. 1–8. ACM Press (2014)Google Scholar
  52. 52.
    Roto, V., Law, E., Vermeeren, A., Hoonhout, J.: UX White Paper. Bringing clarity to the concept of user experience, Result from Dagstuhl Seminar on Demarcating User Experience (2011)Google Scholar
  53. 53.
    Sun, H.: Building a culturally-competent corporate web site: an exploratory study of cultural markers in multilingual web design. In: Proceedings of the 19th Annual International Conference On Computer Documentation (SIGDOC), pp. 95–102. ACM Press (2001)Google Scholar
  54. 54.
    Swallow, D., Blythe, M., Wright, P.: Grounding experience: relating theory and method to evaluate the user experience of smartphones. In: Proceedings of the 2005 Annual Conference on European Association of Cognitive Ergonomics (EACE), pp. 91–98. ACM Press (2005)Google Scholar
  55. 55.
    Tuch, A., Trusell, R., Hornbæk, K.: Analyzing users’ narratives to understand experience with interactive products. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 2079–2088. ACM Press (2013)Google Scholar
  56. 56.
    Tullis, T., Albert, B.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics, 2nd edn. Morgan Kaufmann, Waltham (2013)Google Scholar
  57. 57.
    Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., Väänänen-Vainio-Mattila, K.: User experience evaluation methods: current state and development needs. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction (NordiCHI), pp. 521–530. ACM Press (2010)Google Scholar
  58. 58.
    Walsh, T., Nurkka, P., Walsh, R.: Cultural differences in smartphone user experience evaluation. In: Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM), pp. 1–9. ACM Press (2010)Google Scholar
  59. 59.
    Walsh, T., Nurkka, P.: Approaches to cross-cultural design: two case studies with UX web-surveys. In: Proceedings of the 24th Australian Computer-Human Interaction Conference (OZCHI), pp. 633–642. ACM Press (2012)Google Scholar
  60. 60.
    Walsh, T., Vainio, T.: Cross-Cultural Design for mHealth Applications. In: Proceedings of the 23rd Australian Computer-Human Interaction Conference (OZCHI). ACM Press (2011)Google Scholar
  61. 61.
    Yatani, K., Novati, M., Trusty, A., Truong, K.: Review spotlight: a user interface for summarizing user-generated reviews using adjective-noun word pairs. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 1541–1550. ACM Press (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  1. 1.LMU Munich, Human-Computer Interaction GroupMunichGermany

Personalised recommendations