Abstract
User behavior models see increased usage in automated evaluation and design of user interfaces (UIs). Obtaining training data for the models is costly, since it generally requires the involvement of human subjects. For interaction’s subjective quality parameters, like aesthetic impressions, it is even inevitable. In our paper, we study applicability of trained user behavior models between different domains of websites. We collected subjective assessments of Aesthetics, Complexity and Orderliness from 137 human participants for more than 3000 homepages from 7 domains, and used them to train 21 artificial neural network (ANN) models. The input neurons were 32 quantitative metrics obtained via computer vision-based analysis of the homepages screenshots. Then, we tested how well each ANN model can predict subjective assessments for websites from other domains, and correlated the changes in prediction accuracies with the pairwise distances between the domains. We found that the Complexity scale was rather domain-independent, whereas “foreign-domain” models for Aesthetics and Orderliness had on average greater prediction errors for other domains, by 60% and 45%, respectively. The results of our study provide web designers and engineers with a first framework to assess the reusability and difference in prediction accuracy of the models, for more informed decisions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
using http://curlie.org/.
- 4.
Our full implementation is available at https://colab.research.google.com/drive/1PFFMkE9vSE7aWBlKdFSLEu0jnSQX7fHw.
References
Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 152–158 (1994)
Nebeling, M. et al.: Crowdstudy: general toolkit for crowdsourced evaluation of web interfaces. In: Proceedings of the 5th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 255–264 (2013)
Frick, T.: Designing for Sustainability: a Guide to Building Greener Digital Products and Services. O’Reilly Media, Inc., Sebastopol (2016)
Nielsen, J.: Putting A/B Testing in Its Place. Nielsen Norman Group, 14 August 2005. https://www.nngroup.com/articles/putting-ab-testing-in-its-place/. Accessed 13 Jan 2020
Beirekdar, A., Keita, M., Noirhomme, M., Randolet, F., Vanderdonckt, J., Mariage, C.: Flexible reporting for automated usability and accessibility evaluation of web sites. In: Costabile, M.F., Paternò, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 281–294. Springer, Heidelberg (2005). https://doi.org/10.1007/11555261_25
Bakaev, M.: Assessing similarity for case-based web user interface design. In: Alexandrov, D.A., Boukhanovsky, A.V., Chugunov, A.V., Kabanov, Y., Koltsova, O. (eds.) DTGS 2018, Part I. CCIS, vol. 858, pp. 353–365. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02843-5_28
Grigera, J., et al.: Automatic detection of usability smells in web applications. Int. J. Hum Comput Stud. 97, 129–148 (2017)
Speicher, M., Both, A., Gaedke, M.: Ensuring web interface quality through usability-based split testing. In: Casteleyn, S., Rossi, G., Winckler, M. (eds.) ICWE 2014. LNCS, vol. 8541, pp. 93–110. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08245-5_6
Nielsen, J.: The Negativity Bias in User Experience. Nielsen Norman Group, 23 October 2016. https://www.nngroup.com/articles/negativity-bias-ux/. Accessed 14 Jan 2020
Speicher, M. et al.: TellMyRelevance! predicting the relevance of web search results from cursor interactions. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pp. 1281–1290 (2013)
Chen, X., et al.: The emergence of interactive behaviour: a model of rational menu search. In: Proceedings of the 33rd ACM Conference on Human Factors in Computing Systems, pp. 4217–4226 (2015)
Lin, Y.-P., Jung, T.-P.: Improving EEG-based emotion classification using conditional transfer learning. Front. Hum. Neurosci. 11, 334 (2017)
Torrey, L., Shavlik, J.: Transfer learning. In: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, pp. 242–264. IGI Global (2010)
Bakaev, M., et al.: Auto-extraction and integration of metrics for web user interfaces. J. Web Eng. 17(6), 561–590 (2018)
Estuka, F., Miller, J.: A pure visual approach for automatically extracting and aligning structured web data. ACM Trans. Internet Technol. 19(4), 1–26 (2019)
Oulasvirta, A. et al.: Aalto Interface Metrics (AIM): a service and codebase for computational GUI evaluation. In: 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings, pp. 16–19. ACM (2018)
Kim, D. et al: Domain switch-aware holistic recurrent neural network for modeling multi-domain user behavior. In: 12th ACM International Conference on Web Search and Data Mining, pp. 663–671 (2019)
Dou, Q., et al.: Webthetics: quantifying webpage aesthetics with deep learning. Int. J. Hum Comput Stud. 124, 56–66 (2019)
Boychuk, E., Bakaev, M.: Entropy and compression based analysis of web user interfaces. In: Bakaev, M., Frasincar, F., Ko, I.-Y. (eds.) ICWE 2019. LNCS, vol. 11496, pp. 253–261. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19274-7_19
Hachenberg, C., Gottron, T.: Locality sensitive hashing for scalable structural classification and clustering of web documents. In: Proceedings of the ACM CIKM (2013)
Acknowledgment
The reported study was funded by RFBR and DST according to the research project No. 19-57-45006. We thank Vladimir Khvorostov from NSTU for his technical work on collecting the screenshots, the assessments, and the metrics. We are also grateful to all the colleagues who participated and organized assessments of websites.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bakaev, M., Speicher, M., Heil, S., Gaedke, M. (2020). I Don’t Have That Much Data! Reusing User Behavior Models for Websites from Different Domains. In: Bielikova, M., Mikkonen, T., Pautasso, C. (eds) Web Engineering. ICWE 2020. Lecture Notes in Computer Science(), vol 12128. Springer, Cham. https://doi.org/10.1007/978-3-030-50578-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-50578-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50577-6
Online ISBN: 978-3-030-50578-3
eBook Packages: Computer ScienceComputer Science (R0)