Abstract
Page load time (PLT) is still the most common application Quality of Service (QoS) metric to estimate the Quality of Experience (QoE) of Web users . Yet, recent literature abounds with proposals for alternative metrics (e.g., Above The Fold, SpeedIndex and their variants) that aim at better estimating user QoE. The main purpose of this work is thus to thoroughly investigate a mapping between established and recently proposed objective metrics and user QoE. We obtain ground truth QoE via user experiments where we collect and analyze 3,400 Web accesses annotated with QoS metrics and explicit user ratings in a scale of 1 to 5, which we make available to the community. In particular, we contrast domain expert models (such as ITU-T and IQX) fed with a single QoS metric, to models trained using our ground-truth dataset over multiple QoS metrics as features. Results of our experiments show that, albeit very simple, expert models have a comparable accuracy to machine learning approaches. Furthermore, the model accuracy improves considerably when building per-page QoE models, which may raise scalability concerns as we discuss.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
https://googlewebmastercentral.blogspot.fr/2010/04/using-site-speed-in-web-search-ranking.html
http://googleresearch.blogspot.fr/2009/06/speed-matters.html
https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index
Alexa Internet Inc. http://www.alexa.com
Approximate ATF chrome extension. https://github.com/TeamRossi/ATF
Bampis, C.G., Bovik, A.C.: Learning to predict streaming video QoE: distortions, rebuffering and memory. CoRR, abs/1703.00633 (2017)
Belshe, M., Peon, R., et al.: Hypertext Transfer Protocol Version 2 (HTTP/2). RFC 7540 (2015)
Bocchi, E., De Cicco, L., et al.: Measuring the quality of experience of web users. In: ACM SIGCOMM CCR (2016)
Bocchi, E., De Cicco, L., Mellia, M., Rossi, D.: The web, the users, and the MOS: influence of HTTP/2 on user experience. In: Kaafar, M.A., Uhlig, S., Amann, J. (eds.) PAM 2017. LNCS, vol. 10176, pp. 47–59. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54328-4_4
Brutlag, J., Abrams, Z., et al.: Above the fold time: Measuring web page performance visually (2011)
Butkiewicz, M., Madhyastha, H.V., et al.: Characterizing web page complexity and its impact. IEEE/ACM Trans. Netw. 22(3), 943 (2014)
Charonyktakis, P., Plakia, M., et al.: On user-centric modular QoE prediction for VoIP based on machine-learning algorithms. IEEE Trans. Mob. Comput. 15, 1443–1456 (2016)
Erman, J., Gopalakrishnan, V., et al.: Towards a SPDY’ier mobile web? In: ACM CoNEXT, pp. 303–314 (2013)
Fiedler, M., Hoßfeld, T., et al.: A generic quantitative relationship between quality of experience and quality of service. IEEE Netw. 24(2), 36 (2010)
Gao, Q., Dey, P., et al.: Perceived performance of top retail webpages in the wild: insights from large-scale crowdsourcing of above-the-fold QoE. In: Proceedings of ACM Internet-QoE Workshop (2017)
Google: SPDY, an experimental protocol for a faster web. https://www.chromium.org/spdy/spdy-whitepaper
ITU-T: Estimating end-to-end performance in IP networks for data application (2014)
Kelton, C., Ryoo, J., et al.: Improving user perceived page load time using gaze. In: Proceedings of USENIX NSDI (2017)
Langley, A., Riddoch, A., et al.: The QUIC transport protocol: design and internet-scale deployment. In: Proceedings of ACM SIGCOMM (2017)
Minutes of TPAC Web Performance WG meeting. https://www.w3.org/2016/09/23-webperf-minutes.html
Qian, F., Gopalakrishnan, V., et al.: TM3: flexible transport-layer multi-pipe multiplexing middlebox without head-of-line blocking. In: ACM CoNEXT (2015)
Schatz, R., Hoßfeld, T., Janowski, L., Egger, S.: From packets to people: quality of experience as a new measurement challenge. In: Biersack, E., Callegari, C., Matijasevic, M. (eds.) Data Traffic Monitoring and Analysis. LNCS, vol. 7754, pp. 219–263. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36784-7_10
Spetebroot, T., Afra, S., et al.: From network-level measurements to expected quality of experience: the Skype use case. In: M & N Workshop (2015)
Varvello, M., Blackburn, J., et al.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: Proceedings of ACM CoNEXT (2016)
Varvello, M., Schomp, K., et al.: Is The Web HTTP/2 Yet?. In: Proceedings of PAM (2016)
Wang, X.S., Balasubramanian, A., et al.: How speedy is SPDY? In: USENIX NSDI, pp. 387–399. USENIX Association, Seattle (2014)
Wang, X.S., Krishnamurthy, A., et al.: Speeding up web page loads with Shandian. In: USENIX NSDI (2016)
Web QoE dataset. https://newnet.telecom-paristech.fr/index.php/webqoe/
Acknowledgments
We are grateful to our shepherd Mike Wittie and to the anonymous reviewers, whose useful comments helped us improving our work. This work has been carried out at LINCS (http://www.lincs.fr) and benefited from support of NewNet@Paris, Ciscos Chair “Networks for the Future” at Telecom ParisTech and the EU Marie curie ITN program METRICS (grant no. 607728).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D. (2018). Narrowing the Gap Between QoS Metrics and Web QoE Using Above-the-fold Metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds) Passive and Active Measurement. PAM 2018. Lecture Notes in Computer Science(), vol 10771. Springer, Cham. https://doi.org/10.1007/978-3-319-76481-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-76481-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76480-1
Online ISBN: 978-3-319-76481-8
eBook Packages: Computer ScienceComputer Science (R0)