Narrowing the Gap Between QoS Metrics and Web QoE Using Above-the-fold Metrics

  • Diego Neves da HoraEmail author
  • Alemnew Sheferaw Asrese
  • Vassilis Christophides
  • Renata Teixeira
  • Dario Rossi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10771)


Page load time (PLT) is still the most common application Quality of Service (QoS) metric to estimate the Quality of Experience (QoE) of Web users . Yet, recent literature abounds with proposals for alternative metrics (e.g., Above The Fold, SpeedIndex and their variants) that aim at better estimating user QoE. The main purpose of this work is thus to thoroughly investigate a mapping between established and recently proposed objective metrics and user QoE. We obtain ground truth QoE via user experiments where we collect and analyze 3,400 Web accesses annotated with QoS metrics and explicit user ratings in a scale of 1 to 5, which we make available to the community. In particular, we contrast domain expert models (such as ITU-T and IQX) fed with a single QoS metric, to models trained using our ground-truth dataset over multiple QoS metrics as features. Results of our experiments show that, albeit very simple, expert models have a comparable accuracy to machine learning approaches. Furthermore, the model accuracy improves considerably when building per-page QoE models, which may raise scalability concerns as we discuss.



We are grateful to our shepherd Mike Wittie and to the anonymous reviewers, whose useful comments helped us improving our work. This work has been carried out at LINCS ( and benefited from support of NewNet@Paris, Ciscos Chair “Networks for the Future” at Telecom ParisTech and the EU Marie curie ITN program METRICS (grant no. 607728).


  1. 1.
  2. 2.
  3. 3.
  4. 4.
    Alexa Internet Inc.
  5. 5.
    Approximate ATF chrome extension.
  6. 6.
    Bampis, C.G., Bovik, A.C.: Learning to predict streaming video QoE: distortions, rebuffering and memory. CoRR, abs/1703.00633 (2017)Google Scholar
  7. 7.
    Belshe, M., Peon, R., et al.: Hypertext Transfer Protocol Version 2 (HTTP/2). RFC 7540 (2015)Google Scholar
  8. 8.
    Bocchi, E., De Cicco, L., et al.: Measuring the quality of experience of web users. In: ACM SIGCOMM CCR (2016)Google Scholar
  9. 9.
    Bocchi, E., De Cicco, L., Mellia, M., Rossi, D.: The web, the users, and the MOS: influence of HTTP/2 on user experience. In: Kaafar, M.A., Uhlig, S., Amann, J. (eds.) PAM 2017. LNCS, vol. 10176, pp. 47–59. Springer, Cham (2017). CrossRefGoogle Scholar
  10. 10.
    Brutlag, J., Abrams, Z., et al.: Above the fold time: Measuring web page performance visually (2011)Google Scholar
  11. 11.
    Butkiewicz, M., Madhyastha, H.V., et al.: Characterizing web page complexity and its impact. IEEE/ACM Trans. Netw. 22(3), 943 (2014)CrossRefGoogle Scholar
  12. 12.
    Charonyktakis, P., Plakia, M., et al.: On user-centric modular QoE prediction for VoIP based on machine-learning algorithms. IEEE Trans. Mob. Comput. 15, 1443–1456 (2016)CrossRefGoogle Scholar
  13. 13.
    Erman, J., Gopalakrishnan, V., et al.: Towards a SPDY’ier mobile web? In: ACM CoNEXT, pp. 303–314 (2013)Google Scholar
  14. 14.
    Fiedler, M., Hoßfeld, T., et al.: A generic quantitative relationship between quality of experience and quality of service. IEEE Netw. 24(2), 36 (2010)CrossRefGoogle Scholar
  15. 15.
    Gao, Q., Dey, P., et al.: Perceived performance of top retail webpages in the wild: insights from large-scale crowdsourcing of above-the-fold QoE. In: Proceedings of ACM Internet-QoE Workshop (2017)Google Scholar
  16. 16.
    Google: SPDY, an experimental protocol for a faster web.
  17. 17.
    ITU-T: Estimating end-to-end performance in IP networks for data application (2014)Google Scholar
  18. 18.
    Kelton, C., Ryoo, J., et al.: Improving user perceived page load time using gaze. In: Proceedings of USENIX NSDI (2017)Google Scholar
  19. 19.
    Langley, A., Riddoch, A., et al.: The QUIC transport protocol: design and internet-scale deployment. In: Proceedings of ACM SIGCOMM (2017)Google Scholar
  20. 20.
    Minutes of TPAC Web Performance WG meeting.
  21. 21.
    Qian, F., Gopalakrishnan, V., et al.: TM3: flexible transport-layer multi-pipe multiplexing middlebox without head-of-line blocking. In: ACM CoNEXT (2015)Google Scholar
  22. 22.
    Schatz, R., Hoßfeld, T., Janowski, L., Egger, S.: From packets to people: quality of experience as a new measurement challenge. In: Biersack, E., Callegari, C., Matijasevic, M. (eds.) Data Traffic Monitoring and Analysis. LNCS, vol. 7754, pp. 219–263. Springer, Heidelberg (2013). CrossRefGoogle Scholar
  23. 23.
    Spetebroot, T., Afra, S., et al.: From network-level measurements to expected quality of experience: the Skype use case. In: M & N Workshop (2015)Google Scholar
  24. 24.
    Varvello, M., Blackburn, J., et al.: EYEORG: a platform for crowdsourcing web quality of experience measurements. In: Proceedings of ACM CoNEXT (2016)Google Scholar
  25. 25.
    Varvello, M., Schomp, K., et al.: Is The Web HTTP/2 Yet?. In: Proceedings of PAM (2016)Google Scholar
  26. 26.
    Wang, X.S., Balasubramanian, A., et al.: How speedy is SPDY? In: USENIX NSDI, pp. 387–399. USENIX Association, Seattle (2014)Google Scholar
  27. 27.
    Wang, X.S., Krishnamurthy, A., et al.: Speeding up web page loads with Shandian. In: USENIX NSDI (2016)Google Scholar
  28. 28.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Telecom ParistechParisFrance
  2. 2.InriaParisFrance
  3. 3.Aalto UniversityEspoFinland

Personalised recommendations