Skip to main content
SpringerLink
Log in

VLQoE: Video QoE instrumentation on the smartphone

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The usage of network-demanding applications is growing rapidly such as video streaming on mobile terminals. However, network and/or service providers might not guarantee the perceived quality for video streaming that demands high packet transmission rate. In order to satisfy the user expectations and to minimize user churn, it is important for network operators to infer the end-user perceived quality in video streaming. Today, the most reliable method to obtain end-user perceived quality is through subjective tests, and the preferred location is the user interface as it is the closest point of application to the end-user. The end-user perceived quality on video streaming is highly influenced by occasional freezes; technically the extraordinary time gaps between two consecutive pictures that are displayed to the user, i.e., high inter-picture time. In this paper, we present a QoE instrumentation for video streaming, VLQoE. We added functionality to the VLC player to record a set of metrics from the user interface, application-level, network-level, and from the available sensors of the device. To the best of our knowledge, VLQoE is the first tool of its kind that can be used in user experiments for video streaming. By using the tool, we present a two state model based on the inter-picture time, for the HTTP- and RTSP-based video streaming via 3.5G. Next, we studied the influence of inter-picture time on the user perceived quality through out a user study. We investigated the minimum user perceived inter-picture time, and the user response time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Barakovic S, Kapov LS (2013) Survey and challenges of QoE management issues in wireless networks. J Comput Netw Commun 2013, Article ID 165146, 28 pages

  2. Bonald T, Roberts JW (2012) Internet and the Erlang formula. SIGCOMM Comput Commun Rev 42(1):23–30

    Article  Google Scholar 

  3. Cherif W, Ksentini A, Negru D, Sidibe M (2011) APSQA: efficient real time video streaming QoE tool in a future media internet context. In: IEEE Conference in Multimedia and Expo (ICME), pp 1–6

  4. Cisco Visual Networking Index: Global mobile data traffic forecast update 2012–2017. [Online], Available: http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-.520862.html. Accessed Sept 2013

  5. Dalal AC, Bouchard AK, Cantor S, Guo Y, Johnson A (2012) Assessing QoE of on-demand TCP video streams in real time. In: IEEE International Conference on Communications (ICC), pp 1165–1170

  6. Darwin Streaming Server. [Online], Available: http://developer.apple.com/opensource/server/streaming. Accessed Mar 2013

  7. De la Cruz Ramos P, Vidal FG, Leal RP (2010) Perceived video quality estimation from spatial and temporal information contents and network performance parameters in IPTV. In: Fifth International Conference on Digital Telecommunications (ICDT), vol., no., June

  8. Dialogic Corporation Quality of Experience for Mobile Video Users. White Paper, Canada, 2009

  9. Egger S, Hossfeld T, Schatz R, Fiedler M (2012) Waiting times for quality of experience for web based services. In: Proceedings of the 4th international workshop on quality of multimedia experience

  10. Fenimore C, Libert J (2013) Perceptual effects of noise in digital video compression. [Online], Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.196.1791&rep=rep1&type=pdf. Accessed Sept 2013

  11. Fiedler M (2011) Traffic models for quality of experience assessment, tutorial at Proceedings of the 23rd international Teletraffic Congress. San Francisco

  12. French H, Lin J, Phan T, Dala AC (2011) Real time video QoE analysis of RTMP streams. 30th International IEEE Performance Computing and Communications Conference (IPCCC), pp 1–2

  13. Gardlo B, Ries M, Rupp M, Jarina R (2011) A QoE evaluation methodology for HD video streaming using social networking. In: IEEE International Symposium on Multimedia (ISM), pp 222–227

  14. Grondin S (2010) Timing and time perception: a review of recent behavioral and neuroscience findings and theoretical directions. Atten Percept Psychophys 72(3):561–582

    Article  Google Scholar 

  15. Hossfeld T, Schatz R, Egger S (2011) SOS: the MOS is not enough. In: Proceedings of the third international workshop on quality of multimedia experience, pp 131–136

  16. Hossfeld T, Schatz R, Seufert M, Hirth M, Zinner T, Tran-Gia P (2011) Quantification of YouTube QoE via crowdsourcing. In: IEEE international workshop on multimedia quality of experience-modeling, evaluation, and directions (MQoE 2011). USA

  17. Hossfeld T, Schatz R, Biersack E, Plissonneau L (2013) Internet video delivery in YouTube: from traffic measurements to quality of experience. Data Traffic Monit Anal Lect Notes Comput Sci 7754:264–301

    Article  Google Scholar 

  18. HTTP Live Streaming draft-pantos-http-live-streaming-02. [Online], Available: http://tools.ietf.org/pdf/draft-pantos-http-live-streaming-02.pdf. Accessed Sept 2013

  19. Huang J, Xu Q, Tiwana B, Mao ZM, Zhang M, Bahl P (2010) Anatomizing application performance differences on smartphones. In: Proceedings of the 8th international conference on mobile systems, applications, and services (MobiSys ’10). ACM, New York, pp 165–178

  20. Ickin S (2013) Identification of influential factors on Android Smartphone-based video quality of experience. Licentiate Dissertation, Sweden

  21. Ickin S, Janowski L, Wac K, Fiedler M (2012) Studying the challenges in assessing the perceived quality of mobile-phone based video. In: Fourth international workshop on Quality of Multimedia Experience (QoMEX), pp 164,169

  22. Ickin S, Wac K, Fiedler M, Janowski L, Hong JH, Dey AK (2012) Factors influencing quality of experience of commonly-used mobile applications. IEEE Commun Mag 50(4):48–56

    Article  Google Scholar 

  23. Intille S, Kukla C, Ma X (2002) Eliciting user preferences using image-based experience sampling and reflection. In: Proceedings of human factors in computing systems CHI

  24. ITU-T Rec. P. 800 (1996) Methods for subjective determination of transmission quality

  25. ITU-T Recommendation P.910Subjective video quality assessment methods for multimedia applications, Recommendations of the ITU (Telecommunication Standardization Sector), [Online], Available: http://www.itu.int/rec/T-REC-P.910-200804-I/en. Accessed Sept 2013

  26. Janowski L, Romaniak P, Papir Z (2011) Content driven QoE assessment for video frame rate and frame resolution reduction. In: Multimedia tools and applications, pp 1–18

  27. Juluri P, Plissonneau L, Medhi D (2011) Pytomo: a tool for analyzing playback quality of YouTube videos. In: 23rd International Teletraffic Congress (ITC), pp 304,305, 6–9

  28. Katabi D, Handley M, Rohrs C (2002) Congestion control for high bandwidth-delay product networks. In: Proceedings of the conference on applications, technologies, architectures, and protocols for computer communications (SIGCOMM ’02). ACM, New York, pp 89–102

  29. Khan A, Sun L, Jammeh E, Ifeachor E (2010) Quality of experience-driven adaptation scheme for video applications over wireless networks. IET Commun 4(11):1337–1347

    Article  Google Scholar 

  30. Laghari KR, Connelly K (2012) Toward total quality of experience: a QoE model in a communication ecosystem. IEEE Commun Mag 50(4):58–65

    Article  Google Scholar 

  31. Latre S, De Turck F (2012) Autonomic quality of experience management of multimedia networks. In: IEEE IFIP network operations and management symposium, pp 872–879

  32. Le Callet P, Moller S, Perkis A (eds) Qualinet white paper on definitions of quality of experience (QoE). [Online], Available: http://www.qualinet.eu/. Accessed Mar 2013

  33. Li M, Li F, Claypool M, Kinicki R (2005) Weather forecasting - predicting performance for streaming video over wireless LANs. In: Proceeedings of NOSSDAV. Stevenson, Washington

  34. Menkovski V (2013) Intelligent control for adaptive video streaming. In: Proceedings of the international conference on consumer electronics. Las Vegas

  35. Menkovski V, Liotta A (2013) QoE for mobile streaming, mobile multimedia - user and technology perspectives, InTech Europe, (2012), [Online], Available: http://cdn.intechopen.com/pdfs/26696/InTech-Qoe_for_mobile_streaming.pdf. Accessed: September

  36. Menkovski V, Exarchakos G, Liotta A (2011) The value of relative quality in video delivery. J Mob Multimed 7(3):151–162

    Google Scholar 

  37. Migliorini D, Mingozzi E, Vallati C (2010) QoE oriented performance evaluation of video streaming over WiMAX. In: Eighth international conference on wired/wireless Internet communications. Sweden

  38. Minhas T (2012) Network impact on quality of experience of mobile video, Licentiate dissertation, Blekinge Institute of Technology. Sweden

  39. Musser G (2013) Time on the brain: how you are always living in the past, and other quirks of perception. [Online], Available: http://blogs.scientificamerican.com/observations/2011/09/15/time-on-the-brain-how-you-are-always-living-in-the-past-and-other-quirks-of-perception/. Accessed Apr

  40. Myung IJ (2003) Tutorial on maximum likelihood estimation. J Math Psychol 47(1):90–100

    Article  MATH  MathSciNet  Google Scholar 

  41. Oliver E (2010) The challenges in large-scale smartphone user studies. In: Proceedings of the 2nd ACM international workshop on hot topics in planet-scale measurement. ACM, New York, Article 5, 5 p

  42. PEVQ Perceptual Evaluation of Video Quality. [Online], Available: http://www.pevq.org/. Accessed Mar 2013

  43. Qiao Z (2011) Smarter phone based live QoE measurement. In: 15th international conference on intelligence in next generation networks

  44. Quan HT, Ghanbari M (2008) Temporal aspect of perceived quality of mobile video broadcasting. IEEE Trans Broadcast 54(3):641–651

    Article  Google Scholar 

  45. RFC2326: Real Time Streaming Protocol (RTSP). [Online], Available: http://www.ietf.org/rfc/rfc2326.txt. Accessed Sept 2013

  46. Seow SC (2008) Designing and engineering time: the psychology of time perception in software. Addison-Wesley Professional

  47. Singh KD, HadjadjAoul Y, Rubino G (2012) Quality of experience estimation for adoptive HTTP/TCP video streaming using H.264/AVC. In: IEEE Consumer Communications & Networking Conference, (CCNC), pp 127–131

  48. Schatz R, Hossfeld T, Janowski L, Egger S (2013) From packets to people: quality of experience as a new measurement challenge. Data Traffic Monit Anal Lect Notes Comput Sci 7754:219–263

    Article  Google Scholar 

  49. Staehle B, Hirth M, Pries R, Wamser F, Staehle D (2010) YoMo: a Youtube application comfort monitoring tool. In: Proceedings of EuroITV Workshop QoE for Multimedia Content Sharing (QoEMCS’10). Tampere

  50. Staelens N, Moens S, Van den Broeck W, Marien I, Vermeulen B, Lambert P, Van De Walle R, Demeester P (2010) Assessing quality of experience of IPTV and video on demand services in real-life environments. IEEE Trans Broadcast 56(4):458–466

    Article  Google Scholar 

  51. Top Project Listings. VideoLAN, Retrieved: 2013-01-22

  52. Van Kester S, Xiao T, Kooij RE, Brunnstrm K, Ahmed OK (2011) Estimating the impact of single and multiple freezes on video quality. In: Proceedings of SPIE, vol 7865

  53. Venkataraman M, Chatterja M (2009) Evaluating quality of experience for streaming video in real time. In: IEEE GLOBECOM

  54. Vidal RP, Gicquel J, Colomes C, Cherifi H (2004) Sporadic frame dropping impact on quality perception. Human Vis Electron Imaging IX, vol 5292

  55. Vishwanath A, Dutta P, Chetlur M, Gupta P, Kalyanaraman S, Ghosh A (2010) Perspectives on quality of experience for video streaming over WiMAX. ACM SIGMOBILE Mobile Comput Commun Rev 13(4):15–25

    Article  Google Scholar 

  56. VLC Media Player for Android (2013) [Online], Available, http://www.videolan.org/vlc/download-android.html. Accessed Mar 2013

  57. Voice quality and MOS. [Online], Available: http://searchunifiedcommunications.techtarget.com/tip/Voice-quality-and-MOS. Accessed Apr 2013

  58. Vuppala A, Sriram LN (2011) Measurement of user related performance problems of live video streaming in user interface. Masters Thesis, Blekinge Institute of Technology

  59. Wang Y, Claypool M (2005) RealTracer - tools for measuring the performance of RealVideo on the internet. In: Kluwer multimedia tools and applications, vol 27, issue no 3

  60. Wang Z, Lu L, Bovik AC (2004) Video quality assessment based on structural distortion measurement. Proc Sig Process Image Commun Spec Issue Objective Video Qual Metrics 19(2):121–132

    Google Scholar 

  61. Wang B, Kurose J, Shenoy P, Towsley D (2008) Multimedia streaming via TCP: an analytic performance study. ACM Trans Multimedia Comput Commun Appl 4, 2, Article 16, 22 p

  62. Winkler S, Mohandas P (2008) The evaluation of video quality measurement: from PSNR to hybrid metrics. IEEE Trans Broadcast 54(3):660–668

    Article  Google Scholar 

  63. World Wide Smartphone Sales Share. [Online], Available: http://en.wikipedia.org/wiki/File:World_Wide_Smartphone_Sales_Share.png. Accessed Apr 2013

  64. Yang KC, Guest CC, El-Maleh K, Das PK (2007) Perceptual temporal quality metric for compressed video. IEEE Trans Multimedia 9

  65. Zinner T, Abboud O, Hohlfeld O, Hossfeld T, Tran Gia P (2010) Towards QoE management for scalable video streaming. In: 21st ITC specialist seminar on multimedia applications, traffic, performance and QoE. Japan

  66. Zupernick H, et al. (2008) Quality of experience based cross-layer design of mobile video systems. In: Proceedings of the 18th ITC specialist seminar on quality of experience

Download references

Acknowledgments

The work of Katarzyna Wac has been supported by AAL-MyGuardian and WayFIS projects. We would like to thank anonymous reviewers for the valuable comments and suggestions to improve our paper.

Author information

Authors and Affiliations

  1. School of Computing, Blekinge Institute of Technology, Blekinge, Sweden

    Selim Ickin, Markus Fiedler & Patrik Arlos

  2. Institute of Services Science, University of Geneva, Geneva, Switzerland

    Katarzyna Wac

  3. Blekinge Institute of Technology, Blekinge, Sweden

    Canberk Temiz

  4. Department of Electronics and Telecommunications Engineering, University of Dar es Salaam, Dar es Salaam, Tanzania

    Khadija Mkocha

Authors
  1. Selim Ickin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Fiedler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katarzyna Wac
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patrik Arlos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Canberk Temiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Khadija Mkocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Selim Ickin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ickin, S., Fiedler, M., Wac, K. et al. VLQoE: Video QoE instrumentation on the smartphone. Multimed Tools Appl 74, 381–411 (2015). https://doi.org/10.1007/s11042-014-1919-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-1919-0

Keywords

Search

Navigation