Advertisement

QoE Active Control

  • Vlado MenkovskiEmail author
Chapter
  • 255 Downloads
Part of the Springer Theses book series (Springer Theses)

Abstract

Adaptive networked multimedia services deliver superior overall quality by optimizing a trade-off between the available resources and the delivered quality. Typically this means achieving an efficient system where the maxium amount of users are serviced with the sufficient quality. Successful management also entails adjusting to changes in the environment. Reacting to increase in demand or to the requirements of new devices is essential to meeting customer expectations. However, not all management decisions can be taken reactively. Most services are faced with real-time fluctuations in available resources and real-time requirements that need to be met, which makes real-time adaptation a crucial function. In this chapter we present a framework for QoE aware active control of multimedia services, based on optimal control and reinforcement learning (RL). To demonstrate the applicability of our approach we implement a decision support solution using the suggested approach for the case of adaptive video streaming client.

Keywords

Reinforcement Learning Video Streaming Network Throughput Exponentially Weighted Move Average Reinforcement Learning Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    M. Puterman, Markov decision processes: Discrete stochastic dynamic programming. (Wiley, 1994)Google Scholar
  2. 2.
    ISO/IEC 23009-1:2012 - information technology–dynamic adaptive streaming over HTTP (DASH)–part 1: Media presentation description and segment formats. [Online], http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=57623
  3. 3.
    G. Van der Auwera, P. David, M. Reisslein, L. Karam, Traffic and quality characterization of the h. 264/avc scalable video coding extension. Adv. Multimed. 2008(2), 1 (2008)CrossRefGoogle Scholar
  4. 4.
    I. Sodagar, The mpeg-dash standard for multimedia streaming over the internet. IEEE Multimedia 18(4), 62–67 (2011)CrossRefGoogle Scholar
  5. 5.
    F. Bertone, V. Menkovski, and A. Liotta, in Streaming Media with Peer-to-Peer Networks. Adaptive P2P streaming, IGI Global, 2012, pp. 52–73. [Online], http://www.igi-global.com/chapter/adaptive-p2p-streaming/66305
  6. 6.
    S. Akhshabi, A. Begen, C. Dovrolis, An experimental evaluation of rate-adaptation algorithms in adaptive streaming over http. ACM MMSys 11, 157–168 (2011)Google Scholar
  7. 7.
    C. Müller, S. Lederer, and C. Timmerer, An evaluation of dynamic adaptive streaming over http in vehicular environments, in Proceedings of the 4th Workshop on Mobile Video. ACM, 2012, pp. 37–42Google Scholar
  8. 8.
    R. Sutton and A. Barto, Reinforcement Learning: An Introduction, vol. 1, no. 1. (Cambridge Univ Press, 1998)Google Scholar
  9. 9.
    E. Alpaydin, Introduction to Machine Learning. (MIT press, 2004)Google Scholar
  10. 10.
    T. Cormen, C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms. (MIT press, 2001)Google Scholar
  11. 11.
    V. Menkovski and A. Liotta, in Signal Processing: Image Communication. Adaptive Psychometric Scaling for Video Quality Assessment, 2012Google Scholar
  12. 12.
    J. Hamilton, Time series analysis, vol. 2. (Cambridge Univ Press, 1994)Google Scholar
  13. 13.
    E. Goldoni, G. Rossi, P. Gamba, Improving Available Bandwidth Estimation Using Averaging Filtering Techniques (Università degli Studi di Pavia, Laboratorio Reti, Rapport technique, 2008)Google Scholar
  14. 14.
    G. Barnard, Control charts and stochastic processes. J. Roy. Stat. Soc. Ser. B (Methodological), 239–271 (1959)Google Scholar
  15. 15.
    L. Burgstahler and M. Neubauer, New modifications of the exponential moving average algorithm for bandwidth estimation, in Proceedings of the 15th ITC Specialist Seminar, 2002Google Scholar
  16. 16.
    M. Kim and B. Noble, Mobile network estimation, in Proceedings of the 7th annual international conference on Mobile computing and networking. ACM, 2001, pp. 298–309Google Scholar
  17. 17.
    Z. Sun, D. He, L. Liang, and H. Cruickshank, Internet qos and traffic modelling IET. Software IEE Proc. 151(5), 248–255 (2004)Google Scholar
  18. 18.
    M. Crovella, A. Bestavros, Self-similarity in world wide web traffic: evidence and possible causes. IEEE/ACM Trans. Netw. 5(6), 835–846 (1997)CrossRefGoogle Scholar
  19. 19.
    W. Leland, M. Taqqu, W. Willinger, D. Wilson, On the self-similar nature of ethernet traffic. ACM SIGCOMM Comput. Commun. Rev. (ACM) 23(4), 183–193 (1993)CrossRefGoogle Scholar
  20. 20.
    M. Zukerman, T. Neame, R. Addie, Internet traffic modeling and future technology implications, in INFOCOM, Twenty-Second Annual Joint Conference of the IEEE Computer and Communications. IEEE Societies, vol. 1. IEEE 2003, 587–596 (2003)Google Scholar
  21. 21.
    V. Paxson, S. Floyd, Wide area traffic: the failure of poisson modeling. IEEE/ACM Trans. Netw. (ToN) 3(3), 226–244 (1995)CrossRefGoogle Scholar
  22. 22.
    V. Menkovski, A. Liotta, Intelligent control for adaptive video streaming, in Proceedings of the International Conference on Consumer Electronics, Las Vegas, US, January 11–14, 2013 (IEEE), doi:http://dx.doi.org/10.1109/ICCE.2013.6486825
  23. 23.
    M. Torres Vega, E. Giordano, D. C. Mocanu, D. Tjondronegoro, A. Liotta, Cognitive No-reference video quality assessment for mobile streaming services, in Proceedings of the 7th International Workshop on Quality of Multimedia Experience, Messinia, Greece, 26–29 May 2015 (IEEE), http://www.qomex.org
  24. 24.
    M. Torres Vega, S. Zou, D. Constantin Mocanu, E. Tangdiongga, A.M.J. Koonen, A. Liotta, End-to-End Performance Evaluation in High-Speed Wireless Networks, in Proceedings of the 10th International Conference on Network and Service Management, Rio de Janeiro, Brazil, 17–21 November 2014 (IEEE)Google Scholar
  25. 25.
    D. Constantin Mocanu, G. Santandrea, W. Cerroni, F. Callegati, A. Liotta, Network Performance Assessment with Quality of Experience Benchmarks, in Proceedings of the 10th International Conference on Network and Service Management, Rio de Janeiro, Brazil, 17–21 November 2014 (IEEE)Google Scholar
  26. 26.
    D. Constantin Mocanu, G. Exarchakos, H.B. Ammar, A. Liotta, Reduced Reference Image Quality Assessment via Boltzmann Machines, in Proceedings of the 3rd IEEE/IFIP IM 2015 International Workshop on Quality of Experience Centric Management, Ottawa, Canada, 11–15 May 2015 (IEEE)Google Scholar
  27. 27.
    M. Torres Vega, D. Constantin Mocanu, R. Barresi, G. Fortino, A. Liotta, Cognitive Streaming on Android Devices, in Proceedings of the 1st IEEE/IFIP IM 2015 International Workshop on Cognitive Network & Service Management, Ottawa, Canada, 11–15 May, 2015 (IEEE), http://www.cogman.org
  28. 28.
    V. Menkovski, G. Exarchakos, A. Liotta, A. Cuadra Sánchez, Quality of experience models for multimedia streaming. Int. J. Mobile Comput. Multimed. Commun. 2(4), 1–20. www.igi-global.com/ijmcmc/, doi: 10.4018/jmcmc.2010100101 (ISSN: 1937-9412)
  29. 29.
    A. Liotta, D. Constantin Mocanu, V. Menkovski, L. Cagnetta, G. Exarchakos, Instantaneous video quality assessment for lightweight devices, in Proceedings of the 11th International Conference on Advances in Mobile Computing and Multimedia (ACM), Vienna, Austria, 2–4 Dec 2013, http://dx.doi.org/10.1145/2536853.2536903
  30. 30.
    V. Menkovski, G. Exarchakos, A. Liotta, Online learning for quality of experience management, in Proceedings of the annual machine learning conference of Belgium and the Netherlands, Leuven, Belgium, May 27th–28th, 2010. http://dtai.cs.kuleuven.be/events/Benelearn2010/submissions/benelearn2010_submission_20.pdf
  31. 31.
    V. Menkovski, G. Exarchakos, A. Liotta, Online QoE Prediction, in Proceedings of the 2nd IEEE International Workshop on Quality of Multimedia Experience, Trondheim, Norway, June 21–23, 2010 (IEEE)Google Scholar
  32. 32.
    V. Menkovski, G. Exarchakos, A. Cuadra-Sanchez, A. Liotta, Measuring Quality of Experience on a commercial mobile TV platform, in Proceedings of the 2nd International Conference on Advances in Multimedia, Athens, Greece, 13–19 June 2010 (IEEE)Google Scholar
  33. 33.
    V. Menkovski, A. Oredope, A. Liotta, A. Cuadra-Sanchez, Optimized online learning for QoE prediction, in Proceedings of the 21st Benelux Conference on Artificial Intelligence, Eindhoven, The Netherlands, 29–30 Oct 2009 (Open Access: http://wwwis.win.tue.nl/bnaic2009/proc.html, pp. 169–176) (ISSN: 1568-7805)
  34. 34.
    V. Menkovski, A. Oredope, A. Liotta, A. Cuadra-Sanchez, Predicting quality of experience in multimedia streaming, in Proceedings of the 7th Dec 2009 (ACM). http://dl.acm.org/citation.cfm?id=1821766 (ISBN: 978-1-60558-659-5)
  35. 35.
    D.C. Mocanu, G. Exarchakos, A. Liotta, Deep learning for objective quality assessment of 3D images, in Proceedings of IEEE International Conference on Image Processing, Paris, France, 27–30 Oct 2014 (IEEE)Google Scholar
  36. 36.
    G. Exarchakos, V. Menkovski, L. Druda, A. Liotta, Network analysis on Skype end-to-end video quality. Int. J. Pervasive Comput. Commun. 11(1), 2015. http://www.emeraldinsight.com/doi/abs/10.1108/IJPCC-08-2014-0044
  37. 37.
    F. Agboma, A. Liotta, Quality of experience management in mobile content delivery systems. J. Telecommun. Syst. (special issue on the Quality of Experience issues in Multimedia Provision). 49(1), 85–98 (2012). doi: 10.1007/s11235-010-9355-6
  38. 38.
    F. Agboma, A. Liotta, Addressing user expectations in mobile content delivery. J Mobile Inf. Syst. (special issue on Improving Quality of Service in Mobile Information Systems), 3(3), 153–164 (2007)Google Scholar
  39. 39.
    D.C. Mocanu, A. Liotta, A. Ricci, M. Torres Vega, G. Exarchakos, When does lower bitrate give higher quality in modern video services?, in Proceedings of the 2nd IEEE/IFIP International Workshop on Quality of Experience Centric Management, Krakow, Poland, 9 May 2014 (IEEE). doi:http://dx.doi.org/10.1109/NOMS.2014.6838400
  40. 40.
    A. Liotta, L. Druda, G. Exarchakos, V. Menkovski, Quality of experience management for video streams: the case of Skype. in Proceedings of the 10th International Conference on Advances in Mobile Computing and Multimedia, (ACM), Bali, Indonesia, 3–5 Dec 2012. doi:http://dx.doi.org/10.1145/2428955.2428977
  41. 41.
    G. Exarchakos, L. Druda, V. Menkovski, P. Bellavista, A. Liotta, Skype resilience to high motion videos. Int. J. Wavelets Multiresolution Inf. Proc. 11(3) (2013). doi:http://dx.doi.org/10.1142/S021969131350029X
  42. 42.
    V. Menkovski, G. Exarchakos, A. Liotta, The value of relative quality in video delivery, J. Mobile Multimed. 7(3), 151–162 (2011). ISSN: 1550-4646, http://www.rintonpress.com/xjmm7/jmm-7-3/151-162.pdf
  43. 43.
    J. Okyere-Benya, M. Aldiabat, V. Menkovski, G. Exarchakos, A. Liotta, Video quality degradation on IPTV networks, in Proceedings of International Conference on Computing, Networking and Communications, Maui, Hawaii, USA, Jan 30–Feb 2, 2012 (IEEE)Google Scholar
  44. 44.
    G. Exarchakos, V. Menkovski, A. Liotta, Can Skype be used beyond video calling?, in Proceedings of the 9th International Conference on Advances in Mobile Computing and Multimedia, Ho Chi Minh City, Vietnam, 5–7 Dec 2011 (ACM)Google Scholar
  45. 45.
    M. Alhaisoni, A. Liotta, M. Ghanbari, Scalable P2P Video Streaming. Int. J. Bus. Data Commun. Netw. 6(3), 49–65 (2010). doi: 10.4018/jbdcn.2010070103 (ISSN: 1548-0631)
  46. 46.
    M. Alhaisoni, M. Ghanbari, A. Liotta, Localized multistreams for P2P streaming. Int. J. Digit. Multimed. Broadcast. 843574, 12 (2010). doi: 10.1155/2010/843574
  47. 47.
    M. Alhaisoni, A. Liotta, M. Ghanbari, Resource-awareness and trade-off optimization in P2P video streaming. Int. J. Adv. Media Commun. (special issue on High-Quality Multimedia Streaming in P2P Environments), 4(1), 59–77 (2010). doi: 10.1504/IJAMC.2010.030005 (ISSN 1741-8003)
  48. 48.
    M. Alhaisoni, A. Liotta, Characterization of Signalling and Traffic in Joost. J P2P Netw. Appl. (special issue on Modelling and Applications of Computational P2P), 2, 75–83 (2009). doi: 10.1007/s12083-008-0015-5 (ISSN: 1936-6450)
  49. 49.
    C. Ragusa, A. Liotta, G. Pavlou, An Adaptive Clustering Approach for the Management of Dynamic Systems. IEEE J. Selected Areas Commun. (JSAC) (special issue on Autonomic Communication Systems), 23(12), 2223–2235, IEEE (IF 4.138; SJR 3.34) http://dx.doi.org/10.1109/JSAC.2005.857203
  50. 50.
    A. Liotta, The Cognitive net is coming. IEEE Spectrum, 50(8), 26–31, IEEE. doi:http://dx.doi.org/10.1109/MSPEC.2013.6565557
  51. 51.
    A. Liotta, Farewell to deterministic networks, in Proceedings of the 19th IEEE Symposium on Communications and Vehicular Technology in the Benelux, Eindhoven, the Netherlands, 16 Nov 2012 (IEEE). doi:http://dx.doi.org/10.1109/SCVT.2012.6399413
  52. 52.
    V. Menkovski, G. Exarchakos, A. Liotta, Tackling the sheer scale of subjective QoE, in Proceedings of 7th International ICST Mobile Multimedia Communications Conference, Cagliari, Italy, 5–7 Sep 2011 (Springer, Lecture Notes of ICST, vol. 29, pp. 1–15, 2012). http://www.springerlink.com/content/p1443m265r25756x/, doi: 10.1007/978-3-642-30419-4_1
  53. 53.
    V. Menkovski, G. Exarchakos, A. Liotta, Adaptive testing for video quality assessment, in Proceedings of Quality of Experience for Multimedia Content Sharing. Lisbon, Portugal, 29 June 2011 (ACM)Google Scholar
  54. 54.
    V. Menkovski, G. Exarchakos, A. Cuadra-Sanchez, A. Liotta, Estimations and remedies for quality of experience in multimedia streaming, in Proceedings of the 3rd International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services, Nice, France, Aug 22–27, 2010 (IEEE)Google Scholar
  55. 55.
    V. Menkovski, G. Exarchakos, A. Liotta, Machine learning approach for quality of experience aware networks, in Proceedings of Computational Intelligence in Networks and Systems, Thessaloniki, Greece, 24–26 Nov 2010 (IEEE)Google Scholar
  56. 56.
    F. Agboma, A. Liotta, QoE-aware QoS management, in Proceedings of the 6th International Conference on Advances in Mobile Computing and Multimedia, Linz, Austria, 24–26 Nov 2008Google Scholar
  57. 57.
    F. Agboma, M. Smy, A. Liotta, QoE analysis of a peer-to-peer television system, in Proceedings of the International Conference on Telecommunications, Networks and Systems, Amsterdam, Netherlands, 22–24 July 2008Google Scholar
  58. 58.
    F. Agboma, A. Liotta, Managing the user’s quality of experience. in Proceedings of the second IEEE/IFIP International Workshop on Business-driven IT Management (BDIM 2007), Munich, Germany, 21 May 2007 (IEEE)Google Scholar
  59. 59.
    F. Agboma, A. Liotta, User-centric assessment of mobile content delivery, in Proceedings of the 4th International Conference on Advances in Mobile Computing and Multimedia, Yogyakarta, Indonesia, 4–6 Dec 2006Google Scholar
  60. 60.
    K. Yaici, A. Liotta, H. Zisimopoulos, T. Sammut, User-centric quality of service management in UMTS. in Proceedings of the 4th Latin American Network Operations and Management Symposium (LANOMS’05), Porto Alegre, Brazil, 29–31 Aug 2005 (Springer)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Philips ResearchEindhovenThe Netherlands

Personalised recommendations