Abstract
At present, HTTP Adaptive Streaming (HAS) is developing into a key technology for video delivery over the Internet. In this delivery strategy, the client proactively and adaptively requests a quality version of chunked video segments based on its playback buffer, the perceived network bandwidth and other relevant factors. In this paper, we discuss the use of reinforcement-learning (RL) to learn the optimal request strategy at the HAS client by progressively maximizing a pre-defined Quality of Experience (QoE)-related reward function. Under the framework of RL, we investigate the most influential factors for the request strategy, using a forward variable selection algorithm. The performance of the RL-based HAS client is evaluated by a Video-on-Demand (VOD) simulation system. Results show that given the QoE-related reward function, the RL-based HAS client is able to optimize the quantitative QoE. Comparing with a conventional HAS system, the RL-based HAS client is more robust and flexible under versatile network conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Microsoft, Smooth streaming (2008), http://www.iis.net/downloads/microsoft/smooth-streaming (accessed July 2013)
Pantos, R., May, W.: HTTP live streaming overview (2012), http://tools.ietf.org/html/draft-pantos-http-live-streaming-10 (accessed July 2013)
Adobe, HTTP dynamic streaming: Flexible delivery of on-demand and live video streaming (2010), http://www.adobe.com/products/hds-dynamic-streaming.html (accessed July 2013)
Huang, T.-Y., Handigol, N., Heller, B., McKeown, N., Johari, R.: Confused, timid, and unstable: Picking a video streaming rate is hard. In: ACM Internet Measurement Conference, pp. 225–238 (November 2012)
Akhshabi, S., Anantakrishnan, L., Dovrolis, C., Begen, A.: What happens when HTTP adaptive streaming players compete for bandwidth? In: ACM NOSSDAV, pp. 9–14 (June 2012)
Liu, C., Bouazizi, I., Gabbouj, M.: Rate adaptation for adaptive http streaming. In: ACM MMSys, pp. 169–174 (2011)
De Cicco, L., Mascolo, S., Palmisano, V.: Feedback control for adaptive live video streaming. In: ACM MMSys, pp. 145–156 (February 2011)
Claeys, M., Latré, S., Famaey, J., Wu, T., Van Leekwijck, W., De Turck, F.: Design of a Q-learning-based client quality selection algorithm for HTTP adaptive video streaming. In: Proc. Conference on Autonomous Agents and Multiagent Systems (May 2013)
Huang, T.-Y., Johari, R., McKeown, N.: Downton abbey without the hiccups: Buffer-based rate adaptation for http video streaming. In: ACM FhMN (to appear, August 2013)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (1998)
Mok, R., Chan, E., Chang, R.: Measuring the quality of experience of HTTP video streaming. In: Proc. IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 485–492 (May 2011)
Balachandran, A., Sekar, V., Akella, A., Seshan, S., Stoica, I., Zhang, H.: Developing a predictive model of quality of experience for internet video. In: ACM SIGCOMM (to appear, August 2013)
Vriendt, J.D., Vleeschauwer, D.D., Robinson, D.: Model for estimating qoe of video delivered using HTTP adaptive streaming. In: Proc. IFIP/IEEE Workshop on QoE CENTRIC Management (May 2013)
Hossfeld, T., Egger, S., Schatz, R., Fiedler, M., Masuch, K., Lorentzen, C.: Initial delay vs. interruptions: Between the devil and the deep blue sea. In: 2012 Fourth International Workshop on Quality of Multimedia Experience (QoMEX), pp. 1–6 (July 2012)
Click modular router (2010), http://read.cs.ucla.edu/click/click (accessed July 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wu, T., Van Leekwijck, W. (2014). Factor Selection for Reinforcement Learning in HTTP Adaptive Streaming. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds) MultiMedia Modeling. MMM 2014. Lecture Notes in Computer Science, vol 8325. Springer, Cham. https://doi.org/10.1007/978-3-319-04114-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-04114-8_47
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04113-1
Online ISBN: 978-3-319-04114-8
eBook Packages: Computer ScienceComputer Science (R0)