Multimedia Tools and Applications

, Volume 78, Issue 23, pp 34095–34127 | Cite as

CaR-PLive: Cloud-assisted reinforcement learning based P2P live video streaming: a hybrid approach

  • Majid Sina
  • Mehdi DehghanEmail author
  • Amir Masoud Rahmani


In recent years, live video streaming has become one of the most popular and prevalent applications of the Internet. The Peer-to-Peer (P2P) and Content Delivery Network (CDN) are popular approaches to stream video contents. These approaches respectively have faced some drastic challenges such as obtaining the desired Quality of Service (QoS) level and minimizing economic cost. The cloud computing infrastructures can reveal proper solutions to these problems. The P2P systems can eliminate their bandwidth shortage by renting resources from the cloud environment. This paper depicts CaR-PLive as a hybrid cloud-assisted P2P live streaming system. CaR-PLive uses video servers such as Amazon EC2 from cloud to stream video contents and rents Cloud Storage Services (CSSs) such as Amazon S3 to assist P2P live streaming system to reach the desired playback continuity. In CaR-PLive, we proposed two stages (sub-windows) sliding window for buffer management that a sub-window belongs to the P2P system and another one belongs to CSS. The objective of CAR-PLive is to optimize the size of sub-windows to minimize the overall rental cost of CSS restricted to a desired QoS level. We formulate this problem as an optimization problem and model it with Markov Decision Process (MDP) and then propose a reinforcement learning based algorithm to solve this problem. Finally, we evaluate the performance of CaR-PLive by performing extensive simulations and experiments with realistic settings. Simulation results demonstrate that CaR-PLive efficiently mitigates overall CSS billing cost in different system configurations and provides desired playback continuity in different system settings.


Cloud computing Peer-to-peer networks Live video streaming Resource provisioning Reinforcement learning 



The authors offer their gratitude to Dr. Amir H. Payberah at KTH Royal Institute of Technology, Stockholm, Sweden, Department of Software and Computer System for providing the simulation codes and experimental results of CLIVE.


  1. 1.
    Adhikari VK, Yang G, Fang H, Varvello M, Hilt V, Steiner M, Zhang ZL (2012) Unreeling netflix: understanding and improving multi-CDN movie delivery. 2012 proceedings IEEE INFOCOM, 25–30 march 2012: 1620–1628. doi:
  2. 2.
    Afergan MM, Leighton FT, Parikh JG (2012) Hybrid content delivery network (CDN) and peer-to-peer (P2P) network. Google PatentsGoogle Scholar
  3. 3.
    Aggarwal V, Xu C, Gopalakrishnan V, Jana R, Ramakrishnan KK, Vaishampayan VA (2011) Exploiting virtualization for delivering cloud-based IPTV services. 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS): 637–641. doi:
  4. 4.
    Ahmad S, Bouras C, Buyukkaya E, Dawood M, Hamzaoui R, Kapoulas V, Papazois A, Simon G (2018) Peer-to-peer live video streaming with rateless codes for massively multiplayer online games. Peer-to-Peer Netw Appl 11(1):44–62CrossRefGoogle Scholar
  5. 5.
    Akamai Netsession (2018) Accessed July 20 2018
  6. 6.
    Amazon CloudFront (2019) Accessed July 10 2019
  7. 7.
    Amazon simple storage service (Amazon S3) (2018) Accessed July 5 2018
  8. 8.
    Aslani R, Hakami V, Dehghan M (2018) A token-based incentive mechanism for video streaming applications in peer-to-peer networks. Multimed Tools Appl 77(12):14625–14653CrossRefGoogle Scholar
  9. 9.
    Bharambe AR, Herley C, Padmanabhan VN (2006) Analyzing and improving a BitTorrent networks performance mechanisms. In: Proceedings IEEE INFOCOM 2006. 25TH IEEE international conference on computer communications: 1–12. doi:
  10. 10.
    Bittorrent DNA (2018) Accessed July 20 2018
  11. 11.
    Budhkar S, Tamarapalli V (2018) Delay management in mesh-based P2P live streaming using a three-stage peer selection strategy. J Netw Syst Manag 26(2):401–425CrossRefGoogle Scholar
  12. 12.
    Castro M, Druschel P, Kermarrec A-M, Nandi A, Rowstron A, Singh A (2003) SplitStream: high-bandwidth multicast in cooperative environments. SIGOPS Oper Syst Rev 37(5):298–313. CrossRefGoogle Scholar
  13. 13.
    Chen Z, Yin H, Lin C, Liu X, Chen Y (2007) Towards a trustworthy and controllable peer-server-peer media streaming: an analytical study and an industrial perspective. IEEE GLOBECOM 2007 - IEEE global telecommunications conference: 2086–2090. doi:
  14. 14.
    Chih-Chiang W, Ying-Dar L (2016) CDNPatch: a cost-effective failover mechanism for hybrid CDN-P2P live streaming systems. Int J Commun Syst 29(17):2517–2533. CrossRefGoogle Scholar
  15. 15.
    Cisco V (2018) Cisco visual networking index: forecast and trends, 2017–2022. White paper 1Google Scholar
  16. 16.
    Gao G, Li R (2019) Collaborative caching in P2P streaming networks. J Netw Syst Manag 27(3):815–836CrossRefGoogle Scholar
  17. 17.
    Ghaderzadeh A, Kargahi M, Reshadi M (2018) ReDePoly: reducing delays in multi-channel P2P live streaming systems using distributed intelligence. Telecommun Syst 67(2):231–246CrossRefGoogle Scholar
  18. 18.
    Gheorghe G, Cigno RL, Montresor A (2011) Security and privacy issues in P2P streaming systems: a survey. Peer-to-Peer Netw Appl 4(2):75–91CrossRefGoogle Scholar
  19. 19.
    Gummadi KP, Saroiu S, Gribble SD (2002) King: estimating latency between arbitrary internet end hosts. Paper presented at the proceedings of the 2nd ACM SIGCOMM workshop on internet measurment, Marseille, FranceGoogle Scholar
  20. 20.
    He Y, Guan L (2009) Improving the streaming capacity in P2P VoD systems with helpers. 2009 IEEE International Conference on Multimedia and Expo: 790–793. doi:
  21. 21.
    He J, Wu D, Zeng Y, Hei X, Wen Y (2013) Toward optimal deployment of Cloud-assisted video distribution services. IEEE Trans Circ Syst Video Technol 23(10):1717–1728. CrossRefGoogle Scholar
  22. 22.
    He J, Wen Y, Huang J, Wu D (2014) On the cost–QoE tradeoff for cloud-based video streaming under amazon EC2's pricing models. IEEE Trans Circ Syst Video Technol 24(4):669–680CrossRefGoogle Scholar
  23. 23.
    Hei X, Liang C, Liang J, Liu Y, Ross KW (2007) A measurement study of a large-scale P2P IPTV system. IEEE Trans Multimed 9(8):1672–1687. CrossRefGoogle Scholar
  24. 24.
    Huang Z, Mei C, Li LE, Woo T (2011) CloudStream: Delivering high-quality streaming videos through a cloud-based SVC proxy. 2011 proceedings IEEE INFOCOM: 201–205. doi:
  25. 25.
    Ishakian V, Sweha R, Bestavros A (2017) AngelCast: Cloud-based peer-assisted live streaming using optimized multi-tree construction. Comput Commun 111:14–28CrossRefGoogle Scholar
  26. 26.
    Jin X, Kwok YK (2010) Cloud assisted P2P media streaming for bandwidth constrained Mobile subscribers. 2010 IEEE 16th international conference on parallel and distributed systems: 800–805. doi:
  27. 27.
    Jin Y, Wen Y, Shi G, Wang G, Vasilakos AV (2012) CoDaaS: an experimental cloud-centric content delivery platform for user-generated contents. 2012 International Conference on Computing, Networking and Communications (ICNC): 934–938. doi:
  28. 28.
    Li H, Zhong L, Liu J, Li B, Xu K (2011) Cost-effective partial migration of VoD services to content clouds. 2011 IEEE 4th international conference on Cloud computing: 203–210. doi:
  29. 29.
    Liao X, Jin H, Liu Y, Ni LM, Deng D (2006) AnySee: Peer-to-peer live streaming. Proceedings IEEE INFOCOM 2006. 25TH IEEE international conference on computer communications: 1–10. doi:
  30. 30.
    Lin SH, Pal R, Wang BC, Golubchik L (2017) On market-driven hybrid-P2P video streaming. IEEE Trans Multimed 19(5):984–998. CrossRefGoogle Scholar
  31. 31.
    Liu Y, Guo Y, Liang C (2008) A survey on peer-to-peer video streaming systems. Peer-to-peer Netw Appl 1(1):18–28CrossRefGoogle Scholar
  32. 32.
    Lu Z, Wu J, Chen L, Huang S, Huang Y (2010) CPH-VoD: A Novel CDN–P2P-Hybrid Architecture Based VoD Scheme. In, Berlin, Heidelberg. Web information systems engineering – WISE 2010. Springer Berlin Heidelberg: 578–586CrossRefGoogle Scholar
  33. 33.
    Lu ZH, Gao XH, Huang SJ, Huang Y (2011) Scalable and Reliable live streaming service through coordinating CDN and P2P. In: 2011 IEEE 17th international conference on parallel and distributed systems: 581–588. doi:
  34. 34.
    Magharei N, Rejaie R (2009) PRIME: peer-to-peer receiver-driven mesh-based streaming. IEEE/ACM Trans Netw 17(4):1052–1065. CrossRefGoogle Scholar
  35. 35.
    Mahini H, Dehghan M, Navidi H, Masoud Rahmani A (2016) GaMe-PLive: a new game theoretic mechanism for P2P live video streaming. Int J Commun Syst 29(6):1187–1203CrossRefGoogle Scholar
  36. 36.
    Montresor A, Abeni L (2011) Cloudy weather for P2P, with a chance of gossip. In: 2011 IEEE international conference on peer-to-peer computing: 250–259. doi:
  37. 37.
    Mostafavi S, Dehghan M (2016) Game theoretic bandwidth procurement mechanisms in live P2P streaming systems. Multimed Tools Appl 75(14):8545–8568CrossRefGoogle Scholar
  38. 38.
    Mostafavi S, Dehghan M (2017) A stochastic approximation resource allocation approach for HD live streaming. Telecommun Syst 64(1):87–101CrossRefGoogle Scholar
  39. 39.
    Niu D, Hong X, Li B, Zhao S (2012) Quality-assured cloud bandwidth auto-scaling for video-on-demand applications. In: 2012 proceedings IEEE INFOCOM: 460–468. doi:
  40. 40.
    Padmanabhan VN, Wang HJ, Chou PA (2003) Resilient peer-to-peer streaming. In: 11th IEEE international conference on network protocols, 2003. Proceedings: 16–27. doi:
  41. 41.
    Pal K, Govil MC, Ahmed M (2018) Priority-based scheduling scheme for live video streaming in peer-to-peer network. Multimed Tools Appl 77(18):24427–24457CrossRefGoogle Scholar
  42. 42.
    Pal K, Govil MC, Ahmed M (2018) Slack time–based scheduling scheme for live video streaming in P2P network. Int J Commun Syst 31(2):e3440CrossRefGoogle Scholar
  43. 43.
    Payberah AH (2013) Live streaming in P2P and hybrid P2P-cloud environments for the open internet. KTH Royal Institute of TechnologyGoogle Scholar
  44. 44.
    Payberah AH, Kavalionak H, Kumaresan V, Montresor A, Haridi S (2012) Clive: Cloud-assisted P2P live streaming. 2012 IEEE 12th Int Conf Peer-to-Peer Comput (P2P) 2012:79–90. CrossRefGoogle Scholar
  45. 45.
    Pianese F, Perino D, Keller J, Biersack EW (2007) PULSE: an adaptive, incentive-based, unstructured P2P live streaming system. IEEE Trans Multimed 9(8):1645–1660. CrossRefGoogle Scholar
  46. 46.
    PPTV (2019). Accessed July 10 2019
  47. 47.
    Qiu X, Li H, Wu C, Li Z, Lau FCM (2012) Dynamic scaling of VoD services into hybrid clouds with cost minimization and QoS guarantee. 19th Int Packet Video Workshop (PV) 2012:137–142. CrossRefGoogle Scholar
  48. 48.
    Rocha V, Kon F, Cobe R, Wassermann R (2016) A hybrid cloud-P2P architecture for multimedia information retrieval on VoD services. Computing 98(1–2):73–92MathSciNetCrossRefGoogle Scholar
  49. 49.
    Rodríguez-Silva DA, Adkinson-Orellana L, Gonz'lez-Castaño FJ, Armiño-Franco I, Gonz'lez-Martínez D (2012) Video surveillance based on Cloud storage. In: 2012 IEEE fifth international conference on Cloud computing: 991–992. doi:
  50. 50.
    Rongfei M (2019) Super node selection algorithm combining reputation and capability model in P2P streaming media network. Pers Ubiquit Comput:1–8Google Scholar
  51. 51.
    SopCast (2019). Accessed July 10 2019
  52. 52.
    Sutton RS, Barto AG (1998) Introduction to reinforcement learning. MIT pressGoogle Scholar
  53. 53.
    Tian Y, Babcock R, Taylor C, Ji Y (2018) A new live video streaming approach based on Amazon S3 pricing model. IEEE 8th Ann Comput Commun Workshop Conf (CCWC) 2018:321–328. CrossRefGoogle Scholar
  54. 54.
    Tran DA, Hua KA, Do T (2003) ZIGZAG: an efficient peer-to-peer scheme for media streaming. IEEE INFOCOM 2003. Twenty-second annual joint conference of the IEEE computer and communications societies (IEEE cat. No.03CH37428) 1282:1283–1292. CrossRefGoogle Scholar
  55. 55.
    Varga A, Hornig R (2008) An overview of the OMNeT++ simulation environment. In: Proceedings of the 1st international conference on simulation tools and techniques for communications, networks and systems & workshops, 2008. ICST (Institute for Computer Sciences, Social-Informatics and …: 60Google Scholar
  56. 56.
    Wang J, Ramchandran K (2008) Enhancing peer-to-peer live multicast quality using helpers. 2008 15th IEEE Int Conf Image Process 2008:2300–2303. CrossRefGoogle Scholar
  57. 57.
    Wang M, Xu L, Ramamurthy B (2011) Improving multi-view peer-to-peer live streaming systems with the divide-and-conquer strategy. Comput Netw 55(18):4069–4085CrossRefGoogle Scholar
  58. 58.
    Wang F, Liu J, Chen M CALMS: Cloud-assisted live media streaming for globalized demands with time/region diversities. In: 2012 Proc IEEE INFOCOM, 25–30 march 2012 2012. pp 199–207. doi:
  59. 59.
    Wang M, Xu L, Ramamurthy B (2013) Exploring the design space of multichannel peer-to-peer live video Streaming systems. IEEE/ACM Trans Netw 21(1):162–175. CrossRefGoogle Scholar
  60. 60.
    Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292. CrossRefzbMATHGoogle Scholar
  61. 61.
    Wowza Streaming Cloud (2019) Accessed July 10 2019
  62. 62.
    Wu C, Li B, Zhao S (2008) Multi-Channel Live P2P Streaming: Refocusing on Servers. IEEE INFOCOM 2008-The 27th Conference on Computer Communications. IEEE: 1355–1363Google Scholar
  63. 63.
    Wu D, Liu Y, Ross K (2009) Queuing network models for multi-channel P2P live streaming systems. In: IEEE INFOCOM 2009: 73–81. doi:10.1109/INFCOM.2009.5061908Google Scholar
  64. 64.
    Wu Y, Wu C, Li B, Qiu X, Lau FCM (2011) CloudMedia: When Cloud on Demand Meets Video on Demand. 2011 31st international conference on distributed computing systems, 20–24 June 2011 2011: 268–277. doi:
  65. 65.
    Xiao W, Bao W, Zhu X, Wang C, Chen L, Yang LT (2016) Dynamic request redirection and resource provisioning for Cloud-based video services under heterogeneous environment. IEEE Trans Parallel Distrib Syst 27(7):1954–1967. CrossRefGoogle Scholar
  66. 66.
    Xinyan Z, Jiangchuan L, Bo L, Yum YSP CoolStreaming/DONet: a data-driven overlay network for peer-to-peer live media streaming. In: Proc IEEE 24th Ann Joint Conf IEEE Comput Commun Soc., 13–17 march 2005 2005. pp 2102–2111 vol. 2103. doi:
  67. 67.
    Yin H, Liu X, Zhan T, Sekar V, Qiu F, Lin C, Zhang H, Li B (2010) LiveSky: enhancing CDN with P2P. ACM Trans Multimed Comput Commun Appl 6(3):1–19. CrossRefGoogle Scholar
  68. 68.
    Zattoo (2019) Accessed July 10 2019
  69. 69.
    Zhang X, Liu J, Li B, Yum Y-S (2005) CoolStreaming/DONet: A data-driven overlay network for peer-to-peer live media streaming. Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies, IEEE: 2102–2111Google Scholar
  70. 70.
    Zhang J, Xing W, Wang Y, Lu D (2014) Modeling and performance analysis of pull-based live streaming schemes in peer-to-peer network. Comput Commun 40:22–32CrossRefGoogle Scholar
  71. 71.
    Zhang J, Zhang X, Yang C (2018) Towards the multi-request mechanism in pull-based peer-to-peer live streaming systems. Comput Netw 138:77–89CrossRefGoogle Scholar
  72. 72.
    Zhengye L, Yanming S, Ross KW, Panwar SS, Yao W Substream trading: towards an open P2P live streaming system. In: 2008 IEEE international conference on network protocols, 19–22 Oct. 2008 2008. pp 94–103. doi:
  73. 73.
    Zhou Y, Chiu D, Lui JCS (2011) A simple model for chunk-scheduling strategies in P2P streaming. IEEE/ACM Trans Networking 19(1):42–54. CrossRefGoogle Scholar
  74. 74.
    Zhu W, Luo C, Wang J, Li S (2011) Multimedia Cloud Computing. IEEE Signal Process Mag 28(3):59–69. CrossRefGoogle Scholar
  75. 75.
    Zhu Z, Li S, Chen X (2013) Design QoS-aware multi-path provisioning strategies for efficient Cloud-assisted SVC video streaming to heterogeneous clients. IEEE Trans Multimed 15(4):758–768. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Engineering, Science and Research BranchIslamic Azad UniversityTehranIran
  2. 2.Department of Computer Engineering and Information TechnologyAmirkabir University of TechnologyTehranIran
  3. 3.Computer ScienceUniversity of Human DevelopmentSulaimanyahIraq

Personalised recommendations