Analysis of Virtualized Congestion Control in Applications Based on Hadoop MapReduce

  • Vilson Moro
  • Maurício Aronne Pillon
  • Charles Christian Miers
  • Guilherme Piêgas KoslovskiEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1171)


Among the existing applications for processing massive volumes of data, the Hadoop MapReduce (HMR) is widely used in clouds, having above all internal network flows of different volume and periodicity. In this regard, providers have the challenge of managing data centers with a wide range of operating systems and features. The diversity of algorithms and parameters related to TCP constitutes a heterogeneous communication scenario prone to degradation of communication-intensive applications. Due to total control in the data center, providers can apply the Virtualized Congestion Control (VCC) to generate optimized algorithms. From the tenant’s perspective, virtualization is a transparently performed. Some technologies have made possible to develop such virtualization. Explicit Congestion Notification (ECN) is a technique for congestion identification which acts by monitoring the queues occupancy. Although promising, the specialized literature lacks on a deep analysis of the VCC impact on the applications. Our work characterizes the VCC impact on HMR on scenarios in which there are present applications competing for network resources using optimized and non-optimized TCP stacks. We identified the HMR has its performance substantially influenced by the data volume according to the employed TCP stack. Moreover, we highlight some VCC limitations.


Virtualized Congestion Control Hadoop MapReduce TCP 


  1. 1.
    Alizadeh, M., et al.: Data center TCP (DCTCP). SIGCOMM Comput. Commun. Rev. 40(4), 63–74 (2010)CrossRefGoogle Scholar
  2. 2.
    Alizadeh, M., Javanmard, A., Prabhakar, B.: Analysis of DCTCP: stability, convergence, and fairness. In: Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. SIGMETRICS 2011, pp. 73–84. ACM (2011)Google Scholar
  3. 3.
    Chiu, D.M., Jain, R.: Analysis of the increase and decrease algorithms for congestion avoidance in computer networks. Comput. Netw. ISDN Syst. 17(1), 1–14 (1989)CrossRefGoogle Scholar
  4. 4.
    Chowdhury, M., Zaharia, M., Ma, J., Jordan, M.I., Stoica, I.: Managing data transfers in computer clusters with orchestra. SIGCOMM Comput. Commun. Rev. 41(4), 98–109 (2011)CrossRefGoogle Scholar
  5. 5.
    Cronkite-Ratcliff, B., et al.: Virtualized congestion control. In: Proceedings of the 2016 ACM SIGCOMM Conference. SIGCOMM 2016, pp. 230–243. ACM (2016)Google Scholar
  6. 6.
    Floyd, S.: TCP and explicit congestion notification. SIGCOMM Comput. Commun. Rev. 24(5), 8–23 (1994)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Ha, S., Rhee, I., Xu, L.: CUBIC: a new TCP-friendly high-speed TCP variant. SIGOPS Oper. Syst. Rev. 42(5), 64–74 (2008)CrossRefGoogle Scholar
  8. 8.
    He, K., et al.: AC/DC TCP: virtual congestion control enforcement for datacenter networks. In: Proceedings of the 2016 SIGCOMM Conference. SIGCOMM 2016, pp. 244–257. ACM (2016)Google Scholar
  9. 9.
    Jacobson, V.: Congestion avoidance and control. SIGCOMM Comput. Commun. Rev. 18(4), 314–329 (1988)CrossRefGoogle Scholar
  10. 10.
    Judd, G.: Attaining the promise and avoiding the pitfalls of TCP in the datacenter. In: Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation. NSDI 2015, pp. 145–157. Berkeley (2015)Google Scholar
  11. 11.
    Kühlewind, M., Neuner, S., Trammell, B.: On the state of ECN and TCP options on the Internet. In: Roughan, M., Chang, R. (eds.) PAM 2013. LNCS, vol. 7799, pp. 135–144. Springer, Heidelberg (2013). Scholar
  12. 12.
    Kumar, P., et al.: : PicNIC: predictable virtualized NIC. In: Proceedings of the ACM Special Interest Group on Data Communication. SIGCOMM 2019, pp. 351–366. ACM (2019)Google Scholar
  13. 13.
    Lantz, B., Heller, B., McKeown, N.: A network in a laptop: rapid prototyping for software-defined networks. In: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. HotNets-IX, pp. 19:1–19:6. ACM (2010)Google Scholar
  14. 14.
    Moro, V., Pillon, M.A., Miers, C., Koslovski, G.: Análise da virtualização do controle de congestionamento na execução de aplicações hadoop mapreduce. In: Simpósio de Sistemas Computacionais de Alto Desempenho - WSCAD, October 2018Google Scholar
  15. 15.
    Neves, M.V., De Rose, C.A.F., Katrinis, K.: MRemu: an emulation-based framework for datacenter network experimentation using realistic MapReduce traffic. In: Proceedings of the 2015 IEEE 23rd International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. MASCOTS 2015, pp. 174–177 (2015)Google Scholar
  16. 16.
    Pfaff, B., et al.: The design and implementation of Open vSwitch. In: Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation. NSDI 2015,pp. 117–130 (2015)Google Scholar
  17. 17.
    Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: FairCloud: sharing the network in cloud computing. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication. SIGCOMM 2012, pp. 187–198. ACM (2012)Google Scholar
  18. 18.
    Roy, A., Zeng, H., Bagga, J., Porter, G., Snoeren, A.C.: Inside the social network’s (datacenter) network. SIGCOMM Comput. Commun. Rev. 45(4), 123–137 (2015)CrossRefGoogle Scholar
  19. 19.
    de Souza, F.R., Miers, C.C., Fiorese, A., Koslovski, G.P.: Qos-aware virtual infrastructures allocation on SDN-based clouds. In: Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. CCGrid 2017, pp. 120–129. IEEE Press, Piscataway (2017)Google Scholar
  20. 20.
    Primet, P.V.-B., Anhalt, F., Koslovski, G.: Exploring the virtual infrastructure service concept in Grid’5000. In: 20th ITC Specialist Seminar on Network Virtualization. Hoi An, May 2009Google Scholar
  21. 21.
    Wu, H., Ju, J., Lu, G., Guo, C., Xiong, Y., Zhang, Y.: Tuning ECN for data center networks. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies. CoNEXT 2012, pp. 25–36. ACM (2012)Google Scholar
  22. 22.
    Zahavi, E., Shpiner, A., Rottenstreich, O., Kolodny, A., Keslassy, I.: Links as a service (LaaS): guaranteed tenant isolation in the shared cloud. In: Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems. ANCS 2016, pp. 87–98. ACM (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Graduate Program in Applied ComputingSanta Catarina State UniversityFlorianópolisBrazil

Personalised recommendations