Unleashing the Potential of Data-Driven Networking

  • Junchen JiangEmail author
  • Vyas Sekar
  • Ion Stoica
  • Hui Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10340)


The last few years have witnessed the coming of age of data-driven paradigm in various aspects of computing (partly) empowered by advances in distributed system research (cloud computing, MapReduce, etc.). In this paper, we observe that the benefits can flow the opposite direction: the design and management of networked systems can be improved by data-driven paradigm. To this end, we present DDN, a new design framework for network protocols based on data-driven paradigm. We argue that DDN has the potential to significantly achieve better performance through harnessing more data than one single flow. Furthermore, we systematize existing instantiations of DDN by creating a unified framework for DDN, and use the framework to shed light on the common challenges and reusable design principles. We believe that by systematizing this paradigm as a broader community, we can unleash the unharnessed potential of DDN.



This research is supported in part by NSF award CNS-1345305 and NSF CISE Expeditions Award CCF-1139158, DOE Award SN10040 DE-SC0012463, and DARPA XData Award FA8750-12-2-0331, and gifts from Amazon Web Services, Google, IBM, SAP, The Thomas and Stacey Siebel Foundation, Adatao, Adobe, Apple Inc., Blue Goji, Bosch, Cisco, Cray, Cloudera, Ericsson, Facebook, Fujitsu, Guavus, HP, Huawei, Intel, Microsoft, Pivotal, Samsung, Schlumberger, Splunk, State Farm, Virdata and VMware. Junchen Jiang was supported in part by Juniper Networks Fellowship.


  1. 1.
    ACM SIGCOMM Workshop on QoE-Based Analysis and Management of Data Communication Networks (Internet-QoE 2016).
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
    Technical note on the CFA algorithm.
  8. 8.
    The Data-Driven Approach to Network Management: Innovation Delivered.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
    Abdallah, C.T., Byrne, R., Benites-Read, J., Dorato, P.: Delayed positive feedback can stabilize oscillatory systems. In: Proceedings of ACC (American control conference) (1993)Google Scholar
  13. 13.
    Agarwal, A., Bird, S., Cozowicz, M., Hoang, L., Langford, J., Lee, S., Li, J., Melamed, D., Oshri, G., Ribas, O., et al.: A multiworld testing decision service. arXiv preprint arXiv:1606.03966 (2016)
  14. 14.
    Chen, F., Zhang, C., Wang, F., Liu, J.: Crowdsourced live streaming over the cloud. In: INFOCOM (2015)Google Scholar
  15. 15.
    Clark, D.D., Partridge, C., Ramming, J.C., Wroclawski, J.T.: A knowledge plane for the internet. In: ACM SIGCOMM 2003Google Scholar
  16. 16.
    Crankshaw, D., Bailis, P., Gonzalez, J.E., Li, H., Zhang, Z., Franklin, M.J., Ghodsi, A., Jordan, M.I.: The missing piece in complex analytics: low latency, scalable model management and serving with velox. In: Conference on Innovative Data Systems Research (CIDR) (2015)Google Scholar
  17. 17.
    Datta, A., Sen, S., Zick, Y.: Algorithmic transparency via quantitative input influence. In: Proceedings of 37th IEEE Symposium on Security and Privacy (2016)Google Scholar
  18. 18.
    Dong, M., Li, Q., Zarchy, D., Godfrey, P.B., Schapira, M.: PCC: re-architecting congestion control for consistent high performance. In: Proceedings of NSDI (2015)Google Scholar
  19. 19.
    Dudík, M., Langford, J., Li, L.: Doubly robust policy evaluation and learning. In: Proceedings of International Conference on Machine Learning (2011)Google Scholar
  20. 20.
    Dukkipati, N., Refice, T., Cheng, Y., Chu, J., Herbert, T., Agarwal, A., Jain, A., Sutin, N.: An argument for increasing TCP’s initial congestion window. ACM SIGCOMM CCR 40, 27–33 (2010)CrossRefGoogle Scholar
  21. 21.
    Floyd, S., Jacobson, V.: Random early detection gateways for congestion avoidance. IEEE/ACM Trans. Netw. 1(4), 397–413 (1993)CrossRefGoogle Scholar
  22. 22.
    Floyd, S., Paxson, V.: Difficulties in simulating the internet. IEEE/ACM Trans. Netw. (ToN) 9(4), 392–403 (2001)CrossRefGoogle Scholar
  23. 23.
    Ganjam, A., Sekar, V., Zhang, H.: In-situ quality of experience monitoring: the case for prioritizing coverage over fidelityGoogle Scholar
  24. 24.
    Ganjam, A., Siddiqi, F., Zhan, J., Stoica, I., Jiang, J., Sekar, V., Zhang, H.: C3: internet-scale control plane for video quality optimization. In: NSDI. USENIX (2015)Google Scholar
  25. 25.
    Halevy, A., Norvig, P., Pereira, F.: The unreasonable effectiveness of data. IEEE Intell. Syst. 24(2), 8–12 (2009)CrossRefGoogle Scholar
  26. 26.
    Haq, O., Dogar, F.R.: Leveraging the power of cloud for reliable wide area communication. In: ACM Workshop on Hot Topics in Networks (2015)Google Scholar
  27. 27.
    Huang, T.-Y., Handigol, N., Heller, B., McKeown, N., Johari, R.: Confused, timid, and unstable: picking a video streaming rate is hard. In: Proceedings of SIGCOMM IMC (2012)Google Scholar
  28. 28.
    Jacobson, V.: Congestion avoidance and control. ACM SIGCOMM Comput. Commun. Rev. 18, 314–329 (1988). ACMCrossRefGoogle Scholar
  29. 29.
    Jacobson, V., Smetters, D.K., Thornton, J.D., Plass, M.F., Briggs, N.H., Braynard, R.L.: Networking named content. In: Proceedings of CoNext (2009)Google Scholar
  30. 30.
    Jiang, J., Das, R., Anathanarayanan, G., Chou, P., Padmanabhan, V., Sekar, V., Dominique, E., Goliszewski, M., Kukoleca, D., Vafin, R., Zhang, H.: VIA: improving internet telephony call quality using predictive relay selection. To Appear in Proceedings of SIGCOMM (2016)Google Scholar
  31. 31.
    Jiang, J., Liu, X., Sekar, V., Stoica, I., Zhang, H.: EONA: Experience-Oriented Network Architecture. In: ACM HotNets (2014)Google Scholar
  32. 32.
    Jiang, J., Sekar, V., Milner, H., Shepherd, D., Stoica, I., Zhang, H.: CFA: a practical prediction system for video QoE optimization. In Proceedings of NSDI (2016)Google Scholar
  33. 33.
    Jiang, J., Sekar, V., Zhang, H.: Improving fairness, efficiency, and stability in HTTP-based adaptive streaming with festive. In: ACM CoNEXT 2012Google Scholar
  34. 34.
    Kandoi, R., Antikainen, M.: Denial-of-service attacks in OpenFlow SDN networks. In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 1322-1326. IEEE (2015)Google Scholar
  35. 35.
    Krishnan, S., Sitaraman, R.: Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs (2012)Google Scholar
  36. 36.
    Krishnan, S.S., Sitaraman, R.K.: Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. IEEE/ACM Trans. Netw. 21(6), 2001–2014 (2013)CrossRefGoogle Scholar
  37. 37.
    Kumar, A., Jain, S., Naik, U., Raghuraman, A., Kasinadhuni, N., Zermeno, E.C., Gunn, C.S., Ai, J., Carlin, B., Amarandei-Stavila, M., et al.: BwE: flexible, hierarchical bandwidth allocation for WAN distributed computing. In: Proceedings of SIGCOMM (2015)Google Scholar
  38. 38.
    Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp. 661–670. ACM (2010)Google Scholar
  39. 39.
    Liu, H.H., Viswanathan, R., Calder, M., Akella, A., Mahajan, R., Padhye, J., Zhang, M.: Efficiently delivering online services over integrated infrastructure. In: Proceedings of NSDI (2016)Google Scholar
  40. 40.
    Liu, X., Dobrian, F., Milner, H., Jiang, J., Sekar, V., Stoica, I., Zhang, H.: A case for a coordinated internet video control plane. In: ACM SIGCOMM, pp. 359–370. ACM (2012)Google Scholar
  41. 41.
    Lu, T., Pál, D., Pál, M.: Contextual multi-armed bandits. In: AISTATS, pp. 485–492 (2010)Google Scholar
  42. 42.
    Madhyastha, H.V., Isdal, T., Piatek, M., Dixon, C., Anderson, T., Krishnamurthy, A., Venkataramani, A.: iPlane: an information plane for distributed services. In: USENIX OSDI 2006Google Scholar
  43. 43.
    Pelsser, C., Cittadini, L., Vissicchio, S., Bush, R.: From Paris to Tokyo: on the suitability of ping to measure latency. In: Proceedings of the 2013 Conference on Internet Measurement Conference, pp. 427–432. ACM (2013)Google Scholar
  44. 44.
    Precup, D., Sutton, R.S., Singh, S.: Eligibility traces for off-policy policy evaluation. In: Proceedings of the Seventeenth International Conference on Machine Learning (2000)Google Scholar
  45. 45.
    Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., Stoica, I.: Low latency geo-distributed data analytics. In: Proceedings of SIGCOMM (2015)Google Scholar
  46. 46.
    Rabkin, A., Arye, M., Sen, S., Pai, V.S., Freedman, M.J.: Aggregation and degradation in JetStream: streaming analytics in the wide area. In: Proceedings of NSDI (2014)Google Scholar
  47. 47.
    Rexford, J., Wang, J., Xiao, Z., Zhang, Y.: BGP routing stability of popular destinations. In: Proceedings of SIGCOMM IMW (2002)Google Scholar
  48. 48.
    Richard, J.-P.: Time-delay systems: an overview of some recent advances and open problems. Automatica 39(10), 1667–1694 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Rigollet, P., Zeevi, A.: Nonparametric bandits with covariates. In: Proceedings of the Conference on Learning Theory (2010)Google Scholar
  50. 50.
    Saltzer, J.H., Reed, D.P., Clark, D.D.: End-to-end arguments in system design. ACM Trans. Comput. Syst. (TOCS) 2(4), 277–288 (1984)CrossRefGoogle Scholar
  51. 51.
    Seshan, S., Stemm, M., Katz, R.H.: SPAND: shared passive network performance discovery. In: USENIX Symposium on Internet Technologies and Systems, pp. 1–13 (1997)Google Scholar
  52. 52.
    Shalita, A., Karrer, B., Kabiljo, I., Sharma, A., Presta, A., Adcock, A., Kllapi, H., Stumm, M.: Social hash: an assignment framework for optimizing distributed systems operations on social networks. In: Proceedings of NSDI (2016)Google Scholar
  53. 53.
    Slivkins, A.: Contextual bandits with similarity information. J. Mach. Learn. Res. 15(1), 2533–2568 (2014)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Sun, Y., Yin, X., Jiang, J., Sekar, V., Lin, F., Wang, N., Liu, T., Sinopoli, B.: CS2P: improving video bitrate selection and adaptation with data-driven throughput prediction. To Appear in Proceedings of SIGCOMM (2016)Google Scholar
  55. 55.
    Vellido, A., Martin-Guerroro, J., Lisboa, P.: Making machine learning models interpretable. In: Proceedings of the 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium, pp. 163–172 (2012)Google Scholar
  56. 56.
    Venkataraman, S., Yang, Z., Franklin, M., Recht, B., Stoica, I.: Ernest: efficient performance prediction for large-scale advanced analytics. In: Proceedings of NSDI (2016)Google Scholar
  57. 57.
    Winstein, K., Balakrishnan, H.: TCP ex Machina: computer-generated congestion control. In: Proceedings of SIGCOMM (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Junchen Jiang
    • 1
    Email author
  • Vyas Sekar
    • 1
  • Ion Stoica
    • 2
    • 3
    • 4
  • Hui Zhang
    • 1
    • 3
  1. 1.CMUPittsburghUSA
  2. 2.UC BerkeleyBerkeleyUSA
  3. 3.ConvivaNew York CityUSA
  4. 4.DatabricksSan FranciscoUSA

Personalised recommendations