Skip to main content

Gossip-Based Networking for Internet-Scale Distributed Systems

  • Conference paper

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 78))

Abstract

In the era of Internet-scale applications, an increasing number of services are distributed over pools of thousands to millions of networked computers. Along with the obvious advantages in performance and capacity, such a massive scale comes also with challenges. Continuous changes in the system become the norm rather than the exception, either because of inevitable hardware failures or merely due to standard maintenance and upgrading procedures. Rather than trying to impose rigid control on the massive pools of resources, we should equip Internet-scale applications with enough flexibility to work around inevitable faults. In that front, gossiping protocols have emerged as a promising component due to their highly desirable properties: self-healing, self-organizing, symmetric, immensely scalable, and simple.

Through visiting a representative set of fundamental gossiping protocols, this paper provides insight on the principles that govern their behavior. By focusing on the rationale and incentives behind gossiping protocols, we introduce the reader to the alternative way of managing massive scale systems through gossiping, and we intrigue her or his interest to delve deeper into the subject by providing an extensive list of pointers.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Maymounkov, P., Mazières, D.: Kademlia: A peer-to-peer information system based on the XOR metric. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, pp. 53–65. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  2. Loo, B.T., Huebsch, R., Hellerstein, J.M., Shenker, S., Stoica, I.: Enhancing p2p file-sharing with an internet-scale query processor. In: Proc. 30th International Conference on Very Large Data Bases, VLDB (August 2004)

    Google Scholar 

  3. Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N., Shenker, S.: Making gnutella-like p2p systems scalable. In: SIGCOMM 2003: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 407–418. ACM Press, New York (2003)

    Chapter  Google Scholar 

  4. Felber, P., Kropf, P., Leonini, L., Luu, T., Rajman, M., Rivière, E.: Collaborative ranking and profiling: Exploiting the wisdom of crowds in tailored web search. In: Eliassen, F., Kapitza, R. (eds.) DAIS 2010. LNCS, vol. 6115, pp. 226–242. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Castro, M., Druschel, P., Kermarrec, A.M., Nandi, A., Rowstron, A., Singh, A.: Splitstream: high-bandwidth multicast in cooperative environments. In: SOSP 2003: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 298–313. ACM Press, New York (2003)

    Chapter  Google Scholar 

  6. Zhang, C., Jin, H., Deng, D., Yang, S., Yuan, Q., Yin, Z.: Anysee: Multicast-based peer-to-peer media streaming service system. In: Proceedings of the Asia-Pacific Conference on Communications(APCC05), Pernth, Western Australia (October 2005)

    Google Scholar 

  7. Rowstron, A., Kermarrec, A.M., Castro, M., Druschel, P.: SCRIBE: The design of a large-scale event notification infrastructure. In: Crowcroft, J., Hofmann, M. (eds.) NGC 2001. LNCS, vol. 2233, pp. 30–43. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  8. Voulgaris, S., Rivière, E., Kermarrec, A.M., van Steen, M.: Sub-2-sub: Self-organizing content-based publish subscribe for dynamic large scale collaborative networks. In: Proceedings of IPTPS 2006: 5th International Workshop on Peer-to-Peer Systems, Santa Barbara, USA (February 2006)

    Google Scholar 

  9. Bhagwan, R., Savage, S., Voelker, G.M.: Understanding availability. In: Kaashoek, M.F., Stoica, I. (eds.) IPTPS 2003. LNCS, vol. 2735, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  10. Stutzbach, D., Rejaie, R.: Understanding churn in peer-to-peer networks. In: IMC 2006: Proceedings of the 6th ACM SIGCOMM on Internet Measurement, pp. 189–202. ACM Press, New York (2006)

    Google Scholar 

  11. Kermarrec, A.M., van Steen, M.: Gossiping in distributed systems. SIGOPS Oper. Syst. Rev. 41(5), 2–7 (2007)

    Article  Google Scholar 

  12. Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D., Terry, D.: Epidemic algorithms for replicated database maintenance. In: PODC 1887: Proceedings of the Sixth Annual ACM Symposium on Principles of Distributed Computing, pp. 1–12. ACM Press, New York (1987)

    Chapter  Google Scholar 

  13. Birman, K.P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., Minsky, Y.: Bimodal multicast. ACM Transactions on Computer Systems 17(2), 41–88 (1999)

    Article  Google Scholar 

  14. Eugster, P.T., Guerraoui, R., Kermarrec, A.M., Massoulié, L.: Epidemic information dissemination in distributed systems. Computer 37, 60–67 (2004)

    Article  Google Scholar 

  15. Gupta, I., Kermarrec, A.M., Ganesh, A.: Efficient epidemic-style protocols for reliable and scalable multicast. In: Proceedings of the 21st Symposium on Reliable Distributed Systems (SRDS 2002), p. 180 (2002)

    Google Scholar 

  16. Khambatti, M., Ryu, K., Dasgupta, P.: Push-pull gossiping for information sharing in peer-to-peer communities. In: Proceedings of International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), pp. 1393–1399 (June 2003)

    Google Scholar 

  17. Eugster, P.T., Guerraoui, R., Handurukande, S.B., Kouznetsov, P., Kermarrec, A.M.: Lightweight probabilistic broadcast. ACM Transactions on Computer Systems 21(4), 341–374 (2003)

    Article  Google Scholar 

  18. Vogels, W., van Renesse, R., Birman, K.: The power of epidemics: robust communication for large-scale distributed systems. SIGCOMM Comput. Commun. Rev. 33(1), 131–135 (2003)

    Article  Google Scholar 

  19. Kermarrec, A.M., Massoulié, L., Ganesh, A.J.: Probabilistic reliable dissemination in large-scale systems. IEEE Transactions on Parallel and Distributed Systems 14(3), 248–258 (2003)

    Article  Google Scholar 

  20. Renesse, R.V., Birman, K.P., Vogels, W.: Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining. ACM Trans. Comput. Syst. 21(2), 164–206 (2003)

    Article  Google Scholar 

  21. Locher, T., Meier, R., Schmid, S., Wattenhofer, R.: Push-to-pull peer-to-peer live streaming. In: Pelc, A. (ed.) DISC 2007. LNCS, vol. 4731, pp. 388–402. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  22. Rhea, S.C.: Opendht: a public dht service. PhD thesis, Berkeley, CA, USA, AAI3211499 (2005)

    Google Scholar 

  23. Patel, J.A., Gupta, I., Contractor, N.: Jetstream: Achieving predictable gossip dissemination by leveraging social network principles. In: NCA 2006: Proceedings of the Fifth IEEE International Symposium on Network Computing and Applications, pp. 32–39. IEEE Computer Society, Washington, DC, USA (2006)

    Google Scholar 

  24. Serbu, S., Rivière, E., Felber, P.: Network-friendly gossiping. In: Guerraoui, R., Petit, F. (eds.) SSS 2009. LNCS, vol. 5873, pp. 655–669. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  25. Fernandess, Y., Fernández, A., Monod, M.: A generic theoretical framework for modeling gossip-based algorithms. SIGOPS Oper. Syst. Rev. 41(5), 19–27 (2007)

    Article  Google Scholar 

  26. Allavena, A., Demers, A., Hopcroft, J.E.: Correctness of a gossip based membership protocol. In: PODC 2005: Proceedings of the Twenty-Fourth Annual ACM Symposium on Principles of Distributed Computing, pp. 292–301. ACM Press, New York (2005)

    Google Scholar 

  27. DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. SIGOPS Oper. Syst. Rev. 41(6), 205–220 (2007)

    Article  Google Scholar 

  28. Jelasity, M., Voulgaris, S., Guerraoui, R., Kermarrec, A.M., van Steen, M.: Gossip-based peer sampling. ACM Transactions on Computer Systems 25(3) (August 2007)

    Google Scholar 

  29. Jelasity, M., Kowalczyk, W., van Steen, M.: Newscast Computing. Technical Report IR-CS-006, Vrije Universiteit Amsterdam, Department of Computer Science, Amsterdam, The Netherlands (November 2003)

    Google Scholar 

  30. Voulgaris, S., Jelasity, M., van Steen, M.: A robust and scalable peer-to-peer gossiping protocol. In: Moro, G., Sartori, C., Singh, M.P. (eds.) AP2PC 2003. LNCS (LNAI), vol. 2872, pp. 47–58. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  31. Voulgaris, S., Gavidia, D., van Steen, M.: Cyclon: Inexpensive membership management for unstructured p2p overlays. Journal of Network and Systems Management 13(2) (June 2005)

    Google Scholar 

  32. Erdös, P., Rényi, A.: On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences 5, 17–61 (1960)

    Google Scholar 

  33. Barabasi, A.L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)

    Article  Google Scholar 

  34. Barabasi, A.L., Albert, R.: Statistical mechanics of complex networks. Reviews of Modern Physics (2002)

    Google Scholar 

  35. Barabasi, A.L.: LINKED: The New Science of Networks. Perseus Books Group (2002)

    Google Scholar 

  36. Ganesh, A.J., Kermarrec, A.M., Massoulié, L.: SCAMP: Peer-to-peer lightweight membership service for large-scale group communication. In: Crowcroft, J., Hofmann, M. (eds.) NGC 2001. LNCS, vol. 2233, pp. 44–55. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  37. Leitão, J., Pereira, J., Rodrigues, L.: Hyparview: a membership protocol for reliable gossip-based broadcast. In: Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Edinburgh, UK, pp. 419–428 (June 2007)

    Google Scholar 

  38. Jelasity, M., Babaoglu, O.: T-man: Gossip-based overlay topology management. Engineering Self-Organising Systems 1(15) (2005)

    Google Scholar 

  39. Jelasity, M., Montresor, A., Babaoglu, O.: T-man: Gossip-based fast overlay topology construction. Computer Networks 53(13), 2321–2339 (2009)

    Article  Google Scholar 

  40. Voulgaris, S., van Steen, M.: Epidemic-style management of semantic overlays for content-based searching. In: Cunha, J.C., Medeiros, P.D. (eds.) Euro-Par 2005. LNCS, vol. 3648, pp. 1143–1152. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  41. Voulgaris, S.: Epidemic-Based Self-Organization in Peer-to-Peer Systems. PhD thesis, Vrije Universiteit Amsterdam (2006)

    Google Scholar 

  42. Lua, E.K., Crowcroft, J., Pias, M., Sharma, R., Lim, S.: A survey and comparison of peer-to-peer overlay network schemes. IEEE Communications Survey and Tutorial (March 2004)

    Google Scholar 

  43. Kleinberg, J.: The small-world phenomenon: An algorithmic perspective. In: Proceedings of the 32nd ACM Symposium on Theory of Computing, Portland, OR, USA, pp. 163–170 (May 2000)

    Google Scholar 

  44. Watts, D.J., Strogatz, S.H.: Collective dynamics of ’small-world’ networks. Nature 393, 440–442 (1998)

    Article  Google Scholar 

  45. eDonkey (no date), http://www.edonkey2000.com

  46. Guerraoui, R., Handurukande, S.B., Huguenin, K., Kermarrec, A.M., Fessant, F.L., Rivière, E.: GosSkip, an efficient, fault-tolerant and self organizing overlay using gossip-based construction and skip-lists principles. In: P2P 2006: Proceedings of the Sixth IEEE International Conference on Peer-to-Peer Computing, pp. 12–22. IEEE Computer Society, Cambridge (2006)

    Google Scholar 

  47. Pugh, W.: Skip lists: A probabilistic alternative to balanced trees. Communication of the ACM 32(10), 668–676 (1990)

    Article  Google Scholar 

  48. Harvey, N.J.A., Jones, M.B., Saroiu, S., Theimer, M., Wolman, A.: Skipnet: A scalable overlay network with practical locality properties. In: The Fourth USENIX Symposium on Internet Technologies and Systems (USITS 2003), Seattle, WA (2003)

    Google Scholar 

  49. Beaumont, O., Kermarrec, A.M., Rivière, E.: Peer to peer multidimensional overlays: Approximating complex structures. In: Tovar, E., Tsigas, P., Fouchal, H. (eds.) OPODIS 2007. LNCS, vol. 4878, pp. 315–328. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  50. Bonnet, F., Kermarrec, A.M., Raynal, M.: Small-world networks: From theoretical bounds to practical systems. In: Tovar, E., Tsigas, P., Fouchal, H. (eds.) OPODIS 2007. LNCS, vol. 4878, pp. 372–385. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  51. Milgram, S.: The small world problem. Psychology Today 2, 60–67 (1967)

    Google Scholar 

  52. Patel, J.A., Rivière, E., Gupta, I., Kermarrec, A.M.: Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems. Computer Networks 53(13), 2304–2320 (2009)

    Article  Google Scholar 

  53. Jelasity, M., Kermarrec, A.M.: Ordered slicing of very large-scale overlay networks. In: P2P 2006: Proceedings of the Sixth IEEE International Conference on Peer-to-Peer Computing, pp. 117–124. IEEE Computer Society, Cambridge (September 2006)

    Google Scholar 

  54. Fernandez, A., Gramoli, V., Jimenez, E., Kermarrec, A.M., Raynal, M.: Distributed slicing in dynamic systems. In: Proceedings of the International Conference on Distributed Computing Systems (ICDCS 2007), IEEE Computer Society Press, Toronto (June 2007)

    Google Scholar 

  55. Gramoli, V., Vigfusson, Y., Birman, K., Kermarrec, A.M., van Renesse, R.: Slicing distributed systems. IEEE Transactions on Computers – Special Issue on Autonomic Network Computing (IEEE TC) 58(11), 1444–1455 (2009)

    Google Scholar 

  56. Jelasity, M., Montresor, A., Babaoglu, O.: Gossip-based aggregation in large dynamic networks. ACM Trans. Comp. Syst. 23(3), 219–252 (2005)

    Article  Google Scholar 

  57. Jelasity, M., Montresor, A.: Epidemic-Style Proactive Aggregation in Large Overlay Networks. In: 24th Int’l Conf. on Distributed Computing Systems, pp. 102–109 (2004)

    Google Scholar 

  58. Montresor, A., Jelasity, M., Babaoglu, O.: Robust aggregation protocols for large-scale overlay networks. In: DSN 2004: Proceedings of the 2004 International Conference on Dependable Systems and Networks (DSN 2004), p. 19. IEEE Computer Society, Washington, DC, USA (2004)

    Google Scholar 

  59. Kempe, D., Dobra, A., Gehrke, J.: Gossip-based computation of aggregate information. In: FOCS 2003: Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science, p. 482. IEEE Computer Society, Washington, DC, USA (2003)

    Chapter  Google Scholar 

  60. Kowalczyk, W., Vlassis, N.: Newscast EM. In: Advances in Neural Information Processing Systems (NIPS), vol. 17, MIT Press, Cambridge (2004)

    Google Scholar 

  61. Voulgaris, S., van Steen, M.: An epidemic protocol for managing routing tables in very large peer-to-peer networks. In: Brunner, M., Keller, A. (eds.) DSOM 2003. LNCS, vol. 2867, pp. 41–54. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  62. Kostoulas, D., Psaltoulis, D., Gupta, I., Birman, K., Demers, A.: Decentralized schemes for size estimation in large and dynamic groups. In: NCA 2005: Proceedings of the Fourth IEEE International Symposium on Network Computing and Applications, pp. 41–48. IEEE Computer Society, Washington, DC, USA (2005)

    Google Scholar 

  63. Massoulié, L., Merrer, E.L., Kermarrec, A.M., Ganesh, A.: Peer counting and sampling in overlay networks: random walk methods. In: PODC 2006: Proceedings of the Twenty-Fifth Annual ACM Symposium on Principles of Distributed Computing, pp. 123–132. ACM Press, New York (2006)

    Chapter  Google Scholar 

  64. Merrer, E.L., Kermarrec, A.M., Massoulié, L.: Peer to peer size estimation in large and dynamic networks: A comparative study. In: Proceedings of the 15th IEEE International Symposium on High Performance Distributed Computing, Paris, France, pp. 7–17 (June 2006)

    Google Scholar 

  65. Baldoni, R., Beraldi, R., Quema, V., Querzoni, L., Tucci-Piergiovanni, S.: Tera: topic-based event routing for peer-to-peer architectures. In: DEBS 2007: Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems, pp. 2–13. ACM Press, New York (June 2007)

    Chapter  Google Scholar 

  66. Matos, M., Nunes, A., Oliveira, R., Pereira, J.: Stan: Exploiting shared interests without disclosing them in gossip-based publish/subscribe. In: Proc. of IPTPS 2010: 9th International Workshop on Peer-to-Peer Systems, San Jose, CA, USA (2010)

    Google Scholar 

  67. Anceaume, E., Gradinariu, M., Datta, A.K., Simon, G., Virgillito, A.: A semantic overlay for self-* peer-to-peer publish/subscribe. In: Proceedings of the International Conference on Distributed Computing Systems, ICDCS 2006 (June 2006)

    Google Scholar 

  68. Kermarrec, A.M., Pace, A., Quema, V., Schiavoni, V.: Nat-resilient gossip peer sampling. In: Proceedings of the 2009 29th IEEE International Conference on Distributed Computing Systems. ICDCS 2009, pp. 360–367. IEEE Computer Society, Washington, DC, USA (2009)

    Chapter  Google Scholar 

  69. Leitão, J., van Renesse, R., Rodrigues, L.: Balancing gossip exchanges in networks with firewalls. In: Proceedings of the 9th International Conference on Peer-to-Peer Systems. IPTPS 2010, p. 7. USENIX Association, Berkeley (2010)

    Google Scholar 

  70. Vigfusson, Y., Abu-Libdeh, H., Balakrishnan, M., Birman, K., Burgess, R., Li, H., Chockler, G., Tock, Y.: Dr. multicast: Rx for data center communication scalability. In: Proceedings of Eurosys 2010, Paris, France (April 2010)

    Google Scholar 

  71. Bortnikov, E., Gurevich, M., Keidar, I., Kliot, G., Shraer, A.: Brahms: Byzantine resilient random membership sampling. Computer Networks 53(13), 2340–2359 (2009)

    Article  Google Scholar 

  72. Jesi, G.P., Montresor, A., van Steen, M.: Secure Peer Sampling. Elsevier Computer Networks - Special Issue on Collaborative Peer-to-Peer Systems 54(12), 2086–2098 (2010)

    Google Scholar 

  73. Bakker, A., van Steen, M.: Puppetcast: A secure peer sampling protocol. In: Proc. of the European Conference on Computer Network Defense (EC2ND 2008), Dublin, Ireland, pp. 3–10 (December 2008)

    Google Scholar 

  74. Li, H., Clement, A., Wong, E., Napper, J., Alvisi, L., Dahlin, M.: Bar gossip. In: Proc. of 7th Symposium on Operating System Design and Implementation, OSDI 2006 (2006)

    Google Scholar 

  75. Kihlstrom, K.P., Elliott, R.S.: Performance of an intrusion-tolerant gossip protocol. In: Proc. of the 21st IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS), Cambridge, MA, USA (November 2009)

    Google Scholar 

  76. Schiavoni, V., Rivière, E., Felber, P.: Whisper: Middleware for confidential communication in large-scale networks. In: Proc. of ICDCS 2011: 31st Int’l Conference on Distributed Computing Systems, Minneapolis, Minnesota, USA (2011)

    Google Scholar 

  77. Yan, Y., Ping, Y., Yi-Ping, Z., Shi-Yong, Z.: Gossip-based scalable and reliable group key distribution framework. In: InfoSecu 2004: Proceedings of the 3rd International Conference on Information Security, pp. 53–61. ACM Press, New York (2004)

    Google Scholar 

  78. van Renesse, R., Minsky, Y., Hayden, M.: A gossip-style failure detection service. In: IFIP (ed.) Proc. of Middleware, the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing, The Lake District, UK, pp. 55–70 (1998)

    Google Scholar 

  79. Guo, K., Hayden, M., van Renesse, R., Vogels, W., Birman, K.P.: Gsgc: An efficient gossip-style garbage collection scheme for scalable reliable multicast. Technical report, Cornell University, Ithaca, NY, USA (1997)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rivière, E., Voulgaris, S. (2011). Gossip-Based Networking for Internet-Scale Distributed Systems. In: Babin, G., Stanoevska-Slabeva, K., Kropf, P. (eds) E-Technologies: Transformation in a Connected World. MCETECH 2011. Lecture Notes in Business Information Processing, vol 78. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20862-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-20862-1_18

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-20861-4

  • Online ISBN: 978-3-642-20862-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics