Tuning Paxos for High-Throughput with Batching and Pipelining

  • Nuno Santos
  • André Schiper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7129)


Paxos is probably the most popular state machine replication protocol. Two optimizations that can greatly improve its performance are batching and pipelining. Nevertheless, tuning these two optimizations to achieve high-throughput can be challenging, as their effectiveness depends on many parameters like the network latency and bandwidth, the speed of the nodes, and the properties of the application. We address this question, by first presenting an analytical model of the performance of Paxos that can be used to obtain values for tuning batching and pipelining. We then present results of experiments validating the model and investigating how these two optimizations interact in a WAN. Results for LAN are also mentioned. The results show that although batching by itself is usually sufficient to maximize the throughput in a LAN environment, in a WAN it must be complemented with pipelining.


Window Size Batch Size Maximum Throughput Outgoing Link Request Size 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amir, Y., Kirsch, J.: Paxos for system builders. Tech. Rep. CNDS-2008-2, Johns Hopkins University (2008)Google Scholar
  2. 2.
    Bartoli, A., Calabrese, C., Prica, M., Di Muro, E.A., Montresor, A.: Adaptive Message Packing for Group Communication Systems. In: Meersman, R. (ed.) OTM-WS 2003. LNCS, vol. 2889, pp. 912–925. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. 3.
    Carmeli, B., Gershinsky, G., Harpaz, A., Naaman, N., Nelken, H., Satran, J., Vortman, P.: High throughput reliable message dissemination. In: Proceedings of the 2004 ACM Symposium on Applied Computing, NY, USA (2004)Google Scholar
  4. 4.
    Défago, X., Schiper, A., Urbán, P.: Total order broadcast and multicast algorithms: Taxonomy and survey. ACM Comput. Surv. 36 (December 2004)Google Scholar
  5. 5.
    Friedman, R., Hadad, E.: Adaptive batching for replicated servers. In: Symposium on Reliable Distributed Systems, SRDS 2006 (October 2006)Google Scholar
  6. 6.
    Friedman, R., Renesse, R.: Packing messages as a tool for boosting the performance of total ordering protocols. Tech. Rep. TR95-1527, Department of Computer Science, Cornell University (1995)Google Scholar
  7. 7.
    Guerraoui, R., Levy, R.R., Pochon, B., Quéma, V.: Throughput optimal total order broadcast for cluster environments. ACM Trans. Comput. Syst. 28(2) (2010)Google Scholar
  8. 8.
    Lamport, L.: The part-time parliament. ACM Transactions on Computer Systems 16(2) (May 1998)Google Scholar
  9. 9.
    Marandi, P., Primi, M., Schiper, N., Pedone, F.: Ring Paxos: A high-throughput atomic broadcast protocol. In: Dependable Systems and Networks (DSN 2010) (June 2010)Google Scholar
  10. 10.
    Nagle, J.: Congestion control in IP/TCP internetworks. Tech. Rep. RFC 896, IETF (January 1984)Google Scholar
  11. 11.
    Padmanabhan, V.N., Mogul, J.C.: Improving HTTP latency. Computer Networks and ISDN Systems 28(1-2) (1995)Google Scholar
  12. 12.
    Santos, N., Kończak, J., Żurkowski, T., Wojciechowski, P., Schiper, A.: JPaxos - State machine replication in Java. Tech. Rep. 167765, EPFL (July 2011)Google Scholar
  13. 13.
    Santos, N., Schiper, A.: Tuning Paxos for high-throughput with batching and pipelining. Tech. Rep. 165372, EPFL (July 2011)Google Scholar
  14. 14.
    White, B., et al.: An integrated experimental environment for distributed systems and networks. In: Proc. of the Fifth Symposium on Operating Systems Design and Implementation, Boston, MA (December 2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nuno Santos
    • 1
  • André Schiper
    • 1
  1. 1.Ecole Polytechnique Fédérale de Lausanne (EPFL)Switzerland

Personalised recommendations