Practical Evaluation of Large Scale Applications

  • Tiago JorgeEmail author
  • Francisco Maia
  • Miguel Matos
  • José Pereira
  • Rui Oliveira
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9038)


Designing and implementing distributed systems is a hard endeavor, both at an abstract level when designing the system, and at a concrete level when implementing, debugging and evaluating it. This stems not only from the inherent complexity of writing and reasoning about distributed software, but also from the lack of tools for testing and evaluating it under realistic conditions. Moreover, the gap between the protocols’ specifications found on research papers and their implementations on real code is huge, leading to inconsistencies that often result in the implementation no longer following the specification. As an example, the specification of the popular Chord DHT comprises a few dozens of lines, while its Java implementation, OpenChord, is close to twenty thousand lines, excluding libraries. This makes it hard and error prone to change the implementation to reflect changes in the specification, regardless of programmers’ skill. Besides, critical behavior due to the unpredictable interleaving of operations and network uncertainty, can only be observed on a realistic setting, limiting the usefulness of simulation tools. We believe that being able to write an algorithm implementation very close to its specification, and evaluating it in a real environment is a big step in the direction of building better distributed systems. Our approach leverages the MINHA platform to offer a set of built in primitives that allows one to program very close to pseudo-code. This high level implementation can interact with off-the-shelf existing middleware and can be gradually replaced by a production-ready Java implementation. In this paper, we present the system design and showcase it using a well-known algorithm from the literature.


Testing and evaluation Distributed systems Simulation and emulation 


  1. 1.
    Alvarez, G.A., Cristian, F.: Applying simulation to the design and performance evaluation of fault-tolerant systems. In: Proceedings of the Sixteenth Symposium on Reliable Distributed Systems, pp. 35–42. IEEE (1997)Google Scholar
  2. 2.
    Barr, R., Haas, Z.J., van Renesse, R.: Jist: An efficient approach to simulation using virtual machines. Software: Practice and Experience 35(6), 539–576 (2005)Google Scholar
  3. 3.
    Bavier, A.C., Bowman, M., Chun, B.N., Culler, D.E., Karlin, S., Muir, S., Peterson, L.L., Roscoe, T., Spalink, T., Wawrzoniak, M.: Operating systems support for planetary-scale network services. In: NSDI, vol. 4, p. 19 (2004)Google Scholar
  4. 4.
    Carvalho, N.A., Bordalo, J., Campos, F., Pereira, J.: Experimental evaluation of distributed middleware with a virtualized java environment. In: Proceedings of the 6th Workshop on Middleware for Service Oriented Computing, p. 3. ACM (2011)Google Scholar
  5. 5.
    Galuba, W., Aberer, K., Despotovic, Z., Kellerer, W.: Protopeer: From simulation to live deployment in one step. In: Eighth International Conference on Peer-to-Peer Computing, P2P 2008, pp. 191–192. IEEE (2008)Google Scholar
  6. 6.
    Ierusalimschy, R., De Figueiredo, L.H., Celes Filho, W.: The implementation of lua 5.0. J. UCS 11(7), 1159–1176 (2005)Google Scholar
  7. 7.
    JCP - Java Community Process: JSR-223 Java Scripting API (2006),
  8. 8.
    Leonini, L., Rivière, É., Felber, P.: Splay: Distributed systems evaluation made simple (or how to turn ideas into live systems in a breeze). In: NSDI, vol. 9, pp. 185–198 (2009)Google Scholar
  9. 9.
    Maia, F., Matos, M., Vilaça, R., Pereira, J., Oliveira, R., Riviere, E.: Dataflasks: epidemic store for massive scale systems. In: 2014 IEEE 33rd International Symposium on Reliable Distributed Systems (SRDS), pp. 79–88. IEEE (2014)Google Scholar
  10. 10.
    Matos, M., Felber, P., Oliveira, R., Pereira, J.O., Riviere, E.: Scaling up publish/subscribe overlays using interest correlation for link sharing. IEEE Transactions on Parallel & Distributed Systems 24(12), 2462–2471 (2013)CrossRefGoogle Scholar
  11. 11.
    Montresor, A., Jelasity, M.: PeerSim: A scalable P2P simulator. In: Proc. of the 9th Int. Conference on Peer-to-Peer (P2P 2009), Seattle, WA, pp. 99–100 (September 2009)Google Scholar
  12. 12.
    The Network Simulator NS-2,
  13. 13.
    Stoica, I., Morris, R., Liben-Nowell, D., Karger, D.R., Kaashoek, M.F., Dabek, F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Transactions on Networking 11(1), 17–32 (2003)CrossRefGoogle Scholar
  14. 14.
    Urban, P., Défago, X., Schiper, A.: Neko: A single environment to simulate and prototype distributed algorithms. In: Proceedings of 15th International Conference on Information Networking 2001, pp. 503–511. IEEE (2001)Google Scholar
  15. 15.
    Vahdat, A., Yocum, K., Walsh, K., Mahadevan, P., Kostić, D., Chase, J., Becker, D.: Scalability and accuracy in a large-scale network emulator. ACM SIGOPS Operating Systems Review 36(SI), 271–284 (2002)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Tiago Jorge
    • 1
    Email author
  • Francisco Maia
    • 1
  • Miguel Matos
    • 1
  • José Pereira
    • 1
  • Rui Oliveira
    • 1
  1. 1.INESC TEC and U. MinhoBragaPortugal

Personalised recommendations