Tools for Performance Evaluation of Computer Systems: Historical Evolution and Perspectives

  • Giuliano Casale
  • Marco Gribaudo
  • Giuseppe Serazzi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6821)


The development of software tools for performance evaluation and modeling has been an active research area since the early years of computer science. In this paper, we offer a short overview of historical evolution of the field with an emphasis on popular performance modeling techniques such as queuing networks and Petri nets. A review of recent works that provide new perspectives to software tools for performance modeling is presented, followed by a number of ideas on future research directions for the area.


Discrete Event Simulation Fault Tree Queue Network Model Performance Evaluation Tool Dynamic Fault Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Baskett, F., Chandy, K.M., Muntz, R.R., Palacios, F.G.: Open, closed, and mixed networks of queues with different classes of customers. Journal of the ACM 22(2), 248–260 (1975)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Buchholz, P., Ciardo, G., Donatelli, S., Kemper, P.: Complexity of memory-efficient kronecker operations with applications to the solution of markov models. INFORMS Journal on Computing 12(3), 203–222 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Buzen, J.P.: Computational algorithms for closed queueing networks with exponential servers. Comm. of the ACM 16(9), 527–531 (1973)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Carlson, G.: A user’s view of hardware performance monitors. In: Proc. IFIP Congress, vol. 71, pp. 128–132. North-Holland (1971)Google Scholar
  5. 5.
    Casale, G., Muntz, R.R., Serazzi, G.: Tools for computer performance modeling and reliability analysis. ACM Performance Evaluation Review 36(4) (2009)Google Scholar
  6. 6.
    Chiola, G., Dutheillet, C., Franceschinis, G., Haddad, S.: Stochastic well-formed colored nets and symmetric modeling applications. IEEE Transactions on Computers 42(11), 1343–1360 (1993)CrossRefGoogle Scholar
  7. 7.
    Chung, M.-Y., Ciardo, G., Donatelli, S., He, N., Plateau, B., Stewart, W., Sulaiman, E., Yu, J.: A comparison of structural formalisms for modeling large markov models. In: Proc. of IPDPS, vol. 11, p. 196 (2004)Google Scholar
  8. 8.
    Buzen, J.P., et al.: Best/1 - design of a tool for computer system capacity planning. In: Proc. of the 1978 National Computer Conf., pp. 447–455. AFIPS Press (1978)Google Scholar
  9. 9.
    Ferrari, D., Serazzi, G., Zeigner, A.: Measurement and Tuning of Computer Systems. Prentice-Hall (1983)Google Scholar
  10. 10.
    Fishman, G.S.: Statistical analysis for queueing simulations. Management Science 20(3), 363–369 (1973)CrossRefzbMATHGoogle Scholar
  11. 11.
    Gribaudo, M.: Fspnedit: a fluid stochastic petri net modeling and analysis tool. In: Proc. of Tools of Aachen 2001, pp. 24–28 (2001)Google Scholar
  12. 12.
    Reiser, M., Haring, G., Lindemann, C. (eds.): Dagstuhl Seminar 1997. LNCS, vol. 1769. Springer, Heidelberg (2000)Google Scholar
  13. 13.
    Heidelberger, P., Welch, P.D.: A spectral method for confidence interval generation and run length control in simulations. Comm. of the ACM 24(4), 233–245 (1981)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Herman, D.J.: Scert: a computer evaluation tool. Datamation 13(2), 26–28 (1967)Google Scholar
  15. 15.
    Herman, D.J., Ihrer, F.: The use of a computer to evaluate computers. In: Proc. Conf. 1964 SJCC, Washington DC, pp. 383–395. Spartan Books (1964)Google Scholar
  16. 16.
    Hillston, J.: Fluid flow approximation of pepa models. In: QEST 2005, pp. 33–42, 19–22 (2005)Google Scholar
  17. 17.
    IBM. General purpose systems simulator iii user’s manual. Technical Report Form H20-0163, IBM (1965)Google Scholar
  18. 18.
    Jensen, K.: Coloured Petri Nets: Basic Concepts, Analysis Methods, and Practical Use, 2nd edn. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  19. 19.
    Kartson, D., Balbo, G., Donatelli, S., Franceschinis, G., Conte, G.: Modelling with Generalized Stochastic Petri Nets. John Wiley & Sons, Inc., New York (1994)Google Scholar
  20. 20.
    Kleinrock, L.: Queueing Systems, Theory, vol. 1. John Wiley & Sons, New York (1976)Google Scholar
  21. 21.
    Lavenberg, S.S.: A perspective on queueing models of computer performance. Performance Evaluation 10(1), 53–76 (1989)CrossRefGoogle Scholar
  22. 22.
    LeMay, E., Unkenholz, W., Parks, D., Muehrcke, C., Keefe, K., Sanders, W.H.: Adversary-driven state-based system security evaluation. In: MetriSec 2010: Proceedings of the 6th International Workshop on Security Measurements and Metrics, pp. 1–9. ACM, New York (2010)CrossRefGoogle Scholar
  23. 23.
    Markowitz, H.M., Hausner, B., Karr, H.W.: Simscript: a simulation programming language. Prentice Hall (1963)Google Scholar
  24. 24.
    Martens, A., Koziolek, H., Becker, S., Reussner, R.: Automatically improve software architecture models for performance, reliability, and cost using evolutionary algorithms. In: WOSP/SIPEW, pp. 105–116 (2010)Google Scholar
  25. 25.
    Montani, S., Portinale, L., Bobbio, A., Codetta-Raiteri, D.: Radyban: A tool for reliability analysis of dynamic fault trees through conversion into dynamic bayesian networks. Reliability Engineering and System Safety 93(7), 922–932 (2008); Bayesian Networks in DependabilityCrossRefGoogle Scholar
  26. 26.
    Nielsen, N.R.: Computer simulation of computer system performance. In: Proc. of ACM National Meeting, pp. 581–590 (1967)Google Scholar
  27. 27.
    Pawlikowski, K.: Steady-sate simulation of queueing processes: A survey of problems and solutions. ACM Computing Surveys 22(2), 123–168 (1990)CrossRefMathSciNetGoogle Scholar
  28. 28.
    Potier, D., Veran, M.: The markovian solver of QNAP2 and examples. In: Hasegawa, T., et al. (eds.) Computer Networking and Perf. Eval., pp. 259–279. North-Holland, Amsterdam (1986)Google Scholar
  29. 29.
    Sauer, C.H., Chandy, K.M.: Computer Systems Performance Modeling. Prentice-Hall (1981)Google Scholar
  30. 30.
    Sauer, C.H., McNair, E.A., Kurose, J.F.: The research queueing (RESQ) package, version 2: Introduction and examples. Technical Report IBM rep. no. RA 138, IBM (1982)Google Scholar
  31. 31.
    Saure, D., Glynn, P., Zeevi, A.: A linear programming algorithm for computing the stationary distribution of semi-martingale reflecting brownian motion (under submission)Google Scholar
  32. 32.
    Schwetman, H.: CSIM Reference Manual (1988)Google Scholar
  33. 33.
    Smith, C., Llado, C.: Performance model interchange format (pmif 2.0): Xml definition and implementation. In: Proc. of QUEST 2004. IEEE Press (2004)Google Scholar
  34. 34.
    Smith, C., Lladó, C., Puigjaner, R.: Performance model interchange format (pmif 2): A comprehensive approach to queueing network model interoperability. Perform. Eval. 67(7), 548–568 (2010)CrossRefGoogle Scholar
  35. 35.
    Smith, C.U., Williams, L.G.: Performance Engineering Evaluation of CORBA-Based Distributed Systems with SPE∙ED. In: Puigjaner, R., Savino, N.N., Serra, B. (eds.) TOOLS 1998. LNCS, vol. 1469, pp. 321–335. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  36. 36.
    Smith, C., Williams, L.: Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. Addison-Wesley (2001)Google Scholar
  37. 37.
    Stewart, W.J.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton (1994)zbMATHGoogle Scholar
  38. 38.
    Trivedi, K.S., Bobbio, A., Ciardo, G., German, R., Puliafito, A., Telek, M.: Non-markovian petri nets. In: SIGMETRICS 1995/PERFORMANCE 1995, pp. 263–264. ACM, New York (1995)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Giuliano Casale
    • 1
  • Marco Gribaudo
    • 2
  • Giuseppe Serazzi
    • 2
  1. 1.Imperial College LondonLondonUK
  2. 2.Politecnico di MilanoMilanItaly

Personalised recommendations