From the Origins of Performance Evaluation to New Green ICT Performance Engineering

  • Carlos Juiz
  • Ramon Puigjaner
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6821)


This paper intends to present an overview of the evolution of performance evaluation since its first steps the Erlang works for modelling telephone networks, based on simple queues until the present current challenges in Green ICT that will require the development of new paradigms and mathematical tools, and rapidly passing across the modelling works of Khintchine and Pollaczeck; Jackson; Baskett, Chandy, Muntz and Palacios; Buzen; Reiser and Lavenberg; and many others, and benchmarking standards that have produced solutions to the problems appearing in these hundred of years. Finally, we analyze some of the challenges of computer performance evaluation appearing today, mainly those related to the energy consumption and sustainability, globally known as Green ICT.


Performance evaluation of telephony computer systems and communication networks Performance modeling Queuing theory Queuing network theory Simulation Benchmarking Green ICT 


  1. 1.
    Erlang, A.K.: The Theory of Probabilities and Telephone Conversations. Nyt Tidsskrift for Matematik, B 20 (1909)Google Scholar
  2. 2.
    Erlang, A.K.: Solution of some Problems in the Theory of Probabilities of Significance in Automatic Telephone Exchanges. Elektroteknikeren 17 (1917)Google Scholar
  3. 3.
    Erlang, A.K.: Telephone Waiting Times. Matematisk Tidsskrift, B 31 (1920)Google Scholar
  4. 4.
    Cobham, A.: Priority Assignment in Waiting Line Problems. Journal of the Operations Research Society of America (1954)Google Scholar
  5. 5.
    Jackson, J.R.: Jobshop like Queueing Sysems. Management Science 10 (1963)Google Scholar
  6. 6.
    Gordon, W.J., Newell, G.F.: Closed Queueing Systems with Exponential Servers. Operations Research 15(2) (1967)Google Scholar
  7. 7.
    Buzen, J.P.: Computational Algorithms for Closed Queueing Networks with Exponential Servers. CACM 16(9) (1973)Google Scholar
  8. 8.
    Baskett, F.T., Chandy, K.M., Muntz, R.R., Palacios, F.G.: Open, Closed and Mixed Networks with Different Classes of Customers. JACM 22(2) (1975)Google Scholar
  9. 9.
    Reiser, M., Kobayashi, H.: Queueing networks with multiple closed chains: Theory and computational algorithms. 1BM J. Res. Develop. 19, 3 (1975)Google Scholar
  10. 10.
    Reiser, M., Lavenberg, S.S.: Mean Value Analysis of Closed Multichain Queueing Networks. JACM 27(2) (1980)Google Scholar
  11. 11.
    Denning, P.J., Buzen, J.P.: The Operational Analysis of Queueing Network Models. ACM Computing Surveys 10(3) (1978)Google Scholar
  12. 12.
    Buzen, J.P., Denning, P.J.: Operational Treatment of Queue Length Distributions and Mean Value Analysis. Computer Performance 1(1) (1980)Google Scholar
  13. 13.
    Courtois, P.J.: Decomposability: Queueing and Computer Systems Applications. Academic Press (1977)Google Scholar
  14. 14.
    Marie, R.: Modélisation par Réseaux de Files d’Attente. Thèse de Docteur ès Sciences Mathematiques. Université de Rennes (1977)Google Scholar
  15. 15.
    Gelenbe, E., Mitrani, I.: Analysis and Synthesis of Computer Systems. Academic Press (1981)Google Scholar
  16. 16.
  17. 17.
    SPEC: Standard Performance Evaluation Corporation,
  18. 18.
    OPNET: Applications and Network Performance,
  19. 19.
    ns-2: The Network Simulator,
  20. 20.
    Barroso, L.A., Hözle: The case for energy proportional computing. IEEE Computer 40, 12 (2007)Google Scholar
  21. 21.
    COST IC0804. Energy efficiency in large scale distributed systems,
  22. 22.
    Chen, G., He, W., Liu, J., Nath, S., Rigas, L., Xiao, L., Zhao, F.: Energy-aware server provisioning and load dispatching for connection-intensive internet services. In: NSDI, San Francisco, CA (2008)Google Scholar
  23. 23.
    Standard Performance Evaluation Corporation (SPEC). SPEC power,
  24. 24.
    Fan, X., Weber, W.D., Barroso, L.A.: Power provisioning for a warehouse-sized computer. In: Proceedings of the International Symposium on Computer Architecture, ISCA (2007)Google Scholar
  25. 25.
    Heath, T., Diniz, B., et al.: Energy conservation in heterogeneous server clusters. In: Proceedings of the Symposium on Principles and Practice of Parallel Programming, PPoPP (2005)Google Scholar
  26. 26.
    Chun, B.G., Iannaccone, G., Katz, R.H., Lee, G., Niccolini, L.: An Energy Case for Hybrid Datacenters. In: Workshop on Power Aware Computing and Systems, HotPower 2009 (2009)Google Scholar
  27. 27.
    Flinn, J., Satyanarayanan, M.: Energy-aware adaptation for mobile applications. In: SOSP 1999: Proceedings of the 17th ACM Symposium on Operating Systems Principles (1999)Google Scholar
  28. 28.
    Mankoff, J., Kravets, R., Blevis, E.: Some computer science issues in creating a sustainable world. IEEE Computer 41(8) (2008)Google Scholar
  29. 29.
    Im, C., Ha, S.: Energy optimization for latency- and quality-constrained video applications. IEEE Design and Test of Computers 21(5) (2004)Google Scholar
  30. 30.
    Iyer, S., Luo, L., Mayo, R., Ranganathan, P.: Energy-adaptive display system designs for future mobile environments. In: ACM MobiSys (2003)Google Scholar
  31. 31.
    Gilly, K., Alcaraz, S., Juiz, C., Puigjaner, R.: Analysis of burstiness monitoring and detection in an adaptive Web system. Computer Networks 53(5) (2009)Google Scholar
  32. 32.
    Executive Summary. Chapter 9: Projections of Future Climate Change. Climate Change 2001: The Scientific Basis (2005),
  33. 33.
    Feng, X., Ge, R., Cameron, K.W.: Power and Energy Profiling of Scientific Applications on Distributed Systems. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2005 (2005)Google Scholar
  34. 34.
    Kumar, R., Farkas, K.I., Jouppi, N.P., Ranganathan, P., Tullsen, D.M.: Single-ISA heterogeneous multi-core architectures: The potential for processor power reduction. In: ACM/IEEE MICRO (2003)Google Scholar
  35. 35.
    Pinheiro, E., Bianchini, R., et al.: Load balancing and unbalancing for power and performance in cluster-based systems. In: Proceedings of the Workshop on Compilers and Operating Systems for Low Power, COLP (2001)Google Scholar
  36. 36.
    Ranganathan, P., Rivoire, S., Moore, J.: Power modeling and measurement. In: Advances in Computers. Elsevier (2009)Google Scholar
  37. 37.
    U.S. EPA. Report to congress on server and data center energy efficiency. Tech. Rep. (2007) Google Scholar
  38. 38.
    Kusic, D., Kephart, J.O., Hanson, J.E., Kandasamy, N., Jiang, G.: Power and performance management of virtualized computing environments via lookahead control. Cluster Computing 12(1) (2009)Google Scholar
  39. 39.
    Alcaraz, S., Gilly, K., Juiz, C., Puigjaner, R.: Handling HTTP flows over a DiffServ framework. In: LANC 2007 (2007)Google Scholar
  40. 40.
    Narayanan, D., Donnelly, A., Rowstron, A.: Write off-loading: Practical power management for enterprise storage. In: FAST (2008)Google Scholar
  41. 41.
    Nedevschi, S., Popa, L., Iannaccone, G., Ratnasamy, S., Wetherall, D.: Reducing network energy consumption via sleeping and rate-adaptation. In: NSDI (2008)Google Scholar
  42. 42.
    Park, S., Jiang, W., Zhou, Y., Adve, S.: Managing energy-performance tradeoffs for multithreaded applications on multiprocessor architectures. In: SIGMETRICS (2007)Google Scholar
  43. 43.
    Raghavendra, R., Ranganathan, P., Talwar, V., Wang, Z., Zhu, X.: No power struggles: A unified multi-level power management architecture for the data center. In: ASPLOS (2008)Google Scholar
  44. 44.
    Menasce, D.A., Almeida, V., Dowdy, L.: Performance by Design: Computer Capacity Planning by Example. Prentice Hall PTR (2004)Google Scholar
  45. 45.
    Standard Performance Evaluation Corporation (SPEC). SPECweb2009,
  46. 46.
    Menasce, D.A., Almeida, V.: Capacity Planning for Web Services: metrics, models, and methods. Prentice Hall PTR (2001)Google Scholar
  47. 47.
    Killelea, P.: Web Performance Tuning. O’Reilly (2002)Google Scholar
  48. 48.
    National greenhouse gas inventory data for the period 1990-2007. UN FCCC (2009),
  49. 49.
    Xie, F., Martonosi, M., Malik, S.: Compile-time dynamic voltage scaling settings: opportunities and limits. SIGPLAN Not. 38(5) (2003)Google Scholar
  50. 50.
    Banerjee, P., Patel, C.D., Bash, C., Ranganathan, P.: Sustainable Data Centers: Enabled by Supply and Demand Side Management. In: DAC 2009. ACM Press (2009)Google Scholar
  51. 51.
    Katz, R.: Tech Titans Building Boom. IEEE Spectrum (February 2009)Google Scholar
  52. 52.
    Patel, C.D., Ranganathan, P.: Enterprise power and cooling. ASPLOS Tutorial (2006)Google Scholar
  53. 53.
    Laudon, J.: Performance/Watt: The new server focus. SIGARCH Computer Architecture News 33(4) (2005)Google Scholar
  54. 54.
    Sun Microsystems. SWaP (Space, Watts and Performance) metric,
  55. 55.
    Rivoire, S., Shah, M.A., Ranganathan, P., Kozyrakis, C.: JouleSort: A Balanced Energy-Efficiency Benchmark,
  56. 56.
    Molero, X., Juiz, C., Rodeño, M.J.: Evaluación y Modelado del Rendimiento de Sistemas Informáticos. Pearson Prentice Hall (2004)Google Scholar
  57. 57.
    Deloitte España. Barómetro de Empresas (2009). Google Scholar
  58. 58.
    Pierson, J.M., Hlavacs, H. (eds.): Proceedings of the COST Action IC0804 on Large Scale Distributed Systems, 1st Year (2010)Google Scholar
  59. 59.
    Slack, N., Chambers, S., Betts, A., Johnston, R.: Operations and Process Management: Principles and practice for strategic impact. FT Prentice Hall (2005)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Carlos Juiz
    • 1
  • Ramon Puigjaner
    • 1
  1. 1.Universitat de les Illes BalearsPalmaSpain

Personalised recommendations