A Methodology for Comparing the Execution Time of Metaheuristics Running on Different Hardware

  • Julián Domínguez
  • Enrique Alba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7245)


In optimization, search, and learning, it is very common to compare our new results with previous works but, sometimes, we can find some troubles: it is not easy to reproduce the results or to obtain an exact implementation of the original work, or we do not have access to the same processor where the original algorithm was tested for running our own algorithm. With the present work we try to provide the basis for a methodology to characterize the execution time of an algorithm in a processor, given its execution time in another one, so that we could fairly compare algorithms running in different processors. In this paper, we present a proposal for such a methodology, as well as an example of its use applied to two well-known algorithms (Genetic Algorithms and Simulated Annealing) and solving the MAXSAT problem.


Comparisons metaheuristics performance CPU run time 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cheeseman, P., Kanefsky, B., Taylor, W.M.: Where the really hard problems are. In: Proceedings of the IJCAI 1991, pp. 331–337 (1991)Google Scholar
  2. 2.
    Curnow, H.J., Wichmann, B.A.: A synthetic benchmark. The Computer Journal 19(1), 43–49 (1976)CrossRefGoogle Scholar
  3. 3.
    Dongarra, J.: Performance of Various Computers Using Standard Linear Algebra Software in a Fortran Environment,
  4. 4.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1990)Google Scholar
  5. 5.
    Holland, J.H.: Adaptation in Natural and Artificial Systems, 2nd edn. The MIT Press, Cambridge (1992)Google Scholar
  6. 6.
    Kirkpatrik, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science, 671–680 (1983)Google Scholar
  7. 7.
    Longbottom, R.: Dhrystone 2.1 Benchmark Results On PCs (2011),
  8. 8.
    McMahon, F.H.: The Livermore Fortran Kernels: A Computer Test Of The Numerical Performance Range, Lawrence Livermore National Laboratory, Livermore, California, UCRL-53745 (December 1986)Google Scholar
  9. 9.
    Weicker, R.P.: Dhrystone: a synthetic systems programming benchmark. Commun. ACM 27, 10 (1984)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Julián Domínguez
    • 1
  • Enrique Alba
    • 1
  1. 1.ETSIIUniversidad de MálagaMálagaSpain

Personalised recommendations