A Methodology for Comparing the Execution Time of Metaheuristics Running on Different Hardware
In optimization, search, and learning, it is very common to compare our new results with previous works but, sometimes, we can find some troubles: it is not easy to reproduce the results or to obtain an exact implementation of the original work, or we do not have access to the same processor where the original algorithm was tested for running our own algorithm. With the present work we try to provide the basis for a methodology to characterize the execution time of an algorithm in a processor, given its execution time in another one, so that we could fairly compare algorithms running in different processors. In this paper, we present a proposal for such a methodology, as well as an example of its use applied to two well-known algorithms (Genetic Algorithms and Simulated Annealing) and solving the MAXSAT problem.
KeywordsComparisons metaheuristics performance CPU run time
Unable to display preview. Download preview PDF.
- 1.Cheeseman, P., Kanefsky, B., Taylor, W.M.: Where the really hard problems are. In: Proceedings of the IJCAI 1991, pp. 331–337 (1991)Google Scholar
- 3.Dongarra, J.: Performance of Various Computers Using Standard Linear Algebra Software in a Fortran Environment, http://netlib.org/benchmark/performance.pdf
- 4.Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1990)Google Scholar
- 5.Holland, J.H.: Adaptation in Natural and Artificial Systems, 2nd edn. The MIT Press, Cambridge (1992)Google Scholar
- 6.Kirkpatrik, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science, 671–680 (1983)Google Scholar
- 7.Longbottom, R.: Dhrystone 2.1 Benchmark Results On PCs (2011), http://www.roylongbottom.org.uk/dhrystone%20results.htm
- 8.McMahon, F.H.: The Livermore Fortran Kernels: A Computer Test Of The Numerical Performance Range, Lawrence Livermore National Laboratory, Livermore, California, UCRL-53745 (December 1986)Google Scholar