The Refutation of Amdahl’s Law and Its Variants
Amdahl’s law, imposing a restriction on the speedup achievable by a multiple number of processors, based on the concept of sequential and parallelizable fractions of computations, has been used to justify, among others, asymmetric chip multiprocessor architectures and concerns of “dark silicon”. This paper demonstrates flaws in Amdahl’s law that (i) in theory no inherently sequential fractions of computations exists (ii) sequential fractions appearing in practice are inherently different from parallelizable fractions and therefore usually have different growth rates and that (iii) the time requirement of sequential fractions can be proportional to the number of processors. However, mathematical analyses are also provided to demonstrate that sequential fractions have negligible effect on speedup if the growth rate of the parallelizable fraction is higher than that of the sequential fraction. Examples from computational geometry are given that Amdahl’s law and its variants fail to represent limits to parallel computation. In particular, Gustafson’s law, claimed to be a refutation of Amdahl’s law by some authors, is shown to contradict established theoretical results. We can conclude that no simple formula or law governing concurrency exists.
The author thanks three anonymous reviewers for their support and constructive criticism that helped to improve the presentation of the paper.
- 2.Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of Spring Joint Computer Conference, pp. 483–485. ACM, New York (1967). doi: 10.1145/1465482.1465560
- 5.Borkar, S.: Thousand core chips: a technology perspective. In: Proceedings of 44th Annual Design Automation Conference, DAC 2007, pp. 746–749. ACM, New York (2007). http://doi.acm.org/10.1145/1278480.1278667
- 7.Cook, S.A., Dwork, C.: Bounds on the time for parallel RAM’s to compute simple functions. In: Proceedings of 14th Annual ACM Symposium on Theory of Computing, STOC 1982, pp. 231–233. ACM, New York (1982). doi: 10.1137/0215006
- 15.Fich, F.E., Meyer auf der Heide, F., Ragde, P., Wigderson, A.: One, two, three ... infinity: lower bounds for parallel computation. In: Proceedings of 17th Annual ACM Symposium on Theory of Computing, STOC 1985, pp. 48–58. ACM, New York (1985). http://doi.acm.org/10.1145/22145.22151
- 16.Fich, F.E., Hendler, D., Shavit, N.: Linear lower bounds on real-world implementations of concurrent objects. In: Proceedings of 46th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2005, pp. 165–173 (2005). http://dx.doi.org/10.1109/SFCS.2005.47
- 20.Herlihy, M., Shavit, N.: The Art of Multiprocessor Programming, Revised Reprint, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2012)Google Scholar
- 31.Patterson, D., Hennessy, J.: Computer Organization and Design: The Hardware/Software Interface. The Morgan Kaufmann Series in Computer Architecture and Design, ARM®edn. Elsevier Science, Amsterdam (2016)Google Scholar
- 36.Roughgarden, T., Vassilvitskii, S., Wang, J.R.: Shuffles and circuits: (on lower bounds for modern parallel computation). In: Proceedings of 28th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2016, pp. 1–12. ACM, New York (2016). http://doi.acm.org/10.1145/2935764.2935799