Skip to main content
SpringerLink
Log in

The Refutation of Amdahl’s Law and Its Variants

  • Conference paper
  • First Online:
Computational Science and Its Applications – ICCSA 2017 (ICCSA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10405))

Included in the following conference series:

Abstract

Amdahl’s law, imposing a restriction on the speedup achievable by a multiple number of processors, based on the concept of sequential and parallelizable fractions of computations, has been used to justify, among others, asymmetric chip multiprocessor architectures and concerns of “dark silicon”. This paper demonstrates flaws in Amdahl’s law that (i) in theory no inherently sequential fractions of computations exists (ii) sequential fractions appearing in practice are inherently different from parallelizable fractions and therefore usually have different growth rates and that (iii) the time requirement of sequential fractions can be proportional to the number of processors. However, mathematical analyses are also provided to demonstrate that sequential fractions have negligible effect on speedup if the growth rate of the parallelizable fraction is higher than that of the sequential fraction. Examples from computational geometry are given that Amdahl’s law and its variants fail to represent limits to parallel computation. In particular, Gustafson’s law, claimed to be a refutation of Amdahl’s law by some authors, is shown to contradict established theoretical results. We can conclude that no simple formula or law governing concurrency exists.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akl, S.G., Cosnard, M., Ferreira, A.G.: Data-movement-intensive problems: two folk theorems in parallel computation revisited. Theor. Comput. Sci. 95(2), 323–337 (1992). doi:10.1016/0304-3975(92)90271-G

    Article  MathSciNet  MATH  Google Scholar 

  2. Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. In: Proceedings of Spring Joint Computer Conference, pp. 483–485. ACM, New York (1967). doi:10.1145/1465482.1465560

  3. Annavaram, M., Grochowski, E., Shen, J.: Mitigating Amdahl’s law through EPI throttling. SIGARCH Comput. Archit. News 33(2), 298–309 (2005). http://doi.acm.org/10.1145/1080695.1069995

    Article  Google Scholar 

  4. Atallah, M.J., Callahan, P.B., Goodrich, M.T.: P-complete geometric problems. Int. J. Comput. Geom. Appl. 03(04), 443–462 (1993). doi:10.1142/S0218195993000282

    Article  MathSciNet  MATH  Google Scholar 

  5. Borkar, S.: Thousand core chips: a technology perspective. In: Proceedings of 44th Annual Design Automation Conference, DAC 2007, pp. 746–749. ACM, New York (2007). http://doi.acm.org/10.1145/1278480.1278667

  6. Castanho, C.D., Chen, W., Wada, K., Fujiwara, A.: Parallelizability of some P-complete geometric problems in the EREW-PRAM. In: Wang, J. (ed.) COCOON 2001. LNCS, vol. 2108, pp. 59–63. Springer, Heidelberg (2001). doi:10.1007/3-540-44679-6_7

    Chapter  Google Scholar 

  7. Cook, S.A., Dwork, C.: Bounds on the time for parallel RAM’s to compute simple functions. In: Proceedings of 14th Annual ACM Symposium on Theory of Computing, STOC 1982, pp. 231–233. ACM, New York (1982). doi:10.1137/0215006

  8. Denning, P.J., Lewis, T.G.: Exponential laws of computing growth. Commun. ACM 60(1), 54–65 (2017). http://doi.acm.org/10.1145/2976758

    Article  Google Scholar 

  9. Dévai, F.: An optimal hidden-surface algorithm and its parallelization. In: Murgante, B., Gervasi, O., Iglesias, A., Taniar, D., Apduhan, B.O. (eds.) ICCSA 2011. LNCS, vol. 6784, pp. 17–29. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21931-3_2

    Chapter  Google Scholar 

  10. Dévai, F.: Gustafson’s law contradicts theory results. (Letter to the Editor). Commun. ACM 60(4), 8–9 (2017). http://doi.acm.org/10.1145/3056859

    Article  Google Scholar 

  11. Dymond, P.W., Tompa, M.: Speedups of deterministic machines by synchronous parallel machines. J. Comput. Syst. Sci. 30(2), 149–161 (1985). doi:10.1145/800061.808763

    Article  MathSciNet  MATH  Google Scholar 

  12. Ellen, F., Hendler, D., Shavit, N.: On the inherent sequentiality of concurrent objects. SIAM J. Comput. 41(3), 519–536 (2012). doi:10.1137/08072646X

    Article  MathSciNet  MATH  Google Scholar 

  13. Esmaeilzadeh, H., et al.: Power challenges may end the multicore era. Commun. ACM 56(2), 93–102 (2013). http://doi.acm.org/10.1145/2408776.2408797

    Article  Google Scholar 

  14. Eyerman, S., Eeckhout, L.: Modeling critical sections in Amdahl’s law and its implications for multicore design. SIGARCH Comput. Archit. News 38(3), 362–370 (2010). http://doi.acm.org/10.1145/1816038.1816011

    Article  Google Scholar 

  15. Fich, F.E., Meyer auf der Heide, F., Ragde, P., Wigderson, A.: One, two, three ... infinity: lower bounds for parallel computation. In: Proceedings of 17th Annual ACM Symposium on Theory of Computing, STOC 1985, pp. 48–58. ACM, New York (1985). http://doi.acm.org/10.1145/22145.22151

  16. Fich, F.E., Hendler, D., Shavit, N.: Linear lower bounds on real-world implementations of concurrent objects. In: Proceedings of 46th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2005, pp. 165–173 (2005). http://dx.doi.org/10.1109/SFCS.2005.47

  17. Greenlaw, R., Hoover, H.J., Ruzzo, W.L.: Limits to Parallel Computation: P-Completeness Theory. Oxford University Press Inc., New York (1995)

    MATH  Google Scholar 

  18. Gustafson, J.L.: Reevaluating Amdahl’s law. Commun. ACM 31(5), 532–533 (1988). doi:10.1145/42411.42415

    Article  Google Scholar 

  19. Hennessy, J.L., Patterson, D.A.: Computer Architecture: A Quantitative Approach, 5th edn. Morgan Kaufmann Publishers Inc., San Francisco (2011)

    MATH  Google Scholar 

  20. Herlihy, M., Shavit, N.: The Art of Multiprocessor Programming, Revised Reprint, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2012)

    Google Scholar 

  21. Hill, M.D., Marty, M.R.: Amdahl’s law in the multicore era. Computer 41(7), 33–38 (2008). doi:10.1109/MC.2008.209

    Article  Google Scholar 

  22. Juurlink, B., Meenderinck, C.H.: Amdahl’s law for predicting the future of multicores considered harmful. SIGARCH Comput. Archit. News 40(2), 1–9 (2012). doi:10.1145/2234336.2234338

    Article  Google Scholar 

  23. Kuck, D.J.: A survey of parallel machine organization and programming. ACM Comput. Surv. 9(1), 29–59 (1977). http://doi.acm.org/10.1145/356683.356686

    Article  MathSciNet  MATH  Google Scholar 

  24. Kumar, R., Tullsen, D.M., Jouppi, N.P., Ranganathan, P.: Heterogeneous chip multiprocessors. Computer 38(11), 32–38 (2005). doi:10.1109/MC.2005.379

    Article  Google Scholar 

  25. Lamport, L.: A new solution of Dijkstra’s concurrent programming problem. Commun. ACM 17(8), 453–455 (1974). http://doi.acm.org/10.1145/361082.361093

    Article  MathSciNet  MATH  Google Scholar 

  26. Luccio, F., Pagli, L.: The p-shovelers problem: (computing with time-varying data). SIGACT News 23, 72–75 (1992). doi:10.1145/130956.130960

    Article  Google Scholar 

  27. Mak, L.: Parallelism always helps. SIAM J. Comput. 26(1), 153–172 (1997). doi:10.1137/S0097539794265402

    Article  MathSciNet  MATH  Google Scholar 

  28. McKenna, M.: Worst-case optimal hidden-surface removal. ACM Trans. Graph. 6, 19–28 (1987). doi:10.1145/27625.27627

    Article  Google Scholar 

  29. Mittal, S.: A survey of techniques for architecting and managing asymmetric multicore processors. ACM Comput. Surv. 48(3), 45:1–45:38 (1987). http://doi.acm.org/10.1145/2856125

    Google Scholar 

  30. Morad, A., Yavits, L., Kvatinsky, S., Ginosar, R.: Resistive GP-SIMD processing-in-memory. ACM Trans. Archit. Code Optim. 12(4), 57:1–57:22 (2016). http://doi.acm.org/10.1145/2845084

    Article  Google Scholar 

  31. Patterson, D., Hennessy, J.: Computer Organization and Design: The Hardware/Software Interface. The Morgan Kaufmann Series in Computer Architecture and Design, ARM®edn. Elsevier Science, Amsterdam (2016)

    Google Scholar 

  32. Patterson, D.A., et al.: A case for redundant arrays of inexpensive disks (RAID). SIGMOD Rec. 17(3), 109–116 (1988). doi:10.1145/971701.50214

    Article  Google Scholar 

  33. Paul, J.M., Meyer, B.H.: Amdahl’s law revisited for single chip systems. Int. J. Parallel Program. 35(2), 101–123 (2007). http://dx.doi.org/10.1007/s10766-006-0028-8

    Article  MATH  Google Scholar 

  34. Preparata, F.P.: Should Amdahl’s law be repealed? (abstract). In: Staples, J., Eades, P., Katoh, N., Moffat, A. (eds.) ISAAC 1995. LNCS, vol. 1004, p. 311. Springer, Heidelberg (1995). doi:10.1007/BFb0015436

    Chapter  Google Scholar 

  35. Reif, J.H.: Depth-first search is inherently sequential. Inf. Process. Lett. 20(5), 229–234 (1985). doi:10.1016/0020-0190(85)90024-9

    Article  MathSciNet  MATH  Google Scholar 

  36. Roughgarden, T., Vassilvitskii, S., Wang, J.R.: Shuffles and circuits: (on lower bounds for modern parallel computation). In: Proceedings of 28th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2016, pp. 1–12. ACM, New York (2016). http://doi.acm.org/10.1145/2935764.2935799

  37. Shavit, N.: Data structures in the multicore age. Commun. ACM 54, 76–84 (2011). http://doi.acm.org/10.1145/1897852.1897873

    Article  Google Scholar 

  38. Suleman, M.A., Mutlu, O., Qureshi, M.K., Patt, Y.N.: Accelerating critical section execution with asymmetric multi-core architectures. SIGPLAN Not. 44(3), 253–264 (2009). http://doi.acm.org/10.1145/1508284.1508274

    Article  Google Scholar 

  39. Sun, X.H., Chen, Y.: Reevaluating Amdahl’s law in the multicore era. J. Parallel Distrib. Comput. 70(2), 183–188 (2010). doi:10.1016/j.jpdc.2009.05.002

    Article  MATH  Google Scholar 

  40. Valiant, L.G.: Parallelism in comparison problems. SIAM J. Comput. 4(3), 348–355 (1975). http://dx.doi.org/10.1137/0204030

    Article  MathSciNet  MATH  Google Scholar 

  41. Valiant, L.G.: A bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990). http://doi.acm.org/10.1145/79173.79181

    Article  Google Scholar 

  42. Woo, D.H., Lee, H.H.: Extending Amdahl’s law for energy-efficient computing in the many-core era. Computer 41(12), 24–31 (2008). doi:10.1109/MC.2008.494

    Article  Google Scholar 

  43. Yavits, L., Morad, A., Ginosar, R.: The effect of communication and synchronization on Amdahl’s law in multicore systems. Parallel Comput. 40(1), 1–16 (2014). http://dx.doi.org/10.1016/j.parco.2013.11.001

    Article  MathSciNet  Google Scholar 

  44. Yavits, L., Morad, A., Ginosar, R.: The effect of temperature on Amdahl law in 3D multicore era. IEEE Trans. Comput. 65(6), 2010–2013 (2016). http://dx.doi.org/10.1109/TC.2015.2458865

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The author thanks three anonymous reviewers for their support and constructive criticism that helped to improve the presentation of the paper.

Author information

Authors and Affiliations

  1. London South Bank University, London, UK

    Ferenc Dévai

  2. Hungarian Academy of Sciences, Budapest, Hungary

    Ferenc Dévai

Authors
  1. Ferenc Dévai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ferenc Dévai .

Editor information

Editors and Affiliations

  1. University of Perugia, Perugia, Italy

    Osvaldo Gervasi

  2. University of Basilicata, Potenza, Italy

    Beniamino Murgante

  3. Covenant University, Ota, Nigeria

    Sanjay Misra

  4. University of Trieste, Trieste, Italy

    Giuseppe Borruso

  5. Polytechnic University of Bari, Bari, Italy

    Carmelo M. Torre

  6. University of Minho, Braga, Portugal

    Ana Maria A.C. Rocha

  7. Monash University, Clayton, Victoria, Australia

    David Taniar

  8. Kyushu Sangyo University, Fukuoka, Japan

    Bernady O. Apduhan

  9. Saint Petersburg State University, Saint Petersburg, Russia

    Elena Stankova

  10. University of Trieste, Trieste, Italy

    Alfredo Cuzzocrea

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Dévai, F. (2017). The Refutation of Amdahl’s Law and Its Variants. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2017. ICCSA 2017. Lecture Notes in Computer Science(), vol 10405. Springer, Cham. https://doi.org/10.1007/978-3-319-62395-5_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62395-5_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-62394-8

  • Online ISBN: 978-3-319-62395-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation