Skip to main content
SpringerLink
Log in
Book cover

Parallelism in Matrix Computations pp 3–16Cite as

Parallel Programming Paradigms

  • Chapter
  • First Online:

Part of the book series: Scientific Computation ((SCIENTCOMP))

Abstract

In this chapter, we briefly present the main concepts in parallel computing. This is an attempt to make more precise some definitions that will be used throughout this book rather than a survey on the topic. Interested readers are referred to one of the many books available on the subject, e.g. Arbenz and Petersen, Introduction to Parallel Computing, (2004), Bertsekas and Tsitsiklis, Parallel and Distributed Computation, (1989), Culler et al. Parallel Computer Architecture: A Hardware/Software Approach, (1998), Kumar et al. Introduction to Parallel Computing: Design and Analysis of Algorithms, 2nd edn., (2003), Casanova et al. Parallel Algorithms, (2008), Hennessy and Patterson, Computer Architecture: A Quantitative Approach, (2011), Hockney and Jesshope, Parallel Computers 2: Architecture, Programming and Algorithms, 2nd edn., (1988), Tchuente, Parallel Computation on Regular Arrays. Algorithms and Architectures for Advanced Scientific Computing, (1991) [1–8].

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Arbenz, P., Petersen, W.: Introduction to Parallel Computing. Oxford University Press (2004)

    Google Scholar 

  2. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation. Prentice Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  3. Culler, D., Singh, J., Gupta, A.: Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  4. Kumar, V., Grama, A., Gupta, A., Karypis, G.: Introduction to Parallel Computing: Design and Analysis of Algorithms, 2nd edn. Addison-Wesley (2003)

    Google Scholar 

  5. Casanova, H., Legrand, A., Robert, Y.: Parallel Algorithms. Chapman & Hall/CRC Press (2008)

    Google Scholar 

  6. Hennessy, J., Patterson, D.: Computer Architecture: A Quantitative Approach. Elsevier Science & Technology (2011)

    Google Scholar 

  7. Hockney, R., Jesshope, C.: Parallel Computers 2: Architecture, Programming and Algorithms, 2nd edn. Adam Hilger (1988)

    Google Scholar 

  8. Tchuente, M.: Parallel Computation on Regular Arrays. Algorithms and Architectures for Advanced Scientific Computing. Manchester University Press (1991)

    Google Scholar 

  9. Flynn, M.: Some computer organizations and their effectiveness. IEEE Trans. Comput. C-21, 948–960 (1972)

    Google Scholar 

  10. Jeffers, J., Reinders, J.: Intel Xeon Phi Coprocessor High Performance Programming, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2013)

    Google Scholar 

  11. Hockney, R.: The Science of Computer Benchmarking. SIAM, Philadelphia (1996)

    Book  Google Scholar 

  12. Higham, N.: Accuracy and Stability of Numerical Algorithms, 2nd edn. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  13. Karp, R., Sahay, A., Santos, E., Schauser, K.: Optimal broadcast and summation in the logP model. In: Proceedings of the 5th Annual ACM Symposium on Parallel Algorithms and Architectures SPAA’93, pp. 142–153. ACM Press, Velen (1993). http://doi.acm.org/10.1145/165231.165250

  14. Culler, D., Karp, R., Patterson, D., A. Sahay, K.S., Santos, E., Subramonian, R., von Eicken, T.: LogP: towards a realistic model of parallel computation. In: Principles, Practice of Parallel Programming, pp. 1–12 (1993). http://citeseer.ist.psu.edu/culler93logp.html

    Google Scholar 

  15. Pjesivac-Grbovic, J., Angskun, T., Bosilca, G., Gabriel, G.F.E., Dongarra, J.: Performance analysis of MPI collective operations. In: Fourth International Workshop on Performance Modeling, Evaluation, and Optimization of Parallel and Distributed Systems (PMEO-PDS’05). Denver (2005). (Submitted)

    Google Scholar 

  16. Breshaers, C.: The Art of Concurrency - A Thread Monkey’s Guide to Writing Parallel Applications. O’Reilly (2009)

    Google Scholar 

  17. Rauber, T., Rünger, G.: Parallel Programming—for Multicore and Cluster Systems. Springer (2010)

    Google Scholar 

  18. Darema., F.: The SPMD model : past, present and future. In: Recent Advances in Parallel Virtual Machine and Message Passing Interface. LNCS, vol. 2131/2001, p. 1. Springer, Berlin (2001)

    Google Scholar 

  19. Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press, Cambridge (1994)

    Google Scholar 

  20. Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: MPI: The Complete Reference (1995). http://www.netlib.org/utk/papers/mpi-book/mpi-book.html

  21. Chapman, B., Jost, G., Pas, R.: Using OpenMP: Portable Shared Memory Parallel Programming. The MIT Press, Cambridge (2007)

    Google Scholar 

  22. OpenMP Architecture Review Board: OpenMP Application Program Interface (Version 3.1). (2011). http://www.openmp.org/mp-documents/

  23. Amdahl, G.M.: Validity of the single processor approach to achieving large scale computing capabilities. Proc. AFIPS Spring Jt. Comput. Conf. 31, 483–485 (1967)

    MATH  Google Scholar 

  24. Juurlink, B., Meenderinck, C.: Amdahl’s law for predicting the future of multicores considered harmful. SIGARCH Comput. Archit. News 40(2), 1–9 (2012). doi:10.1145/2234336.2234338. http://doi.acm.org/10.1145/2234336.2234338

    Google Scholar 

  25. Hill, M., Marty, M.: Amdahl’s law in the multicore era. In: HPCA, p. 187. IEEE Computer Society (2008)

    Google Scholar 

  26. Sun, X.H., Chen, Y.: Reevaluating Amdahl’s law in the multicore era. J. Parallel Distrib. Comput. 70(2), 183–188 (2010)

    Article  MATH  Google Scholar 

  27. Flatt, H., Kennedy, K.: Performance of parallel processors. Parallel Comput. 12(1), 1–20 (1989). doi:10.1016/0167-8191(89)90003-3. http://www.sciencedirect.com/science/article/pii/0167819189900033

    Google Scholar 

  28. Kuck, D.: High Performance Computing: Challenges for Future Systems. Oxford University Press, New York (1996)

    Google Scholar 

  29. Kuck, D.: What do users of parallel computer systems really need? Int. J. Parallel Program. 22(1), 99–127 (1994). doi:10.1007/BF02577794. http://dx.doi.org/10.1007/BF02577794

    Google Scholar 

  30. Kumar, V., Gupta, A.: Analyzing scalability of parallel algorithms and architectures. J. Parallel Distrib. Comput. 22(3), 379–391 (1994)

    Article  Google Scholar 

  31. Worley, P.H.: The effect of time constraints on scaled speedup. SIAM J. Sci. Stat. Comput. 11(5), 838–858 (1990)

    Article  MathSciNet  Google Scholar 

  32. Gustafson, J.: Reevaluating Amdahl’s law. Commun. ACM 31(5), 532–533 (1988)

    Article  MATH  Google Scholar 

  33. Grama, A., Gupta, A., Kumar, V.: Isoefficiency: measuring the scalability of parallel algorithms and architectures. IEEE Parallel Distrib. Technol. 12–21 (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering and Informatics Department, University of Patras, Patras, Greece

    Efstratios Gallopoulos

  2. Campus de Beaulieu, INRIA/IRISA, Rennes Cedex, France

    Bernard Philippe

  3. Department of Computer Science, Purdue University, West Lafayette, IN, USA

    Ahmed H. Sameh

Authors
  1. Efstratios Gallopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernard Philippe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed H. Sameh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Efstratios Gallopoulos .

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Gallopoulos, E., Philippe, B., Sameh, A.H. (2016). Parallel Programming Paradigms. In: Parallelism in Matrix Computations. Scientific Computation. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-7188-7_1

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-7188-7_1

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-017-7187-0

  • Online ISBN: 978-94-017-7188-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation