Skip to main content
SpringerLink
Log in
Book cover

Smart Multicore Embedded Systems pp 3–20Cite as

Parallel Programming Models

  • Chapter
  • First Online:

Abstract

The parallel programming landscape is constantly changing and becoming enriched with new languages, tools and techniques. In this chapter we give a survey on the different parallel programming models available today. These models are suitable for general-purpose computing but are also suitable for programming specialized (e.g. embedded) systems that offer the required facilities. We start by categorizing the available models according to the memory abstraction they discuss to the programmer and then present the representative styles and languages in each category. We cover shared-memory models, distributed-memory models, models for devices with private memory spaces such as gpus and accelerators, as well as models that combine the aforementioned ones in some manner. We conclude with a look towards some other models that do not fall directly in the above categories, which however have a significance of their own.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. OpenMP ARB, “OpenMP Application Program Interface V3.1,” July 2011.

    Google Scholar 

  2. J. Protic, M. Tomasevic, and V. Milutinovic, “Distributed shared memory: Concepts and systems,” IEEE Concurrency, vol. 4, pp. 63–79, 1996.

    Google Scholar 

  3. K. Asanovic, R. Bodik, B. C. Catanzaro, J. J. Gebis, P. Husbands, K. Keutzer, D. A. Patterson, W. L. Plishker, J. Shalf, S. W. Williams, and K. A. Yelick, “The landscape of parallel computing research: A view from Berkeley,” EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2006-183, Dec 2006.

    Google Scholar 

  4. IEEE, “IEEE P1003.1c/D10: Draft standard for information technology – Portable Operating System Interface (POSIX),” Sept 1994.

    Google Scholar 

  5. D. Butenhof, Programming With Posix Threads, ser. Addison-Wesley Professional Computing Series. Addison-Wesley, 1997.

    Google Scholar 

  6. IEEE, “IEEE Std 1003.1j-2000: Standard for information technology – Portable Operating System Interface (POSIX)- part 1: System Application Program Interface (API)-Amendment J: Advanced real-time extensions,” pp. 1–88, 2000.

    Google Scholar 

  7. OpenMP ARB, “OpenMP Application Program Interface V3.0,” May 2008.

    Google Scholar 

  8. F. Liu and V. Chaudhary, “A practical OpenMP compiler for system on chips,” in WOMPAT ’03, Int’l Workshop on OpenMP Applications and Tools, Toronto, Canada, 2003, pp. 54–68.

    Google Scholar 

  9. T. Hanawa, M. Sato, J. Lee, T. Imada, H. Kimura, and T. Boku, “Evaluation of multicore processors for embedded systems by parallel benchmark program using OpenMP,” in IWOMP 2009, 5th International Workshop on OpenMP, Dresden, Germany, June 2009, pp. 15–27.

    Google Scholar 

  10. S. N. Agathos, V. V. Dimakopoulos, A. Mourelis, and A. Papadogiannakis, “Deploying OpenMP on an embedded multicore accelerator,” in SAMOS XIII, 13th Int’l Conference on Embedded Computer Systems: Architectures, MOdeling, and Simulation, Samos, Greece, July 2013.

    Google Scholar 

  11. P. Vander Wolf, E. deKock, T. Henriksson, W. Kruijtzer, and G. Essink, “Design and programming of embedded multiprocessors: an interface-centric approach,” in Proc. CODES+ISSS ’04, 2nd IEEE/ACM/IFIP Int’l Conference on Hardware/software Codesign and System Synthesis, New York, USA, 2004, pp. 206–217.

    Google Scholar 

  12. P. Paulin, C. Pilkington, M. Langevin, E. Bensoudane, and G. Nicolescu, “Parallel programming models for a multi-processor SoC platform applied to high-speed traffic management,” in Proc. CODES+ISSS ’04, 2nd IEEE/ACM/IFIP Int’l Conference on Hardware/software Codesign and System Synthesis, New York, USA, 2004, pp. 48–53.

    Google Scholar 

  13. W. Gropp, E. Lusk, and A. Skjellum, Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edition. Cambridge, MA: MIT Press, 1999.

    Google Scholar 

  14. W. Gropp, E. Lusk, and R. Thakur, Using MPI-2: Advanced Features of the Message-Passing Interface. Cambridge, MA: MIT Press, 1999.

    Google Scholar 

  15. A. Agbaria, D.-I. Kang, and K. Singh, “LMPI: MPI for heterogeneous embedded distributed systems,” in ICPADS ’06, 12th International Conference on Parallel and Distributed Systems - Volume 1. Washington, DC, USA: IEEE Computer Society, 2006, pp. 79–86.

    Google Scholar 

  16. L. Benini, E. Flamand, D. Fuin, and D. Melpignano, “P2012: Building an ecosystem for a scalable, modular and high-efficiency embedded computing accelerator,” in Design, Automation Test in Europe Conference Exhibition (DATE), 2012, 2012, pp. 983–987.

    Google Scholar 

  17. NVIDIA, NVIDIA CUDA Programming Guide 2.0, 2008.

    Google Scholar 

  18. D. B. Kirk and W.-m. W. Hwu, Programming Massively Parallel Processors: A Hands-on Approach, 1st ed. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2010.

    Google Scholar 

  19. Khronos Group, The OpenCL Specification Version 1.0, Beaverton, OR, 2009.

    Google Scholar 

  20. E. Ayguadé, R. M. Badia, F. D. Igual, J. Labarta, R. Mayo, and E. S. Quintana-Ortí, “An extension of the StarSs programming model for platforms with multiple GPUs,” in Euro-Par ’09, 15th International Euro-Par Conference on Parallel Processing. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 851–862.

    Google Scholar 

  21. R. Dolbeau, S. Bihan, and F. Bodin, “HMPP: A hybrid multi-core parallel programming environment,” in GPGPU 2007, Workshop on General Purpose Processing on Graphics Processing Units, 2007.

    Google Scholar 

  22. OpenACC, The OpenACC TM Application Programming Interface Version 1.0, Nov 2011.

    Google Scholar 

  23. Y. Wang, Z. Feng, H. Guo, C. He, and Y. Yang, “Scene recognition acceleration using CUDA and OpenMP,” in ICISE ’09, 1st International Conference on Information Science and Engineering, 2009, pp. 1422–1425.

    Google Scholar 

  24. Q.-k. Chen and J.-k. Zhang, “A stream processor cluster architecture model with the hybrid technology of MPI and CUDA,” in ICISE, 2009, 1st International Conference on Information Science and Engineering, 2009, pp. 86–89.

    Google Scholar 

  25. C. Wright, “Hybrid programming fun: Making bzip2 parallel with MPICH2 & Pthreads on the Cray XD1,” in CUG ’06, 48th Cray User Group Conference, 2006.

    Google Scholar 

  26. W. Pfeiffer and A. Stamatakis, “Hybrid MPI / Pthreads parallelization of the RAxML phylogenetics code,” in 9th IEEE International Workshop on High Performance Computational Biology, Atlanta, GA, Apr 2010.

    Google Scholar 

  27. L. Smith and M. Bull, “Development of mixed mode MPI / OpenMP applications,” Scientific Programming, vol. 9, no. 2,3, pp. 83–98, Aug. 2001.

    Google Scholar 

  28. R. Rabenseifner, “Hybrid parallel programming on HPC platforms,” in EWOMP ’03, 5th European Workshop on OpenMP, Aachen, Germany, Sept 2003, pp. 185–194.

    Google Scholar 

  29. B. Estrade, “Hybrid programming with MPI and OpenMP,” in High Performance Computing Workshop, 2009.

    Google Scholar 

  30. F. Cappello and D. Etiemble, “MPI versus MPI+OpenMP on IBM SP for the NAS benchmarks,” in SC ’00, ACM/IEEE Conference on Supercomputing, Dallas, Texas, USA, 2000.

    Google Scholar 

  31. D. S. Henty, “Performance of hybrid message-passing and shared-memory parallelism for discrete element modeling,” in SC ’00, ACM/IEEE Conference on Supercomputing, Dallas, Texas, USA, 2000.

    Google Scholar 

  32. K. Nakajima, “Parallel iterative solvers for finite-element methods using an OpenMP/MPI hybrid programming model on the earth simulator,” Parallel Computing, vol. 31, no. 10–12, pp. 1048–1065, Oct. 2005.

    Google Scholar 

  33. R. Aversa, B. Di Martino, M. Rak, S. Venticinque, and U. Villano, “Performance prediction through simulation of a hybrid MPI/OpenMP application,” Parallel Comput., vol. 31, no. 10–12, pp. 1013–1033, Oct. 2005.

    Google Scholar 

  34. P. D. Mininni, D. Rosenberg, R. Reddy, and A. Pouquet, “A hybrid MPI-OpenMP scheme for scalable parallel pseudospectral computations for fluid turbulence,” Parallel Computing, vol. 37, no. 6–7, pp. 316–326, 2011.

    Article  Google Scholar 

  35. V. V. Dimakopoulos and P. E. Hadjidoukas, “HOMPI: A hybrid programming framework for expressing and deploying task-based parallelism,” in Euro-Par’11, 17th International Conference on Parallel processing, Bordeaux, France, Aug 2011, pp. 14–26.

    Google Scholar 

  36. R. W. Numrich and J. Reid, “Co-arrays in the next Fortran standard,” SIGPLAN Fortran Forum, vol. 24, no. 2, pp. 4–17, Aug. 2005.

    Google Scholar 

  37. UPC Consortium, “UPC language specifications, v1.2,” Lawrence Berkeley National Lab, Tech. Rep. LBNL-59208, 2005.

    Google Scholar 

  38. M. Frigo, C. E. Leiserson, and K. H. Randall, “The implementation of the Cilk-5 multithreaded language,” in PLDI ’98, ACM SIGPLAN 1998 conference on Programming language design and implementation, Montreal, Quebec, Canada, 1998, pp. 212–223.

    Google Scholar 

  39. K. Fatahalian, D. R. Horn, T. J. Knight, L. Leem, M. Houston, J. Y. Park, M. Erez, M. Ren, A. Aiken, W. J. Dally, and P. Hanrahan, “Sequoia: programming the memory hierarchy,” in SC ’06, 2006 ACM/IEEE Conference on Supercomputing, Tampa, Florida, 2006.

    Google Scholar 

  40. G. Bikshandi, J. Guo, D. Hoeflinger, G. Almasi, B. B. Fraguela, M. J. Garzarán, D. Padua, and C. von Praun, “Programming for parallelism and locality with hierarchically tiled arrays,” in PPoPP ’06, 11th ACM SIGPLAN symposium on Principles and practice of parallel programming, New York, New York, USA, 2006, pp. 48–57.

    Google Scholar 

  41. A. Shafi, B. Carpenter, and M. Baker, “Nested parallelism for multi-core HPC systems using Java,” J. Parallel Distrib. Comput., vol. 69, no. 6, pp. 532–545, Jun. 2009.

    Google Scholar 

  42. K. A. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. N. Hilfinger, S. L. Graham, D. Gay, P. Colella, and A. Aiken, “Titanium: A high-performance Java dialect,” Concurrency: Practice and Experience, vol. 10, no. 11–13, pp. 825–836, 1998.

    Article  Google Scholar 

  43. M. Cole, Algorithmic skeletons: structured management of parallel computation. London: Pitman / MIT Press, 1989.

    MATH  Google Scholar 

  44. M. Cole, “Bringing skeletons out of the closet: a pragmatic manifesto for skeletal parallel programming,” Parallel Computing, vol. 30, no. 3, pp. 389–406, 2004.

    Article  Google Scholar 

  45. M. Danelutto and M. Stigliani, “SKElib: parallel programming with skeletons in C,” in Proc. of 6th Intl. Euro-Par 2000 Parallel Processing, ser. LNCS, A. Bode, T. Ludwing, W. Karl, and R. Wismüller, Eds., vol. 1900. Munich, Germany: Springer, Aug. 2000, pp. 1175–1184.

    Google Scholar 

  46. M. Vanneschi, “The programming model of ASSIST, an environment for parallel and distributed portable applications,” Parallel Computing, vol. 28, no. 12, pp. 1709–1732, 2002.

    Article  MATH  Google Scholar 

  47. M. Leyton and J. M. Piquer, “Skandium: Multi-core programming with algorithmic skeletons,” in PDP ’10, 18th Euromicro Int’l Conference on Parallel, Distributed and Network-Based Processing, 2010, pp. 289–296.

    Google Scholar 

  48. M. Aldinucci, M. Danelutto, P. Kilpatrick, and M. Torquati, “FastFlow: high-level and efficient streaming on multi-core,” in Programming Multi-core and Many-core Computing Systems, ser. Parallel and Distributed Computing, S. Pllana and F. Xhafa, Eds. Wiley, Jan. 2013, ch. 13.

    Google Scholar 

  49. H. González-Vélez and M. Leyton, “A survey of algorithmic skeleton frameworks: high-level structured parallel programming enablers,” Software: Practice and Experience, vol. 40, no. 12, pp. 1135–1160, 2010.

    Article  Google Scholar 

  50. H. Sutter and J. Larus, “Software and the concurrency revolution,” ACM Queue, vol. 3, no. 7, pp. 54–62, Sep. 2005.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Ioannina, Ioannina, GR-45110, Greece

    Vassilios V. Dimakopoulos

Authors
  1. Vassilios V. Dimakopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vassilios V. Dimakopoulos .

Editor information

Editors and Affiliations

  1. Computer Science Department, University of Pisa, Pisa, Italy

    Massimo Torquati

  2. Delft University of Technology, Delft, The Netherlands

    Koen Bertels

  3. Informatics and Mathematical Modeling, DTU Informatik, Lyngby, Denmark

    Sven Karlsson

  4. Commissariat à l’énergie atomique et aux énergies alternatives, CEA LetiMinatec, Grenoble, France

    François Pacull

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Dimakopoulos, V.V. (2014). Parallel Programming Models. In: Torquati, M., Bertels, K., Karlsson, S., Pacull, F. (eds) Smart Multicore Embedded Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8800-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-8800-2_1

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-8799-9

  • Online ISBN: 978-1-4614-8800-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation