Advertisement

Iterating Skeletons

Structured Parallelism by Composition
  • Mischa DieterleEmail author
  • Thomas Horstmeyer
  • Jost Berthold
  • Rita Loogen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8241)

Abstract

Algorithmic skeletons are higher-order functions which provide tools for parallel programming at a higher abstraction level, hiding the technical details of parallel execution inside the skeleton implementation. However, this encapsulation becomes an obstacle when the actual algorithm is one that involves iterative application of the same skeleton to successively improve or approximate the result. Striving for a general and portable solution, we propose a skeleton iteration framework in which arbitrary skeletons can be embedded with only minor modifications. The framework is flexible and allows for various parallel iteration control and parallel iteration body variants. We have implemented it in the parallel Haskell dialect Eden using dedicated stream communication types for the iteration. Two non-trivial case studies show the practicality of our approach. The performance of our compositional iteration framework is competitive with customised iteration skeletons.

Keywords

Iteration Control Iteration Step Iteration Scheme Output Stream Iteration Body 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

The authors thank the anonymous referees for their helpful comments on a previous version of this paper. Jost Berthold was partially supported by DSF under contract number 10-092299 (Hiperfit).

References

  1. 1.
    Berthold, J., Dieterle, M., Loogen, R.: Implementing parallel google map-reduce in Eden. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 990–1002. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Bu, Y., Howe, B., Balazinska, M., Ernst, M.D.: The HaLoop approach to large-scale iterative data analysis. VLDB J. 21(2), 169–190 (2012)CrossRefGoogle Scholar
  3. 3.
    Bird, R.S.: Using circular programs to eliminate multiple traversals of data. Acta Inform. 21, 239–250 (1984)CrossRefzbMATHGoogle Scholar
  4. 4.
    Cole, M.: Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, Cambridge (1989)zbMATHGoogle Scholar
  5. 5.
    Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. CACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  6. 6.
    Dieterle, M., Horstmeyer, T., Loogen, R.: Skeleton composition using remote data. In: Carro, M., Peña, R. (eds.) PADL 2010. LNCS, vol. 5937, pp. 73–87. Springer, Heidelberg (2010)Google Scholar
  7. 7.
    Ekanayake, J., Li, H., Zhang, B., Gunarathne, Th., Bae, S., Qiu, J., Fox, G.: Twister: a runtime for iterative mapreduce. In: HPDC ’10. ACM (2010)Google Scholar
  8. 8.
    Ewen, St, Tzoumas, K., Kaufmann, M., Markl, V.: Spinning fast iterative data flows. PVLDB 5(11), 1268–1279 (2012)Google Scholar
  9. 9.
    Loogen, R., Ortega-Mallén, Y., Peña-Marí, R.: Parallel functional programming in Eden. J. Funct. Program. 15(3), 431–475 (2005)CrossRefzbMATHGoogle Scholar
  10. 10.
    Loogen, R.: Eden – parallel functional programming with Haskell. In: Zsók, V., Horváth, Z., Plasmeijer, R. (eds.) CEFP. LNCS, vol. 7241, pp. 142–206. Springer, Heidelberg (2012)Google Scholar
  11. 11.
    Leyton, M., Piquer, J.M.: Skandium: multi-core programming with algorithmic skeletons. In: PDP. IEEE Computer Society (2010)Google Scholar
  12. 12.
    MacKay, D.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003). See chapter 20, p. 284ffzbMATHGoogle Scholar
  13. 13.
    Peña, R., Rubio, F.: Parallel functional programming at two levels of abstraction. In: PPDP’01, pp. 187–198. ACM (2001)Google Scholar
  14. 14.
    Zhang, Y., Gao, Q., Gao, L., Wang, C.: iMapReduce: a distributed computing framework for iterative computation. JOGC 10, 47–68 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mischa Dieterle
    • 1
    Email author
  • Thomas Horstmeyer
    • 1
  • Jost Berthold
    • 2
  • Rita Loogen
    • 1
  1. 1.FB Mathematik und InformatikPhilipps-Universität MarburgMarburgGermany
  2. 2.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark

Personalised recommendations