Solving Large Systems of Differential Equations in Parallel Using Covers and Skeletons

  • M. Südholt
  • C. Piepenbrock
  • K. Obermayer
  • P. Pepper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT)


The design and implementation of parallel algorithms for distributed memory architectures is much harder than the development of sequential algorithms. This is mainly due to the communication and synchronization that is necessary to manage distributed data correctly. This paper applies a methodology for the transformational derivation of parallel programs using data distribution algebras that enable an abstract description of data distribution issues. Algorithms are formulated using skeletons, that is, specialized higher-order functions with particular parallel implementations. The methodology is applied to a the solution of a system of ordinary differential equations where convolutions can be computed using the Fast Fourier transformation. The example illustrates the practical optimization problems for a development model of the visual system that involves large scale neural network simulations. Finally, this algorithm is compared to an implementation of the same system of equations in the programming language C* on a CM-5.


Functional programming parallel programming numerical algorithm program transformation skeleton data distribution algebra 


  1. Bauer, F. et al. (1985), The Munich Project CIP. Vol. I: The Wide Spectrum Language CIP-L, number 183 in ‘LNCS’, Springer Verlag, Berlin.CrossRefGoogle Scholar
  2. Bird, R. S. (1989), Lectures on constructive functional programming, in M. Broy, ed., ‘Constructive Methods in Computing Science’, Vol. 55 of NATO ASI, F, Springer, pp. 151–216.CrossRefGoogle Scholar
  3. Bratvold, T. A. (1993), A skeleton-based parallelising compiler for ML, in R. Plasmeijer & M. van Eekelen, eds, ‘Implementation of Functional Languages’, U. Nijmegen. TR 93–21.Google Scholar
  4. Cole, M. (1989), Algorithmic Skeletons: Structured Management of Parallel Computation, MIT Press.zbMATHGoogle Scholar
  5. Culler, D. E. et al. (1993), Parallel programming in Split-C, in ‘Supercomputing’.Google Scholar
  6. Darlington, J. et al. (1993), Parallel programming using skeleton functions, in A. Bode, M. Reeve & G. Wolf, eds, ‘PARLE ’83’, pp. 146–160.Google Scholar
  7. Erwin, E. & Miller, K. (1995), ‘Modeling joint development of ocular dominance and orientation in primary visual cortex’, Proc. of the CNS.Google Scholar
  8. Erwin, E., Obermayer, K. & Schulten, K. (1995), ‘Models of orientation and ocular dominance columns in the visual cortex: A critical comparison’, Neur. Comp. 7, 425–468.CrossRefGoogle Scholar
  9. Feather, M. S. (1987), A survey and classification of some program transformation approaches and techniques, in L. Meertens, ed., ‘Program Specification & Transformation’, North-Holland.Google Scholar
  10. Geerling, M. (1996), Transformational Development of Data-Parallel Algorithms, PhD thesis, Katholieke Universiteit Nijmegen.Google Scholar
  11. HPF Forum (1993), ‘High Performance Fortran’, Scientific Programming.Google Scholar
  12. Hubel, D. H. & Wiesel, T. N. (1962), ‘Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex’, Journal of Physiology London 160, 106–154.CrossRefGoogle Scholar
  13. Jay, C. & Cockett, J. (1994), Shapely types and shape polymorphism, in D. Sannella, ed., ‘Programming Languages and Systems — ESOP ’94’, LNCS, Springer Verlag, pp. 302–316.CrossRefGoogle Scholar
  14. Linsker, R. (1986), ‘From basic network principles to neural architecture: Emergence of orientation selective cells’, Proceedings of the National Academy of Science USA 83, 8390–8394.CrossRefGoogle Scholar
  15. Meertens, L. (1986), Algorithmics — towards programming as a mathematical activity, in ‘CWI Symposium on Mathematics and Computer Science’, North-Holland, pp. 289–334.Google Scholar
  16. Miller, K. (1994), ‘A model for the development of simple cell receptive fields and the ordered arrangements of orientation columns through activity-dependent competition between ON- and OFF-center inputs’, Journal of Neuroscience 14, 409–441.Google Scholar
  17. Obermayer, K., Blasdel, G. G. & Schulten, K. (1992), ‘A statistical mechanical analysis of self-organization and patte rn formation during the development of visual maps’, Phys. Rev. A 45, 7568–7589.CrossRefGoogle Scholar
  18. Pelagatti, S. (1993), A Methodology for the Development and the Support of Massively Parallel Programs, PhD thesis, Universita di Pisa-Genova-Udine.Google Scholar
  19. Pepper, P. (1993), Deductive derivation of parallel programs, in R. Paige, J. Reif & R. Wachter, eds, ‘Parallel Algorithm Derivation and Program Transformation’, Kluwer Academic, chapter 1, pp. 1–53. Also as TR 92–23, TU Berlin.CrossRefGoogle Scholar
  20. Pepper, P. & Möller, B. (1991), Programming with (finite) mappings, in M. Broy, ed., ‘Informatik and Mathematik’, Springer Verlag, pp. 381–405.CrossRefGoogle Scholar
  21. Pepper, P. & Südholt, M. (1997), Deriving parallel numerical algorithms using data distribution algebras: Wang’s algorithm, in ‘Proc. of the 30rd Hawaii International Conference on System Sciences’.Google Scholar
  22. Pepper, P., Exner, J. & Südholt, M. (1993), Functional development of massively parallel programs, in D. Bjorner et al., eds, ‘Formal Methods in Programming and Their Applications. LNCS 735’, Springer Verlag, pp. 217–238.CrossRefGoogle Scholar
  23. Piepenbrock, C., Ritter, H. & Obermayer, K. (1996), ‘Linear correlation-based learning models require a two-stage process for the development of orientation and ocular dominance’, Neural Processing Letters 3, 31–37.CrossRefGoogle Scholar
  24. Schmidt, H. W. (1992), Data-parallel object-oriented programming, in ‘Fifth Australian Supercomputer Conference, Melbourne’, 263–272.Google Scholar
  25. Skillicorn, D. B. (1992), The Bird-Meertens Formalism as a parallel model, in ‘NATO ARW “Software for Parallel Computation”’.Google Scholar
  26. Skillicorn, D. B. (1993), ‘Deriving parallel programs from specifications using cost information’, Science of Computer Programming.Google Scholar
  27. Tichy, W. F., Philippsen, M. & Hatcher, P. (1992), ‘A critique of the programming language C*’, Communications of the ACM 35 (6), 21–24.CrossRefGoogle Scholar
  28. TMC (1993), CMSSL for C*,Thinking Machines Corp.Google Scholar

Copyright information

© IFIP 1997

Authors and Affiliations

  • M. Südholt
    • 1
  • C. Piepenbrock
    • 2
  • K. Obermayer
    • 2
  • P. Pepper
    • 2
  1. 1.Projet Lande, INRIA/IRISA-RennesFrance
  2. 2.TU BerlinFachbereich Informatik, Institut für Kommunikations- und SoftwaretechnikGermany

Personalised recommendations