Abstract
The design and implementation of parallel algorithms for distributed memory architectures is much harder than the development of sequential algorithms. This is mainly due to the communication and synchronization that is necessary to manage distributed data correctly. This paper applies a methodology for the transformational derivation of parallel programs using data distribution algebras that enable an abstract description of data distribution issues. Algorithms are formulated using skeletons, that is, specialized higher-order functions with particular parallel implementations. The methodology is applied to a the solution of a system of ordinary differential equations where convolutions can be computed using the Fast Fourier transformation. The example illustrates the practical optimization problems for a development model of the visual system that involves large scale neural network simulations. Finally, this algorithm is compared to an implementation of the same system of equations in the programming language C* on a CM-5.
Chapter PDF
Similar content being viewed by others
Keywords
References
Bauer, F. et al. (1985), The Munich Project CIP. Vol. I: The Wide Spectrum Language CIP-L, number 183 in ‘LNCS’, Springer Verlag, Berlin.
Bird, R. S. (1989), Lectures on constructive functional programming, in M. Broy, ed., ‘Constructive Methods in Computing Science’, Vol. 55 of NATO ASI, F, Springer, pp. 151–216.
Bratvold, T. A. (1993), A skeleton-based parallelising compiler for ML, in R. Plasmeijer & M. van Eekelen, eds, ‘Implementation of Functional Languages’, U. Nijmegen. TR 93–21.
Cole, M. (1989), Algorithmic Skeletons: Structured Management of Parallel Computation, MIT Press.
Culler, D. E. et al. (1993), Parallel programming in Split-C, in ‘Supercomputing’.
Darlington, J. et al. (1993), Parallel programming using skeleton functions, in A. Bode, M. Reeve & G. Wolf, eds, ‘PARLE ’83’, pp. 146–160.
Erwin, E. & Miller, K. (1995), ‘Modeling joint development of ocular dominance and orientation in primary visual cortex’, Proc. of the CNS.
Erwin, E., Obermayer, K. & Schulten, K. (1995), ‘Models of orientation and ocular dominance columns in the visual cortex: A critical comparison’, Neur. Comp. 7, 425–468.
Feather, M. S. (1987), A survey and classification of some program transformation approaches and techniques, in L. Meertens, ed., ‘Program Specification & Transformation’, North-Holland.
Geerling, M. (1996), Transformational Development of Data-Parallel Algorithms, PhD thesis, Katholieke Universiteit Nijmegen.
HPF Forum (1993), ‘High Performance Fortran’, Scientific Programming.
Hubel, D. H. & Wiesel, T. N. (1962), ‘Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex’, Journal of Physiology London 160, 106–154.
Jay, C. & Cockett, J. (1994), Shapely types and shape polymorphism, in D. Sannella, ed., ‘Programming Languages and Systems — ESOP ’94’, LNCS, Springer Verlag, pp. 302–316.
Linsker, R. (1986), ‘From basic network principles to neural architecture: Emergence of orientation selective cells’, Proceedings of the National Academy of Science USA 83, 8390–8394.
Meertens, L. (1986), Algorithmics — towards programming as a mathematical activity, in ‘CWI Symposium on Mathematics and Computer Science’, North-Holland, pp. 289–334.
Miller, K. (1994), ‘A model for the development of simple cell receptive fields and the ordered arrangements of orientation columns through activity-dependent competition between ON- and OFF-center inputs’, Journal of Neuroscience 14, 409–441.
Obermayer, K., Blasdel, G. G. & Schulten, K. (1992), ‘A statistical mechanical analysis of self-organization and patte rn formation during the development of visual maps’, Phys. Rev. A 45, 7568–7589.
Pelagatti, S. (1993), A Methodology for the Development and the Support of Massively Parallel Programs, PhD thesis, Universita di Pisa-Genova-Udine.
Pepper, P. (1993), Deductive derivation of parallel programs, in R. Paige, J. Reif & R. Wachter, eds, ‘Parallel Algorithm Derivation and Program Transformation’, Kluwer Academic, chapter 1, pp. 1–53. Also as TR 92–23, TU Berlin.
Pepper, P. & Möller, B. (1991), Programming with (finite) mappings, in M. Broy, ed., ‘Informatik and Mathematik’, Springer Verlag, pp. 381–405.
Pepper, P. & Südholt, M. (1997), Deriving parallel numerical algorithms using data distribution algebras: Wang’s algorithm, in ‘Proc. of the 30rd Hawaii International Conference on System Sciences’.
Pepper, P., Exner, J. & Südholt, M. (1993), Functional development of massively parallel programs, in D. Bjorner et al., eds, ‘Formal Methods in Programming and Their Applications. LNCS 735’, Springer Verlag, pp. 217–238.
Piepenbrock, C., Ritter, H. & Obermayer, K. (1996), ‘Linear correlation-based learning models require a two-stage process for the development of orientation and ocular dominance’, Neural Processing Letters 3, 31–37.
Schmidt, H. W. (1992), Data-parallel object-oriented programming, in ‘Fifth Australian Supercomputer Conference, Melbourne’, 263–272.
Skillicorn, D. B. (1992), The Bird-Meertens Formalism as a parallel model, in ‘NATO ARW “Software for Parallel Computation”’.
Skillicorn, D. B. (1993), ‘Deriving parallel programs from specifications using cost information’, Science of Computer Programming.
Tichy, W. F., Philippsen, M. & Hatcher, P. (1992), ‘A critique of the programming language C*’, Communications of the ACM 35 (6), 21–24.
TMC (1993), CMSSL for C*,Thinking Machines Corp.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1997 IFIP
About this chapter
Cite this chapter
Südholt, M., Piepenbrock, C., Obermayer, K., Pepper, P. (1997). Solving Large Systems of Differential Equations in Parallel Using Covers and Skeletons. In: Bird, R.S., Meertens, L. (eds) Algorithmic Languages and Calculi. IFIP Advances in Information and Communication Technology. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-35264-0_6
Download citation
DOI: https://doi.org/10.1007/978-0-387-35264-0_6
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-5041-2877-3
Online ISBN: 978-0-387-35264-0
eBook Packages: Springer Book Archive