Abstract
We present a system that allows task parallel OpenMP pro- grams to execute on a network of workstations (NOW) with a variable number of nodes. Such adaptivity, generally called adaptive parallelism, is important in a multi-user NOW environment, enabling the system to expand the computation onto idle nodes or withdraw from otherwise occupied nodes.
We focus on task parallel applications in this paper, but the system also lets data parallel applications run adaptively.
When an adaptation is requested, we let all processes complete their current tasks, then the system executes an extra OpenMP join-fork se- quence not present in the application code. Here, the system can change the number of nodes without involving the application, as processes do not have a compute-relevant private process state.
We show that the costs of adaptations is low, and we explain why the costs are lower for task parallel applications than for data parallel appli- cations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
S. P. Amarasinghe, J. M. Anderson, M. S. Lam, and C. W. Tseng. An overview of the suif compiler for scalable parallel machines. In Proceedings of the 7th SIAM Conference on Parallel Processing for Scientific Computing, pages 662–667, San Francisco, February 1995.
C. Amza, A.L. Cox, S. Dwarkadas, P. Keleher, H. Lu, R. Rajamony, W. Yu, and W. Zwaenepoel. Treadmarks: Shared memory computing on networks of workstations. IEEE Computer, 29(2):18–28, February 1996.
J. Arabe, A. Beguelin, B. Lowekamp, E. Seligman, M. Starkey, and P.. Stephan. Dome: Parallel programming in a heterogeneous multi-user environment. Technical Report CMU-CS-95-137, Computer Science Department, Carnegie Mellon University, April 1995.
R.D. Blumofe and P.A. Lisiecki. Adaptive and reliable parallel computing on network of workstations. In Proceedings of the USENIX 1997 Annual Technical Symposium, pages 133–147, January 1997.
N. Carriero, E. Freeman, D. Gelernter, and D. Kaminsky. Adaptive parallelism and piranha. IEEE Computer, 28(1):40–49, January 1995.
G. Edjlali, G. Agrawal, A. Sussman, J. Humphries, and J. Saltz. Compiler and runtime support for programming in adaptive parallel environments. Scientific Programming, 6(2):215–227, January 1997.
P. J. Hatcher and M. J. Quinn. Data-parallel Programming on MIMD Computers. The MIT Press, Cambridge MA, 1991.
L. V. Kalé, B. Ramkumar, A. B. Sinha, and A. Gursoy. The CHARM Parallel Programming Language and System: Part I-Description of Language Features. IEEE Transactions on Parallel and Distributed Systems, 1994.
L. V. Kalé, B. Ramkumar, A. B. Sinha, and V. A. Saletore. The CHARM Parallel Programming Language and System: Part II-The Runtime System. IEEE Transactions on Parallel and Distributed Systems, 1994.
R. Konuru, S. Otto, and J. Walpole. A migratable user-level process package for pvm. Journal of Parallel and Distributed Computing, 40(1):81–102, Jan 1997.
H. Lu, Y. C. Hu, and W. Zwaenepoel. OpenMP on networks of workstations. In Proc. Supercomputing ’98, Orlando, FL, November 1998. ACM/IEEE.
J. E. Moreira, V. K. Naik, and R. B. Konuru. A system for dynamic resource allocation and data distribution. Technical Report RC 20257, IBM Research Division, October 1995.
N. Nedeljkovic and M.J. Quinn. Data-parallel programming on a network of heterogeneous workstations. Concurrency: Practice & Experience, 5(4):257–268, June 1993.
A. Scherer, H. Lu, T. Gross, and W. Zwaenepoel. Transparent Adaptive Parallelism on NOWs using OpenMP. In Proceedings of the Seventh ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPOPP), Atlanta, May 1999. ACM.
S. Shah, G. Haab, P. Petersen, and J. Throop. Flexible Control Structures for Parallelism in OpenMP. In First EuropeanWorkshop on OpenMP (EWOMP ’99), Lund, Sweden, September/October 1999. Kuck & Associates, Incorporated.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Scherer, A., Gross, T., Zwaenepoel, W. (2000). Adaptive Parallelism for OpenMP Task Parallel Programs. In: Dwarkadas, S. (eds) Languages, Compilers, and Run-Time Systems for Scalable Computers. LCR 2000. Lecture Notes in Computer Science, vol 1915. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40889-4_9
Download citation
DOI: https://doi.org/10.1007/3-540-40889-4_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41185-7
Online ISBN: 978-3-540-40889-5
eBook Packages: Springer Book Archive