Abstract
Scientific applications are more and more faced with very large volumes of data and complex, resource-intensive workflows that process or analyze these data. The recent interest in web services and service-oriented architectures has strongly facilitated the development of individual workflow activities as well as their composition and the distributed execution of complete workflows. However, in many applications concurrent scientific workflows may be served by multiple competing providers, with each of them offering only limited resources. At the same time, these workflows need to be executed in a predictable manner, with dedicated Quality of Service guarantees. In this paper, we introduce an approach to Advance Resource Reservation for service-oriented complex scientific workflows that optimize resource consumption based on user-defined criteria (e.g., cost or time). It exploits optimization techniques using genetic algorithms for finding optimal or near-optimal allocations in a distributed system. The approach takes into account the locality of services and in particular enforces constraints imposed by control or data flow dependencies within workflows. Finally, we provide a comprehensive evaluation of the effectiveness of the proposed approach.
This work has been partly supported by the Hasler Foundation within the project COSA (Compiling Optimized Service Architectures).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Affenzeller, M., Winkler, S., Wagner, S., Beham, A.: Genetic Algorithms and Genetic Programming: Modern Concepts and Practical Applications (Numerical Insights). Chapman & Hall, Boca Raton (2009)
Canfora, G., Di Penta, M., Esposito, R., Villani, M.L.: An approach for QoS-aware service composition based on genetic algorithms. In: GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation (2005)
Cardoso, J., Sheth, A., Miller, J., et al.: Quality of Service for Workflows and Web Service Processes. J. Web Sem. 1(3), 281–308 (2004)
Collette, Y., Siarry, P.: Multiobjective Optimization: Principles and Case Studies (Decision Engineering). Springer, Heidelberg (2004)
Droegemeier, K., et al.: Service-Oriented Environments for Dynamically Interacting with Mesoscale Weather. Computing in Science and Eng. 7(6), 12–29 (2005)
Foster, I., Kesselman, C., Lee, C., et al.: A Distributed Resource Management Architecture that supports Advance Reservations and Co-Allocation. In: Proceedings of the International Workshop on Quality of Service, pp. 27–36 (1999)
Freire, J., Silva, C., Callahan, S., et al.: Managing Rapidly-Evolving Scientific Workflows. In: Int’l Provenance and Annotation Workshop, pp. 10–18 (2006)
Gao, Y., Rong, H., Huang, J.Z.: Adaptive grid job scheduling with genetic algorithms. Future Gener. Comput. Syst. 21(1), 151–161 (2005)
Guo, L., McGough, A., Akram, A., et al.: QoS for Service Based Workflow on Grid. In: UK 2007 e-Science All Hands Meeting (August 2007)
Hou, E.S.H., Ansari, N., Ren, H.: A genetic algorithm for multiprocessor scheduling. IEEE Trans. Parallel Distrib. Syst. 5(2), 113–120 (1994)
Langguth, C., Ranaldi, P., Schuldt, H.: Towards Quality of Service in Scientific Workflows by using Advance Resource Reservations. In: IEEE 2009 Third International Workshop on Scientific Workflows, SWF 2009 (2009)
Lin, C., Lu, S., Lai, Z., et al.: Service-Oriented Architecture for VIEW: A Visual Scientific Workflow Management System. In: IEEE SCC, pp. 335–342 (2008)
Ludäscher, B., Altintas, I., Berkley, C., et al.: Scientific workflow management and the Kepler system. In: Concurrency and Computation: Practice and Experience (2006)
Meffert, K., et al.: Java Genetic Algorithms Package, http://jgap.sourceforge.net
Oinn, T., Greenwood, R., Addis, M., et al.: Taverna: Lessons in Creating a Workflow Environment for the Life Sciences. Concurrency and Computation: Practice and Experience 18(10), 1067–1100 (2006)
Pautasso, C., Heinis, T., Alonso, G.: JOpera: Autonomic Service Orchestration. IEEE Data Eng. Bull. 29(3), 32–39 (2006)
Plale, B.: Workload Characterization and Analysis of Storage and Bandwidth Needs of LEAD Workspace. In: Linked Environments for Atmospheric Discovery (2007)
Prodan, R., Fahringer, T.: Dynamic scheduling of scientific workflow applications on the grid: a case study. In: SAC 2005: Proceedings of the 2005 ACM symposium on Applied computing, pp. 687–694. ACM, New York (2005)
RogerBarga, Fay, D., Guo, D., et al.: Efficient Scheduling of Scientific Workflows in a High Performance Computing Cluster. In: Proc. CLADE, pp. 63–68 (2008)
Siddiqui, M., Villazón, A., Fahringer, T.: Grid Allocation and Reservation - Grid Capacity Planning with Negotiation-based Advance Reservation for Optimized QoS. In: Supercomputing, p. 103 (2006)
Simmhan, Y.L., Plale, B., Gannon, D.: A Framework for Collecting Provenance in Data-Centric Scientific Workflows. In: ICWS 2006: Proceedings of the IEEE International Conference on Web Services (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Langguth, C., Schuldt, H. (2010). Optimizing Resource Allocation for Scientific Workflows Using Advance Reservations. In: Gertz, M., Ludäscher, B. (eds) Scientific and Statistical Database Management. SSDBM 2010. Lecture Notes in Computer Science, vol 6187. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13818-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-13818-8_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13817-1
Online ISBN: 978-3-642-13818-8
eBook Packages: Computer ScienceComputer Science (R0)