Advertisement

Computation Scheduling and Data Replication Algorithms for Data Grids

  • Kavitha Ranganathan
  • Ian Foster
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 64)

Abstract

Data Grids seek to harness geographically distributed resources for large-scale data-intensive problems such as those encountered in high energy physics, bioinformatics, and other disciplines. These problems typically involve numerous, loosely coupled jobs that both access and generate large data sets. Effective scheduling in such environments is challenging, because of a need to address a variety of metrics and constraints (e.g., resource utilization, response time, global and local allocation policies) while dealing with multiple, potentially independent sources of jobs and a large number of storage, compute, and network resources.

We describe a scheduling framework that addresses these problems. Within this framework, data movement operations may be either tightly bound to job scheduling decisions or performed by a decoupled, asynchronous process on the basis of observed data access patterns and load. We develop a family of job scheduling and data movement (replication) algorithms and use simulation studies to evaluate various combinations. Our results suggest that while it is necessary to consider the impact of replication on the scheduling strategy, it is not always necessary to couple data movement and computation scheduling. Instead, these two activities can be addressed separately, thus significantly simplifying the design and implementation of the overall Data Grid system.

Keywords

Schedule Algorithm Data Grid Replication Strategy Data Replication Local Scheduler 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media New York 2004

Authors and Affiliations

  • Kavitha Ranganathan
    • 1
  • Ian Foster
    • 1
    • 2
  1. 1.Department of Computer ScienceThe University of ChicagoUSA
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryUSA

Personalised recommendations