Summary
We address the problem of optimizing the flow of compute jobs through a distributed system of compute servers. The goal is to determine the best policy for how to route jobs to different compute clusters as well as to decide which jobs to backlog until a future time. We use an approach that is a hybrid of dynamic programming and a genetic algorithm. Dynamic programming determines the routing and backlog decisions about individual flows of homogeneous jobs, while a genetic algorithm optimizes the order in which the different flows are fed to the dynamic programming algorithm. We demonstrate the effectiveness of this approach on sample problems, some designed to yield a known correct answer and others designed to test the scaling.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Andresen, D., McCune, T.: Towards a Hierarchical Scheduling System for Distributed WWW Server Clusters. In: Proceedings of the The Seventh IEEE International Symposium on High Performance Distributed Computing (1998)
Barolli, L., Koyama, A., Matsumoto, K., Suganuma, T., Shiratori, N.: A Genetic Algorithm Based Routing Method Using Two QoS Parameters. In: Proceedings of the 13th International Workshop on Database and Expert Systems Applications (2002)
Bose, A., Wickman, B., Wood, C.: MARS: A Metascheduler for Distributed Resources in Campus Grids. In: Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (2004)
Casetti, C., Cigno, R., Mellia, M.: QoS-Aware Routing Schemes Based on Hierarchical LoadBalancing for Integrated Services Packet Networks. In: Proceedings of the IEEE International Communication Conference (1999)
Chen, S., Smith, S.: Improving Genetic Algorithms by Search Space Reduction (with Applications to Flow Shop Scheduling). In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 135–140 (1999)
Goldberg, D., Lingle, J.R.: Alleles, Loci, and the Traveling Salesman Problem. In: Proceedings of the First International Conference on Genetic Algorithms, pp. 154–159 (1985)
Goswami, K., Devarakonda, M., Iyer, R.: Prediction-Based Dynamic Load-Sharing Heuristics. IEEE Transactions on Parallel and Distributed Systems (1993)
Grefenstette, J., Gopal, R., Rosmaita, B., van Gucht, D.: Genetic Algorithms for the Traveling Salesman Problem. In: Proceedings of the First International Conference on Genetic Algorithms, pp. 160–165 (1985)
Grimme, C.: Grid Metaschedulers: An Overview and Up-to-date Solutions. PowerPoint presentation (2007)
Key, P., Massoullie, L.: Fluid Models of Integrated Traffic and Multipath Routing. Queueing Systems: Theory and Applications 53(1-2), 85–98 (2006)
Lo, V., Zhou, D., Zappala, D., Liu, Y., Zhao, S.: Cluster Computing on the Fly: P2P Scheduling of Idle Cycles in the Internet. In: International Workshop on Peer-to-Peer Systems (2004)
Mausolf, J.: Grid in Action: Managing the Resource Managers. IBM developerWorks (2005)
Okuhara, K., Tanaka, T., Ishii, H.: Routing and Flow Control by Genetic Algorithm for a Flow Model. Systems and Computers in Japan 34(1), 11–20 (2003)
Othman, O., Schmidt, D.: Issues in the Design of Adaptive Middleware Load Balancing. In: Proceedings of the ACM SIGPLAN Workshop on Optimization of Middleware and Distributed Systems, pp. 205–213 (2001)
Oueslati, S., Roberts, J.: Comparing Flow-Aware and Flow-Oblivious Adaptive Routing. In: 40th Annual Conference on Information Sciences and Systems, pp. 655–660 (2006)
Stone, H.: Multiprocessor Scheduling with the Aid of Network Flow Algorithms. IEEE Transactions on Software Engineering SE-3(1), 85–93 (1977)
Strong, P.: Enterprise Grid Computing. ACM Queue 3(6) (2005)
Syswerda, G.: Schedule Optimization Using Genetic Algorithms. In: Davis, L. (ed.) Handbook of Genetic Algorithms, pp. 332–349. Van Nostrand, Reinhold (1991)
Thain, D., Tannenbaum, T., Livny, M.: Distributed Computing in Practice: The Condor Experience. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2005)
Vadhiyar, S., Dongarra, J.: A Metascheduler for the Grid. In: Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing (2002)
Vazquez, M., Whitley, D.: A Comparison of Genetic Algorithms for the Static Job Shop Scheduling Problem. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 303–312. Springer, Heidelberg (2000)
Whitley, D., Starkweather, T., Fuquay, D.: Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator. In: Proceedings of the Third International Conference on Genetic Algorithms, pp. 133–140 (1989)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Montana, D., Zinky, J. (2008). Optimizing Routing and Backlogs for Job Flows in a Distributed Computing Environment. In: Xhafa, F., Abraham, A. (eds) Metaheuristics for Scheduling in Distributed Computing Environments. Studies in Computational Intelligence, vol 146. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69277-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-69277-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69260-7
Online ISBN: 978-3-540-69277-5
eBook Packages: EngineeringEngineering (R0)