Abstract
This paper addresses the problem of scheduling jobs in soft real-time systems, where the utility of completing each job decreases over time. We present a utility-based framework for making repeated scheduling decisions based on dynamically observed information about unscheduled jobs and system’s resources. This framework generalizes the standard scheduling problem to a resource-constrained environment, where resource allocation (RA) decisions (how many CPUs to allocate to each job) have to be made concurrently with the scheduling decisions (when to execute each job). Discrete-time Optimal Control theory is used to formulate the optimization problem of finding the scheduling/RA policy that maximizes the average utility per time step obtained from completed jobs. We propose a Reinforcement Learning (RL) architecture for solving the NP-hard Optimal Control problem in real time, and our experimental results demonstrate the feasibility and benefits of the proposed approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bertsekas, D., Tsitsiklis, J.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996)
Clark, R.K.: Scheduling Dependent Real-Time Activities. Ph.D. Dissertation, Carnegie Mellon University, CMU-CS-90-155 (1990)
Jensen, E.D., Locke, C.D., Tokuda, H.: A time driven scheduling model for real-time operating systems. In: Proceedings IEEE Real-Time Systems Symposium, pp. 112–122 (1985)
Kepner, J.: HPC Productivity Model Synthesis. In: Kepner, J. (ed.) International Journal of High Performance Computing Applications: Special Issue on HPC Productivity, Winter, vol. 18(4) (November 2004)
Li, P., Ravindran, B., Wu, H., Jensen, E.D.: A Utility Accrual Scheduling Algorithm for Real-Time Activities With Mutual Exclusion Resource Constraints. The MITRE Corporation Working Paper (April 2004)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Tsitsiklis, J.N., Van Roy, B.: Average Cost Temporal-Difference Learning. Automatica 35(11), 1799–1808 (1999)
Wang, L.-X.: Fuzzy systems are universal approximators. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 1992), pp. 1163–1169 (1992)
Wu, H., Ravindran, B., Jensen, E.D., Balli, U.: Utility Accrual Scheduling Under Arbitrary Time/Utility Functions and Multiunit Resource Constraints. In: IEEE Real-Time and Embedded Computing Systems and Applications (August 2004) (to appear)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vengerov, D. (2005). Adaptive Utility-Based Scheduling in Resource-Constrained Systems. In: Zhang, S., Jarvis, R. (eds) AI 2005: Advances in Artificial Intelligence. AI 2005. Lecture Notes in Computer Science(), vol 3809. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11589990_50
Download citation
DOI: https://doi.org/10.1007/11589990_50
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30462-3
Online ISBN: 978-3-540-31652-7
eBook Packages: Computer ScienceComputer Science (R0)