Faster Model-Based Optimization Through Resource-Aware Scheduling Strategies
We present a Resource-Aware Model-Based Optimization framework RAMBO that leads to efficient utilization of parallel computer architectures through resource-aware scheduling strategies. Conventional MBO fits a regression model on the set of already evaluated configurations and their observed performances to guide the search. Due to its inherent sequential nature, an efficient parallel variant can not directly be derived, as only the most promising configuration w.r.t. an infill criterion is evaluated in each iteration. This issue has been addressed by generalized infill criteria in order to propose multiple points simultaneously for parallel execution in each sequential step. However, these extensions in general neglect systematic runtime differences in the configuration space which often leads to underutilized systems. We estimate runtimes using an additional surrogate model to improve the scheduling and demonstrate that our framework approach already yields improved resource utilization on two exemplary classification tasks.
KeywordsBlack-box optimization Hyperparameter tuning Model selection Model-based optimization Resource-aware scheduling Performance management Parallelization
This work was partly supported by Deutsche Forschungsgemeinschaft (DFG) within the Collaborative Research Center SFB 876, A3.
- 2.Bischl, B., Lang, M., Mersmann, O., Rahnenführer, J., Weihs, C.: BatchJobs and BatchExperiments: abstraction mechanisms for using R in batch environments. J. Stat. Comput. Simul. 64(11), 1–25 (2015)Google Scholar
- 3.Bischl, B., Wessing, S., Bauer, N., Friedrichs, K., Weihs, C.: MOI-MBO: multiobjective infill for parallel model-based optimization. In: Pardalos, P.M., Resende, M.G.C., Vogiatzis, C., Walteros, J.L. (eds.) LION 2014. LNCS, vol. 8426, pp. 173–186. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-09584-4_17 Google Scholar
- 4.Delimitrou, C., Kozyrakis, C.: Quasar: resource-efficient and QoS-aware Cluster Management. In: ASPLOS 2014, pp. 127–144. ACM (2014)Google Scholar
- 7.Karatzoglou, A., Meyer, D., Hornik, K.: Support vector machines in R. J. Stat. Softw. 15(1), 1–28 (2006)Google Scholar
- 9.Kotthaus, H., Korb, I., Marwedel, P.: Performance analysis for parallel R programs: towards efficient resource utilization. Technical report 01/2015, Department of Computer Science 12, TU Dortmund University (2015). SFB876 Project A3Google Scholar
- 11.Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: NIPS Workshop on Bayesian Optimization, Sequential Experimental Design, and Bandits, pp. 2960–2968 (2012)Google Scholar
- 12.Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of ACM SIGKDD, pp. 847–855 (2013)Google Scholar