Faster Model-Based Optimization Through Resource-Aware Scheduling Strategies

  • Jakob RichterEmail author
  • Helena KotthausEmail author
  • Bernd Bischl
  • Peter Marwedel
  • Jörg Rahnenführer
  • Michel Lang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10079)


We present a Resource-Aware Model-Based Optimization framework RAMBO that leads to efficient utilization of parallel computer architectures through resource-aware scheduling strategies. Conventional MBO fits a regression model on the set of already evaluated configurations and their observed performances to guide the search. Due to its inherent sequential nature, an efficient parallel variant can not directly be derived, as only the most promising configuration w.r.t. an infill criterion is evaluated in each iteration. This issue has been addressed by generalized infill criteria in order to propose multiple points simultaneously for parallel execution in each sequential step. However, these extensions in general neglect systematic runtime differences in the configuration space which often leads to underutilized systems. We estimate runtimes using an additional surrogate model to improve the scheduling and demonstrate that our framework approach already yields improved resource utilization on two exemplary classification tasks.


Black-box optimization Hyperparameter tuning Model selection Model-based optimization Resource-aware scheduling Performance management Parallelization 



This work was partly supported by Deutsche Forschungsgemeinschaft (DFG) within the Collaborative Research Center SFB 876, A3.


  1. 1.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Bischl, B., Lang, M., Mersmann, O., Rahnenführer, J., Weihs, C.: BatchJobs and BatchExperiments: abstraction mechanisms for using R in batch environments. J. Stat. Comput. Simul. 64(11), 1–25 (2015)Google Scholar
  3. 3.
    Bischl, B., Wessing, S., Bauer, N., Friedrichs, K., Weihs, C.: MOI-MBO: multiobjective infill for parallel model-based optimization. In: Pardalos, P.M., Resende, M.G.C., Vogiatzis, C., Walteros, J.L. (eds.) LION 2014. LNCS, vol. 8426, pp. 173–186. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-09584-4_17 Google Scholar
  4. 4.
    Delimitrou, C., Kozyrakis, C.: Quasar: resource-efficient and QoS-aware Cluster Management. In: ASPLOS 2014, pp. 127–144. ACM (2014)Google Scholar
  5. 5.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Parallel algorithm configuration. In: Hamadi, Y., Schoenauer, M. (eds.) LION 6. LNCS, vol. 7219, pp. 55–70. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-34413-8_5 CrossRefGoogle Scholar
  6. 6.
    Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Karatzoglou, A., Meyer, D., Hornik, K.: Support vector machines in R. J. Stat. Softw. 15(1), 1–28 (2006)Google Scholar
  8. 8.
    Kotthaus, H., Korb, I., Lang, M., Bischl, B., Rahnenführer, J., Marwedel, P.: Runtime and memory consumption analyses for machine learning R programs. J. Stat. Comput. Simul. 85(1), 14–29 (2015)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Kotthaus, H., Korb, I., Marwedel, P.: Performance analysis for parallel R programs: towards efficient resource utilization. Technical report 01/2015, Department of Computer Science 12, TU Dortmund University (2015). SFB876 Project A3Google Scholar
  10. 10.
    Lang, M., Kotthaus, H., Marwedel, P., Weihs, C., Rahnenführer, J., Bischl, B.: Automatic model selection for high-dimensional survival analysis. J. Stat. Comput. Simul. 85(1), 62–76 (2015)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: NIPS Workshop on Bayesian Optimization, Sequential Experimental Design, and Bandits, pp. 2960–2968 (2012)Google Scholar
  12. 12.
    Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of ACM SIGKDD, pp. 847–855 (2013)Google Scholar
  13. 13.
    Tillenius, M., Larsson, E., Badia, R.M., Martorell, X.: Resource-aware task scheduling. ACM Trans. Embed. Comput. Syst. 14(1), 5:1–5:25 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Jakob Richter
    • 1
    Email author
  • Helena Kotthaus
    • 2
    Email author
  • Bernd Bischl
    • 3
  • Peter Marwedel
    • 2
  • Jörg Rahnenführer
    • 1
  • Michel Lang
    • 1
  1. 1.Department of StatisticsTU Dortmund UniversityDortmundGermany
  2. 2.Department of Computer Science 12TU Dortmund UniversityDortmundGermany
  3. 3.Department of StatisticsLMU MünchenMunichGermany

Personalised recommendations