Skip to main content
SpringerLink
Log in
Book cover

International Conference on Learning and Intelligent Optimization

LION 2016: Learning and Intelligent Optimization pp 91–106Cite as

Designing and Comparing Multiple Portfolios of Parameter Configurations for Online Algorithm Selection

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10079))

Abstract

Algorithm portfolios seek to determine an effective set of algorithms that can be used within an algorithm selection framework to solve problems. A limited number of these portfolio studies focus on generating different versions of a target algorithm using different parameter configurations. In this paper, we employ a Design of Experiments (DOE) approach to determine a promising range of values for each parameter of an algorithm. These ranges are further processed to determine a portfolio of parameter configurations, which would be used within two online Algorithm Selection approaches for solving different instances of a given combinatorial optimization problem effectively. We apply our approach on a Simulated Annealing-Tabu Search (SA-TS) hybrid algorithm for solving the Quadratic Assignment Problem (QAP) as well as an Iterated Local Search (ILS) on the Travelling Salesman Problem (TSP). We also generate a portfolio of parameter configurations using best-of-breed parameter tuning approaches directly for the comparison purpose. Experimental results show that our approach lead to improvements over best-of-breed parameter tuning approaches.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Rice, J.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  2. Gomes, C., Selman, B.: Algorithm portfolios. AI 126(1), 43–62 (2001)

    MathSciNet  MATH  Google Scholar 

  3. Huberman, B., Lukose, R., Hogg, T.: An economics approach to hard computational problems. Science 275(5296), 51 (1997)

    Article  Google Scholar 

  4. Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 1–25 (2008)

    Article  Google Scholar 

  5. Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. JAIR 36(1), 267–306 (2009)

    MATH  Google Scholar 

  6. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR 13, 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  7. Eiben, A., Michalewicz, Z., Schoenauer, M., Smith, J.: Parameter control in evolutionary algorithms. In: Lobo, F.G., Lima, C.F., Michalewicz, Z. (eds.) Parameter Setting in Evolutionary Algorithms. SCI, vol. 54, pp. 19–46. Springer, Heidelberg (2007). doi:10.1007/978-3-540-69432-8_2

    Chapter  Google Scholar 

  8. Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. JAIR 32(1), 565–606 (2008)

    MATH  Google Scholar 

  9. Xu, L., Hoos, H.H., Leyton-Brown, K.: Hydra: automatically configuring algorithms for portfolio-based selection. In: AAAI 2010, pp. 210–216 (2010)

    Google Scholar 

  10. Adenso-Diaz, B., Laguna, M.: Fine-tuning of algorithms using fractional experimental designs and local search. OR 54(1), 99–114 (2006)

    Article  MATH  Google Scholar 

  11. Birattari, M., Stützle, T., Paquete, L., Varrentrapp, K.: A racing algorithm for configuring metaheuristics. In: GECCO 2002, pp. 11–18 (2002)

    Google Scholar 

  12. Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC–instance-specific algorithm configuration. In: ECAI 2010, pp. 751–756 (2010)

    Google Scholar 

  13. Lau, H.C., Xiao, F., Halim, S.: A framework for automated parameter tuning in heuristic design. In: MIC 2009, Hamburg, Germany, 13–16 June 2009

    Google Scholar 

  14. Seipp, J., Braun, M., Garimort, J., Helmert, M.: Learning portfolios of automatically tuned planners. In: ICAPS 2012, Atibaia/Sao Paulo, Brazil (2012)

    Google Scholar 

  15. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25566-3_40

    Chapter  Google Scholar 

  16. Lindauer, M., Hoos, H.H., Hutter, F., Schaub, T.: AutoFolio: an automatically configured algorithm selector. J. Artif. Intell. Res. 53, 745–778 (2015)

    MathSciNet  Google Scholar 

  17. Mısır, M., Handoko, S.D., Lau, H.C.: ADVISER: a web-based algorithm portfolio deviser. In: Dhaenens, C., Jourdan, L., Marmion, M.-E. (eds.) LION 2015. LNCS, vol. 8994, pp. 23–28. Springer, Heidelberg (2015). doi:10.1007/978-3-319-19084-6_3

    Chapter  Google Scholar 

  18. Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee, J. (ed.) CP 2011. LNCS, vol. 6876, pp. 454–469. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23786-7_35

    Chapter  Google Scholar 

  19. Hutter, F., Hoos, H.H., Stutzle, T.: Automatic algorithm configuration based on local search. In: AAAI 2007, vol. 22, p. 1152 (2007)

    Google Scholar 

  20. Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 142–157. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04244-7_14

    Chapter  Google Scholar 

  21. KhudaBukhsh, A.R., Xu, L., Hoos, H.H., Leyton-Brown, K.: SATenstein: automatically building local search sat solvers from components. In: IJCAI 2009, pp. 517–524 (2009)

    Google Scholar 

  22. Montgomery, D.: Design and Analysis of Expeirments, 6th edn. Wiley, Hoboken (2005)

    Google Scholar 

  23. Gunawan, A., Lau, H.C., Lindawati, : Fine-tuning algorithm parameters using the design of experiments approach. In: Coello Coello, A. (ed.) LION 2011. LNCS, vol. 6683, pp. 278–292. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25566-3

    Chapter  Google Scholar 

  24. Mısır, M., Verbeeck, K., De Causmaecker, P., Vanden Berghe, G.: An investigation on the generality level of selection hyper-heuristics under different empirical conditions. Appl. Soft Comput. 13(7), 3335–3353 (2013)

    Article  Google Scholar 

  25. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)

    Article  MATH  Google Scholar 

  26. Mısır, M., Handoko, S.D., Lau, H.C.: OSCAR: online selection of algorithm portfolios with case study on memetic algorithms. In: Dhaenens, C., Jourdan, L., Marmion, M.-E. (eds.) LION 2015. LNCS, vol. 8994, pp. 59–73. Springer, Heidelberg (2015). doi:10.1007/978-3-319-19084-6_6

    Chapter  Google Scholar 

  27. Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Tell me who can learn you and I can tell you who you are: landmarking various learning algorithms. In: Proceedings of the 17th International Conference on Machine Learning, pp. 743–750 (2000)

    Google Scholar 

  28. Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997)

    Article  Google Scholar 

  29. He, J., He, F., Dong, H.: Pure strategy or mixed strategy? In: Hao, J.-K., Middendorf, M. (eds.) EvoCOP 2012. LNCS, vol. 7245, pp. 218–229. Springer, Heidelberg (2012). doi:10.1007/978-3-642-29124-1_19

    Chapter  Google Scholar 

  30. Lehre, P.K., Özcan, E.: A runtime analysis of simple hyper-heuristics: to mix or not to mix operators. In: FOGA 2013, Adelaide, Australia (2013)

    Google Scholar 

  31. Cowling, P., Kendall, G., Soubeiga, E.: A hyperheuristic approach to scheduling a sales summit. In: Burke, E., Erben, W. (eds.) PATAT 2000. LNCS, vol. 2079, pp. 176–190. Springer, Heidelberg (2001). doi:10.1007/3-540-44629-X_11

    Chapter  Google Scholar 

  32. Mısır, M., Wauters, T., Verbeeck, K., Vanden Berghe, G.: A new learning hyper-heuristic for the traveling tournament problem. In: Caserta, M., Voß, S. (eds.) Metaheuristics: Intelligent Problem Solving - MIC 2009. Springer, Heidelberg (2010)

    Google Scholar 

  33. Thathachar, M., Sastry, P.: Networks of Learning Automata: Techniques for Online Stochastic Optimization. Kluwer Academic Publishers, Dordrecht (2004)

    Book  Google Scholar 

  34. Ng, K.M., Gunawan, A., Poh, K.L.: A hybrid algorithm for the quadratic assignment problem. In: CSC 2008, Nevada, USA, 14–17 July 2008

    Google Scholar 

  35. Burkard, R., Karisch, S., Rendl, F.: Qaplib-a quadratic assignment problem library. J. Global Optim. 10(4), 391–403 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  36. Halim, S., Yap, R.H.C., Lau, H.C.: An integrated white+black box approach for designing and tuning stochastic local search. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 332–347. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74970-7_25

    Chapter  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centres in Singapore Funding Initiative.

Author information

Authors and Affiliations

  1. School of Information Systems, Singapore Management University, Singapore, Singapore

    Aldy Gunawan & Hoong Chuin Lau

  2. Department of Computer Science, University of Freiburg, Freiburg im Breisgau, Germany

    Mustafa Mısır

  3. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Mustafa Mısır

Authors
  1. Aldy Gunawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoong Chuin Lau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mustafa Mısır
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aldy Gunawan .

Editor information

Editors and Affiliations

  1. Univ of Naples Federico II, Naples, Italy

    Paola Festa

  2. Thomas J. Wastson Research Center, Yorktown Heights, New York, USA

    Meinolf Sellmann

  3. Eindhoven University of Technology, Eindhoven, The Netherlands

    Joaquin Vanschoren

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Gunawan, A., Lau, H.C., Mısır, M. (2016). Designing and Comparing Multiple Portfolios of Parameter Configurations for Online Algorithm Selection. In: Festa, P., Sellmann, M., Vanschoren, J. (eds) Learning and Intelligent Optimization. LION 2016. Lecture Notes in Computer Science(), vol 10079. Springer, Cham. https://doi.org/10.1007/978-3-319-50349-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50349-3_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50348-6

  • Online ISBN: 978-3-319-50349-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation