A Functional Optimization Method for Continuous Domains

  • Viet-Hung Dang
  • Ngo Anh VienEmail author
  • Pham Le-Tuyen
  • Taechoong Chung
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 221)


Smart city solutions are often formulated as adaptive optimization problems in which a cost objective function w.r.t certain constraints is optimized using off-the-shelf optimization libraries. Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is an efficient derivative-free optimization algorithm where a black-box objective function is defined on a parameter space. This modeling makes its performance strongly depends on the quality of chosen features. This paper considers modeling the input space for optimization problems in reproducing kernel Hilbert spaces (RKHS). This modeling amounts to functional optimization whose domain is a function space that enables us to optimize in a very rich function class. Our CMA-ES-RKHS framework performs black-box functional optimization in the RKHS. Adaptive representation of the function and covariance operator is achieved with sparsification techniques. We evaluate CMA-ES-RKHS on simple functional optimization problems which are motivated from many problems of smart cities.


Functional optimization Smart city Cross-entropy Covariance matrix adaptation evolution strategy 



This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 102.05-2016.18. Tuyen and Chung are funded through the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology (2014R1A1A2057735).


  1. 1.
    Adler, R.J.: The Geometry of Random Fields. Wiley, Chichester (1981)zbMATHGoogle Scholar
  2. 2.
    Armas, R., Aguirre, H., Zapotecas-Martínez, S., Tanaka, K.: Traffic signal optimization: minimizing travel time and fuel consumption. In: Bonnevay, S., Legrand, P., Monmarché, N., Lutton, E., Schoenauer, M. (eds.) EA 2015. LNCS, vol. 9554, pp. 29–43. Springer, Cham (2016). Google Scholar
  3. 3.
    Chourabi, H., et al.: Understanding smart cities: an integrative framework. In: 2012 45th Hawaii International Conference on System Science (HICSS), pp. 2289–2297. IEEE (2012)Google Scholar
  4. 4.
    Conway, J.B.: A Course in Functional Analysis, vol. 96. Springer, New York (2013)Google Scholar
  5. 5.
    Doerr, A., Ratliff, N.D., Bohg, J., Toussaint, M., Schaal, S.: Direct loss minimization inverse optimal control. In: Robotics: Science and Systems XI (2015)Google Scholar
  6. 6.
    Edelbrunner, H., Handmann, U., Igel, C., Leefken, I., von Seelen, W.: Application and optimization of neural field dynamics for driver assistance. In: 2001 IEEE Proceedings of the Intelligent Transportation Systems, pp. 309–314. IEEE (2001)Google Scholar
  7. 7.
    Fournier, D., Fages, F., Mulard, D.: A greedy heuristic for optimizing metro regenerative energy usage. In: Railways 2014 (2014)Google Scholar
  8. 8.
    Ha, S., Liu, C.K.: Iterative training of dynamic skills inspired by human coaching techniques. ACM Trans. Graph. 34(1), 1:1–1:11 (2014)CrossRefzbMATHGoogle Scholar
  9. 9.
    Ha, S., Liu, C.K.: Evolutionary optimization for parameterized whole-body dynamic motor skills. In: ICRA, pp. 1390–1397 (2016)Google Scholar
  10. 10.
    Hansen, N.: The CMA Evolution Strategy: A Tutorial. CoRR, abs/1604.00772 (2016)Google Scholar
  11. 11.
    Hansen, N., Auger, A.: Principled design of continuous stochastic search: from theory to practice. In: Borenstein, Y., Moraglio, A. (eds.) Theory and Principled Methods for the Design of Metaheuristics. NCS, pp. 145–180. Springer, Heidelberg (2014). CrossRefGoogle Scholar
  12. 12.
    Hansen, N., Müller, S.D., Koumoutsakos, P.: Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evol. Comput. 11(1), 1–18 (2003)CrossRefGoogle Scholar
  13. 13.
    Heidrich-Meisner, V., Igel, C.: Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search. In: ICML, pp. 401–408 (2009)Google Scholar
  14. 14.
    Heidrich-Meisner, V., Igel, C.: Neuroevolution strategies for episodic reinforcement learning. J. Algorithms 64(4), 152–168 (2009)CrossRefzbMATHGoogle Scholar
  15. 15.
    Kramer, O., Satzger, B., Lässig, J.: Power prediction in smart grids with evolutionary local kernel regression. In: Graña Romay, M., Corchado, E., Garcia Sebastian, M.T. (eds.) HAIS 2010. LNCS (LNAI), vol. 6076, pp. 262–269. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  16. 16.
    Marinho, Z., Boots, B., Dragan, A., Byravan, A., Gordon, G.J., Srinivasa, S.: Functional gradient motion planning in reproducing kernel Hilbert spaces. In: RSS (2016)Google Scholar
  17. 17.
    Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  19. 19.
    Rubinstein, R.: The cross-entropy method for combinatorial and continuous optimization. Methodol. Comput. Appl. Probab. 1(2), 127–190 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Rubinstein, R.Y., Kroese, D.P.: The cross-entropy method: a unified approach to combinatorial optimization. Monte-Carlo Simulation And Machine Learning. Springer, New York (2013)CrossRefzbMATHGoogle Scholar
  21. 21.
    Rückert, E.A., Neumann, G., Toussaint, M., Maass, W.: Learned graphical models for probabilistic planning provide a new class of movement primitives. Front. Comput. Neurosci. 6(97), 1–20 (2013)Google Scholar
  22. 22.
    Schölkopf, B., Smola, A.J.: Learning with kernels support vector machines, regularization, optimization, and beyond. Adaptive Computation and Machine Learning series. MIT Press, Cambridge (2002)Google Scholar
  23. 23.
    Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997). Google Scholar
  24. 24.
    Stolfi, D.H., Alba. E.: Eco-friendly reduction of travel times in European smart cities. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1207–1214. ACM (2014)Google Scholar
  25. 25.
    Stulp, F., Sigaud, O.: Path integral policy improvement with covariance matrix adaptation. In: ICML (2012)Google Scholar
  26. 26.
    Tan, J., Gu, Y., Turk, G., Liu, C.K.: Articulated swimming creatures. ACM Trans. Graph. (TOG) 30(4), 58 (2011)CrossRefGoogle Scholar
  27. 27.
    Toussaint, M.: Newton Methods for K-order Markov Constrained Motion Problems. CoRR, abs/1407.0414 (2014)Google Scholar
  28. 28.
    Ulbrich, M.: Optimization with PDE Constraints. Optimization methods in Banach spaces, pp. 97–156. Springer, Dordrecht (2009)Google Scholar
  29. 29.
    Vien, N.A., Toussaint, M.: POMDP manipulation via trajectory optimization. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2015, September 28 - October 2, Hamburg, pp. 242–249 (2015)Google Scholar
  30. 30.
    Vincent, P., Bengio, Y.: Kernel matching pursuit. Mach. Learn. 48(1–3), 165–187 (2002)CrossRefzbMATHGoogle Scholar
  31. 31.
    Wang, J.M., Hamner, S.R., Delp, S.L., Koltun, V.: Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph 31(4), 25:1–25:11 (2012)Google Scholar
  32. 32.
    Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., Schmidhuber, J.: Natural evolution strategies. J. Mach. Learn. Res. 15(1), 949–980 (2014)MathSciNetzbMATHGoogle Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018

Authors and Affiliations

  • Viet-Hung Dang
    • 1
  • Ngo Anh Vien
    • 2
    Email author
  • Pham Le-Tuyen
    • 3
  • Taechoong Chung
    • 3
  1. 1.Research and Development CenterDuy Tan UniversityDa NangVietnam
  2. 2.Machine Learning and Robotics LabUniversity of StutgartStuttgartGermany
  3. 3.Department of Computer EngineeringKyung Hee UniversitySeoulKorea

Personalised recommendations