Abstract
Optimization algorithms typically perform a series of function evaluations to find an approximation of an optimal point of the objective function. Evaluations can be expensive, e.g., if they depend on the results of a complex simulation. When dealing with higher-dimensional functions, the curse of dimensionality increases the difficulty of the problem rapidly and prohibits a regular sampling. Instead of directly optimizing the objective function, we replace it with a sparse grid interpolant, saving valuable function evaluations. We generalize the standard piecewise linear basis to hierarchical B-splines, making the sparse grid surrogate smooth enough to enable gradient-based optimization methods. Also, we use an uncommon refinement criterion due to Novak and Ritter to generate an appropriate sparse grid adaptively. Finally, we evaluate the new method for various artificial and real-world examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
H. Bachau, E. Cormier, P. Decleva, J.E. Hansen, F. Martín, Applications of B-splines in atomic and molecular physics. Rep. Prog. Phys. 64(12), 1815–1942 (2001)
M.P. Bendsøe, N. Kikuchi, Generating optimal topologies in structural design using a homogenization method. Comput. Methods Appl. Mech. Eng. 71(2) 197–224 (1988)
H.-J. Bungartz, Finite elements of higher order on sparse grids, Habilitationsschrift, Institut für Informatik, TU München, 1998
E. Cohen, R.F. Riesenfeld, G. Elber, Geometric Modeling with Splines: An Introduction (A K Peters, Natick, 2001)
M.G. Cox, The numerical evaluation of B-splines. IMA J. Appl. Math. 10(2), 134–149 (1972)
T.A. Davis, Algorithm 832: UMFPACK V4.3-an unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30(2), 196–199 (2004)
C. de Boor, On calculating with B-splines. J. Approx. Theory 6(1), 50–62 (1972)
C. de Boor, Splines as linear combinations of B-splines. A survey, in Approximation Theory II, ed. by G.G. Lorentz, C.K. Chui, L.L. Schumaker (Academic, New York, 1976), pp. 1–47
F. Delbos, L. Dumas, E. Echagüe, Global optimization based on sparse grid surrogate models for black-box expensive functions, http://dumas.perso.math.cnrs.fr/JOGO.pdf, (2016)
M.M. Donahue, G.T. Buzzard, A.E. Rundell, Parameter identification with adaptive sparse grid-based optimization for models of cellular processes, in Methods in Bioengineering: Systems Analysis of Biological Networks, ed. by A. Jayaraman, J. Hahn (Artech House, Boston/London, 2009), pp. 211–232
I. Ferenczi, Globale Optimierung unter Nebenbedingungen mit dünnen Gittern, Diploma thesis, Department of Mathematics, TU München, 2005
P.E. Gill, W. Murray, M.A. Saunders, SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12(4), 979–1006 (2002)
G. Guennebaud, B. Jacob et al., Eigen, http://eigen.tuxfamily.org/, (2016)
K. Höllig, Finite Element Methods with B-Splines (SIAM, Philadelphia, 2003)
K. Höllig, J. Hörner, Approximation and Modeling with B-Splines (SIAM, Philadelphia, 2013)
Y.-K. Hu, Y.P. Hu, Global optimization in clustering using hyperbolic cross points. Pattern Recognit. 40(6), 1722–1733 (2007)
D. Hübner, Mehrdimensionale Parametrisierung der Mikrozellen in der Zwei-Skalen-Optimierung, Master’s thesis, Department of Mathematics, FAU Erlangen-Nürnberg, 2014
M. Jamil, X.-S. Yang, A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4(2), 150–194 (2013)
Y. Jiang, Y. Xu, B-spline quasi-interpolation on sparse grids. J. Complex. 27(5), 466–488 (2011)
M. Kaltenbacher, Advanced simulation tool for the design of sensors and actuators, in Proceedings of Eurosensors XXIV, Linz, vol. 5, 2010, pp. 597–600
F. Lekien, J. Marsden, Tricubic interpolation in three dimensions. Int. J. Numer. Methods Eng. 63(3), 455–471 (2005)
L. Ljung, System Identification: Theory for the User, 2nd edn. (Prentice Hall, Upper Saddle River, 1999)
C.W. McCurdy, F. Martín, Implementation of exterior complex scaling in B-splines to solve atomic and molecular collision problems. J. Phys. B: At. Mol. Opt. Phys. 37(4), 917–936 (2004)
J.A. Nelder, R. Mead, A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)
J. Nocedal, S.J. Wright, Numerical Optimization (Springer, New York, 1999)
E. Novak, K. Ritter, Global optimization using hyperbolic cross points, in State of the Art in Global Optimization, ed. by C.A. Floudas, P.M. Pardalos (Springer, Boston, 1996), pp. 19–33
D. Pandey, Regression with spatially adaptive sparse grids in financial applications, Master’s thesis, Department of Informatics, TU München, 2008
D. Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, München, 2010)
D. Pflüger, Spatially adaptive refinement, in Sparse Grids and Applications, ed. by J. Garcke, M. Griebel. Lecture Notes in Computational Science and Engineering (Springer, Berlin/Heidelberg, 2012), pp. 243–262
E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. Inf. Rech. Oper. 3(1), 35–43 (1969)
Y. Renard, J. Pommier, Gmm++, http://download.gna.org/getfem/html/homepage/gmm/index.html, (2016)
M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in Proceedings of 1993 IEEE International Conference on Neural Networks, San Francisco, CA, vol. 1, 1993, pp. 586–591
C. Sanderson, Armadillo: an open source C++ linear algebra library for fast prototyping and computationally intensive experiments, Technical report, NICTA, 2010
I.J. Schoenberg, Contributions to the problem of approximation of equidistant data by analytic functions. Q. Appl. Math. 4, 45–99, 112–141 (1946)
R. Storn, K. Price, Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997)
J. Valentin, Hierarchische Optimierung mit Gradientenverfahren auf Dünngitterfunktionen, Master’s thesis, IPVS, Universität Stuttgart, 2014
D. Whitley, S. Rana, J. Dzubera, K.E. Mathias, Evaluating evolutionary algorithms. Artif. Intel. 85(1–2), 245–276 (1996)
X.-S. Yang, Engineering Optimization (Wiley, Hoboken, 2010)
Acknowledgements
This work was financially supported by the Juniorprofessurenprogramm of the Landesstiftung Baden-Württemberg.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Valentin, J., Pflüger, D. (2016). Hierarchical Gradient-Based Optimization with B-Splines on Sparse Grids. In: Garcke, J., Pflüger, D. (eds) Sparse Grids and Applications - Stuttgart 2014. Lecture Notes in Computational Science and Engineering, vol 109. Springer, Cham. https://doi.org/10.1007/978-3-319-28262-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-28262-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28260-2
Online ISBN: 978-3-319-28262-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)