Advertisement

An Adaptive Max-Plus Eigenvector Method for Continuous Time Optimal Control Problems

  • Peter M. DowerEmail author
Chapter
Part of the Springer INdAM Series book series (SINDAMS, volume 29)

Abstract

An adaptive max-plus eigenvector method is proposed for approximating the solution of continuous time nonlinear optimal control problems. At each step of the method, given a set of quadratic basis functions, a standard max-plus eigenvector method is applied to yield an approximation of the value function of interest. Using this approximation, an approximate level set of the back substitution error defined by the Hamiltonian is tessellated according to where each basis function is active in approximating the value function. The polytopes obtained, and their vertices, are sorted according to this back substitution error, allowing “worst-case” basis functions to be identified. The locations of these basis functions are subsequently evolved to yield new basis functions that reduce this worst-case. Basis functions that are inactive in the value function approximation are pruned, and the aforementioned steps repeated. Underlying algebraic properties associated with max-plus linearity, dynamic programming, and semiconvex duality are provided as a foundation for the development, and the utility of the proposed method is illustrated by example.

Keywords

Optimal control Dynamic programming Semiconvexity Max-plus algebra Max-plus eigenvector method Basis adaptation 

Notes

Acknowledgements

This research was partially supported by AFOSR/AOARD grant FA2386-16-1-4066.

References

  1. 1.
    McEneaney, W.: Max-Plus Methods for Nonlinear Control and Estimation. Systems & Control: Foundations & Application. Birkhauser, Basel (2006)Google Scholar
  2. 2.
    McEneaney, W.: A curse-of-dimensionality-free numerical method for solution of certain HJB PDEs. SIAM J. Control Optim. 46(4), 1239–1276 (2007)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Akian, M., Gaubert, S., Lakhoua, A.: The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis. SIAM J. Control Optim. 47(2), 817–848 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)zbMATHGoogle Scholar
  5. 5.
    McEneaney, W.: A new fundamental solution for differential Riccati equations arising in control. Automatica 44, 920–936 (2008)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Dower, P., McEneaney, W., Zhang, H.: Max-plus fundamental solution semigroups for optimal control problems. In: Proceedings of SIAM Conference on Control Theory and Its Applications (Paris), 2015, pp. 368–375 (2015)Google Scholar
  7. 7.
    Qu, Z.: A max-plus based randomized algorithm for solving a class of HJB PDEs. In: Proceedings of 53rd IEEE Conference on Decision and Control (Los Angeles, CA) (2014)Google Scholar
  8. 8.
    Dower, P.: An approximation arising in max-plus based optimal stopping. In: Proceedings of Australian Control Conference (Sydney), pp. 271–276 (2012)Google Scholar
  9. 9.
    Grune, L., Dower, P.: Hamiltonian based a posteriori error estimation for Hamilton-Jacobi-Bellman equations, Technical report. Universitat Bayreuth. https://epub.uni-bayreuth.de/id/eprint/3578 (2018)
  10. 10.
    Rockafellar, R.: Conjugate Duality and Optimization. SIAM Regional Conference Series in Applied Mathematics, vol. 16. SIAM, Philadelphia (1974)CrossRefGoogle Scholar
  11. 11.
    Baccelli, F., Cohen, G., Olsder, G., Quadrat, J.-P.: Synchronization and Linearity. Wiley, New York (1992)zbMATHGoogle Scholar
  12. 12.
    Kolokoltsov, V., Maslov, V.: Idempotent Analysis and Applications. Kluwer Publishing House, Dordrecht (1997)CrossRefGoogle Scholar
  13. 13.
    Litvinov, G., Maslov, V., Shpiz, G.: Idempotent functional analysis: an algebraic approach. Math. Notes 69(5), 696–729 (2001)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Cohen, G., Gaubert, S., Quadrat, J.-P.: Duality and separation theorems in idempotent semimodules. Linear Algebra Appl. 379, 395–422 (2004)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Fleming, W., McEneaney, W.: A max-plus-based algorithm for a Hamilton-Jacobi-Bellman equation of nonlinear filtering. SIAM J. Control Optim. 38(3), 683–710 (2000)MathSciNetCrossRefGoogle Scholar
  16. 16.
    McEneaney, W., Dower, P.: The principle of least action and fundamental solutions of mass-spring and n-body two-point boundary value problems. SIAM J. Control Optim. 53(5), 2898–2933 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Zhang, H., Dower, P.: Max-plus fundamental solution semigroups for a class of difference Riccati equations. Automatica 52, 103–110 (2015)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Dower, P., McEneaney, W.: A max-plus dual space fundamental solution for a class of operator differential Riccati equations. SIAM J. Control Optim. 53(2), 969–1002 (2015)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Dower, P., Zhang, H.: A max-plus primal space fundamental solution for a class of differential Riccati equations. Math. Control Signals Syst. 29(3), 1–33 (2017) [Online]. http://dx.doi.org/10.1007/s00498-017-0200-2
  20. 20.
    Dower, P., McEneaney, W.: Solving two-point boundary value problems for a wave equation via the principle of stationary action and optimal control. SIAM J. Control Optim. 55(4), 2151–2205 (2017)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Dower, P.: Basis adaptation for a max-plus eigenvector method arising in optimal control. In: Proceedings of 23rd International Symposium on Mathematical Theory of Networks and Systems (Hong Kong), pp. 350–355 (2018)Google Scholar
  22. 22.
    Avis, D., Fukuda, K.: A pivoting algorithm for convex hulls and vertex enumeration of arrangements and polyhedra. Discret. Comput. Geom. 8, 295–313 (1992)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Bremner, D., Fukuda, K., Marzetta, A.: Primal-dual methods for vertex and facet enumeration. Discret. Comput. Geom. 20, 333–357 (1998)MathSciNetCrossRefGoogle Scholar
  24. 24.
    de Berg, M., Cheong, O., van Kreveld, M., Overmars, M.: Computational Geometry: Algorithms and Applications, 3rd ed. Springer, Berlin (2008)CrossRefGoogle Scholar
  25. 25.
    Rockafellar, R., Wets, R.: Variational Analysis. Springer, Berlin (1997)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Electrical and Electronic EngineeringUniversity of MelbourneMelbourneAustralia

Personalised recommendations