, Volume 3, Issue 4, pp 345–359 | Cite as

A data-driven indirect method for nonlinear optimal control

  • Gao TangEmail author
  • Kris Hauser
Research Article


Nonlinear optimal control problems are challenging to solve due to the prevalence of local minima that prevent convergence and/or optimality. This paper describes nearest-neighbors optimal control (NNOC), a data-driven framework for nonlinear optimal control using indirect methods. It determines initial guesses for new problems with the help of precomputed solutions to similar problems, retrieved using k-nearest neighbors. A sensitivity analysis technique is introduced to linearly approximate the variation of solutions between new and precomputed problems based on their variation of parameters. Experiments show that NNOC can obtain the global optimal solution orders of magnitude faster than standard random restart methods, and sensitivity analysis can further reduce the solving time almost by half. Examples are shown on optimal control problems in vehicle control and agile satellite reorientation demonstrating that global optima can be determined with more than 99% reliability within time at the order of 10–100 milliseconds.


data-driven approach indirect method optimal control sensitivity analysis 



This work was partially supported by NSF (Grant No. IIS-1816540).


  1. [1]
    Betts, J. T. Survey of numerical methods for trajectory optimization. Journal of Guidance, Control, and Dynamics, 1998, 21(2): 193–207.CrossRefGoogle Scholar
  2. [2]
    Bryson Jr, A. E., Ho Y. C. Applied Optimal Control: Optimization, Estimation and Control. CRC Press, 1975.Google Scholar
  3. [3]
    Jiang, F. H., Baoyin, H. X., Li, J. F. Practical techniques for low-thrust trajectory optimization with homotopic approach. Journal of Guidance, Control, and Dynamics, 2012, 35(1): 245–258.CrossRefGoogle Scholar
  4. [4]
    Jetchev, N., Toussaint, M. Fast motion planning from experience: trajectory prediction for speeding up movement generation. Autonomous Robots, 2013, 34(1–2): 111–127.CrossRefGoogle Scholar
  5. [5]
    Hauser, K. Learning the problem-optimum map: analysis and application to global optimization in robotics. IEEE Transactions on Robotics, 2017, 33(1): 141–152.CrossRefGoogle Scholar
  6. [6]
    Bertrand, R., Epenoy, R. New smoothing techniques for solving bang-bang optimal control problems—Numerical results and statistical interpretation. Optimal Control Applications and Methods, 2002, 23(4): 171–197.MathSciNetCrossRefGoogle Scholar
  7. [7]
    Russell, R. P. Primer vector theory applied to global low-thrust trade studies. Journal of Guidance, Control, and Dynamics, 2007, 30(2): 460–472.CrossRefGoogle Scholar
  8. [8]
    Tang, G., Jiang, F. H., Li, J. F. Fuel-optimal low-thrust trajectory optimization using indirect method and successive convex programming. IEEE Transactions on Aerospace and Electronic Systems, 2018, 54(4): 2053–2066.CrossRefGoogle Scholar
  9. [9]
    Jiang, F. H., Tang, G., Li, J. F. Improving low-thrust trajectory optimization by adjoint estimation with shape-based path. Journal of Guidance, Control, and Dynamics, 2017, 40(12): 3282–3289.CrossRefGoogle Scholar
  10. [10]
    Cassioli, A., Di Lorenzo, D., Locatelli, M., Schoen, F., Sciandrone, M. Machine learning for global optimization. Computational Optimization and Applications, 2012, 51(1): 279–303.MathSciNetCrossRefGoogle Scholar
  11. [11]
    Pan, J., Chen, Z., Abbeel, P. Predicting initialization effectiveness for trajectory optimization. In: Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, 5183–5190.CrossRefGoogle Scholar
  12. [12]
    Bohg, J., Morales, A., Asfour, T., Kragic, D. Data-driven grasp synthesis—A survey. IEEE Transactions on Robotics, 2014, 30(2): 289–309.CrossRefGoogle Scholar
  13. [13]
    Lampariello, R., Nguyen-Tuong, D., Castellini, C., Hirzinger, G., Peters, J. Trajectory planning for optimal robot catching in real-time. In: Proceedings of 2011 IEEE International Conference on Robotics and Automation, 2011, 3719–3726.CrossRefGoogle Scholar
  14. [14]
    Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction. MIT Press, 1998.zbMATHGoogle Scholar
  15. [15]
    Sánchez-Sánchez, C., Izzo, D. Real-time optimal control via Deep Neural Networks: study on landing problems. Journal of Guidance, Control, and Dynamics, 2018, 41(5): 1122–1135.CrossRefGoogle Scholar
  16. [16]
    Tang, G., Sun, W. D., Hauser, K. Learning trajectories for real-time optimal control of quadrotors. In: Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018, 3620–3625.Google Scholar
  17. [17]
    Tang, G., Hauser, K. Discontinuity-sensitive optimal control learning by mixture of experts. arXiv preprint arXiv:1803.02493, 2018.Google Scholar
  18. [18]
    Bemporad, A., Morari, M., Dua, V., Pistikopoulos, E. N. The explicit solution of model predictive control via multiparametric quadratic programming. In: Proceedings of 2000 American Control Conference, 2000, 872–876.Google Scholar
  19. [19]
    Furfaro, R., Bloise, I., Orlandelli, M., Di Lizia, P., Topputo, F., Linares, R. Deep learning for autonomous lunar landing. In: Proceedings of 2018 AAS/AIAA Astrodynamics Specialist Conference, 2018, 1–22.Google Scholar
  20. [20]
    Ampatzis, C., Izzo, D. Machine learning techniques for approximation of objective functions in trajectory optimisation. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) 2009, Workshop on Artificial Intelligence in Space, 2009, 1–6.Google Scholar
  21. [21]
    Mereta, A., Izzo, D., Wittig, A. Machine learning of optimal low-thrust transfers between near-earth objects. In: Proceedings of the 12th International Conference on Hybrid Artificial Intelligence Systems, 2017, 543–553.Google Scholar
  22. [22]
    Izzo, D., Sprague, C., Tailor, D. Machine learning and evolutionary techniques in interplanetary trajectory design. arXiv preprint arXiv:1802.00180, 2018.Google Scholar
  23. [23]
    Tang, G., Hauser, K. A data-driven indirect method for nonlinear optimal control. In: Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017, 4854–4861.Google Scholar
  24. [24]
    Moré, J. J., Garbow, B. S., Hillstrom, K. E. User guide for MINPACK-1. Argonne National Laboratory Report ANL-80-74, Argonne National Laboratory, 1980.Google Scholar
  25. [25]
    Maurer, H., Augustin, D. Sensitivity analysis and real-time control of parametric optimal control problems using boundary value methods. Online Optimization of Large Scale Systems, 2001, 17–55.CrossRefGoogle Scholar
  26. [26]
    Xie, Z. M., Liu, C. K., Hauser, K. K. Differential dynamic programming with nonlinear constraints. In: Proceedings of 2017 IEEE International Conference on Robotics and Automation, 2017, 695–702.CrossRefGoogle Scholar
  27. [27]
    Ritz, R., Hehn, M., Lupashin, S., D’Andrea, R. Quadrocopter performance benchmarking using optimal control. In: Proceedings of 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011, 5179–5186.Google Scholar
  28. [28]
    Tomić T., Maier, M., Haddadin, S. Learning quadrotor maneuvers from optimal control and generalizing in real-time. In: Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, 1747–1754.CrossRefGoogle Scholar
  29. [29]
    Tang, G., Jiang, F. H. Capture of near-Earth objects with low-thrust propulsion and invariant manifolds. Astrophysics and Space Science, 2016, 361(1): 10.MathSciNetCrossRefGoogle Scholar
  30. [30]
    Schaub, H., Junkins, J. L. Analytical Mechanics of Space Systems, 2nd edn. AIAA Education Series, 2009.zbMATHGoogle Scholar
  31. [31]
    Li, J., Xi, X. N. Time-optimal reorientation of the rigid spacecraft using a pseudospectral method integrated homotopic approach. Optimal Control Applications and Methods, 2015, 36(6): 889–918.MathSciNetCrossRefGoogle Scholar
  32. [32]
    Bai, X. L., Junkins, J. L. New results for time-optimal three-axis reorientation of a rigid spacecraft. Journal of Guidance, Control, and Dynamics, 2009, 32(4): 1071–1076.CrossRefGoogle Scholar
  33. [33]
    Yershova, A., Jain, S., Lavalle, S. M., Mitchell, J. C. Generating uniform incremental grids on SO(3) using the Hopf fibration. The International Journal of Robotics Research, 2010, 29(7): 801–812.CrossRefGoogle Scholar

Copyright information

© Tsinghua University Press 2019

Authors and Affiliations

  1. 1.Department of Mechanical Engineering and Material ScienceDuke UniversityDurhamUSA
  2. 2.Department of Electrical and Computer EngineeringDuke UniversityDurhamUSA

Personalised recommendations