Journal of Intelligent & Robotic Systems

, Volume 91, Issue 1, pp 23–34 | Cite as

Control and Machine Intelligence for System Autonomy

  • Panos J. Antsaklis
  • Arash Rahnama


Autonomous systems evolve from control systems by adding functionalities that increase the level of system autonomy. It is very important to the research in the field that autonomy be well defined and so in the present paper a precise, useful definition of autonomy is introduced and discussed. Autonomy is defined as the ability of the system to attain a set of goals under a set of uncertainties. This leads to the notion of degrees or levels of autonomy. The Quest for Autonomy in engineered systems throughout the centuries is noted, connections to research work of 30 years ago are made and a hierarchical functional architecture for autonomous systems together with needed functionalities are outlined. Adaptation and Learning, which are among the most important functions in achieving high levels of autonomy are then highlighted and recent research contributions are briefly discussed.


Autonomy Machine intelligence Adaptation and learning 



  1. 1.
    Antsaklis, P.: Control systems and the quest for autonomy, Editorial. IEEE Trans. Autom. Control 62(3), 1013–1016 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Antsaklis, P.J.: Defining intelligent control. IEEE Control Systems Society Report of the Task Force on Intelligent Control. IEEE Control. Syst. 14(3), 4–5, 58–66 (1994)Google Scholar
  3. 3.
    Antsaklis, P.J.: On intelligent control: report of the IEEE CSS task force on intelligent control. Technical Report of the Interdisciplinary Studies of Intelligent Systems Group. University of Notre Dame 94, 001 (1994)Google Scholar
  4. 4.
    Antsaklis, P.J.: Intelligent learning control. Introduction to Special Issue, IEEE Control. Syst. 15(3), 5–7 (1995)Google Scholar
  5. 5.
    Antsaklis, P.J.: Intelligent control. Wiley Encyclopedia of Electrical and Electronics Engineering (1999)Google Scholar
  6. 6.
    Antsaklis, P.J.: The quest for autonomy revisited. Technical Report of the Interdisciplinary Studies of Intelligent Systems Group, University of Notre Dame 11, 004 (2011)Google Scholar
  7. 7.
    Antsaklis, P.J., Passino, K.: Autonomous control systems: Architecture and concepts for future space vehicles. Final Report, Contract 957856, Jet Propulsion Laboratory (1987)Google Scholar
  8. 8.
    Antsaklis, P.J., Passino, K.M.: Introduction to intelligent control systems with high degrees of autonomy. Kluwer Academic Publishers (1993)Google Scholar
  9. 9.
    Antsaklis, P.J., Passino, K.M., Wang, S.: Towards intelligent autonomous control systems: architecture and fundamental issues. J. Intell. Robot. Syst. 1(4), 315–342 (1989)CrossRefGoogle Scholar
  10. 10.
    Antsaklis, P.J., Passino, K.M., Wang, S.: An introduction to autonomous control systems. IEEE Control. Syst. 11(4), 5–13 (1991)CrossRefGoogle Scholar
  11. 11.
    Åström, K.J., Wittenmark, B.: Adaptive control. Courier Corporation (2013)Google Scholar
  12. 12.
    Aström, K.J., Albertos, P., Blanke, M., Isidori, A., Schaufelberger, W., Sanz, R.: Control of complex systems. Springer, Berlin (2011)Google Scholar
  13. 13.
    Barto, A.G., Bradtke, S.J., Singh, S.P.: Learning to act using real-time dynamic programming. Artif. Intell. 72(1-2), 81–138 (1995)CrossRefGoogle Scholar
  14. 14.
    Bcrtsekas, D.: Dynamic programming and optimal control, vol. I. Athena Scientific, Bellmont (1995)Google Scholar
  15. 15.
    Benard, N., Pons-Prat, J., Periaux, J., Bugeda, G., Bonnet, J.P., Moreau, E.: Multi-input genetic algorithm for experimental optimization of the reattachment downstream of a backward-facing-step with surface plasma actuator. In: 46th AIAA Plasmadynamics and lasers conference, pp. 2957–2980 (2015)Google Scholar
  16. 16.
    Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic programming: an overview. In: Proceedings of the 34Th IEEE conference on decision and control, IEEE, vol. 1, pp 560–564 (1995)Google Scholar
  17. 17.
    Bukkems, B., Kostic, D., De Jager, B., Steinbuch, M.: Learning-based identification and iterative learning control of direct-drive robots. IEEE Trans. Control Syst. Technol. 13(4), 537–549 (2005)CrossRefGoogle Scholar
  18. 18.
    Chi, R., Liu, X., Zhang, R., Hou, Z., Huang, B.: Constrained data-driven optimal iterative learning control. J. Process. Control 55, 10–29 (2017)CrossRefGoogle Scholar
  19. 19.
    Chowdhary, G.V., Johnson, E.N.: Theory and flight-test validation of a concurrent-learning adaptive controller. J. Guid. Control. Dyn. 34(2), 592–607 (2011)CrossRefGoogle Scholar
  20. 20.
    Dai, S.L., Wang, C., Wang, M.: Dynamic learning from adaptive neural network control of a class of nonaffine nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems 25(1), 111–123 (2014)CrossRefGoogle Scholar
  21. 21.
    Doya, K.: Reinforcement learning in continuous time and space. Neural Comput. 12(1), 219–245 (2000)CrossRefGoogle Scholar
  22. 22.
    Dracopoulos, D.C.: Genetic algorithms and genetic programming for control. In: Evolutionary algorithms in engineering applications, pp. 329–343. Springer (1997)Google Scholar
  23. 23.
    Feng, L., Zhang, K., Chai, Y., Yang, Z., Xu, S.: Observer-based fault estimators using iterative learning scheme for linear time-delay systems with intermittent faults. Asian J. Control 19(6), 1991–2008 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Foroutan, S.A., Salmasi, F.R.: Detection of false data injection attacks against state estimation in smart grids based on a mixture gaussian distribution learning method. IET Cyber-Physical Systems: Theory & Applications 2(4), 161–171 (2017)Google Scholar
  25. 25.
    Fu, K.S.: Learning control systems–review and outlook. IEEE Trans. Autom. Control 15(2), 210–221 (1970)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Goebel, G., Allgöwer, F: Semi-explicit mpc based on subspace clustering. Automatica 83, 309–316 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Hein, D., Hentschel, A., Runkler, T., Udluft, S.: Particle swarm optimization for generating interpretable fuzzy reinforcement learning policies. Eng. Appl. Artif. Intel. 65, 87–98 (2017)CrossRefGoogle Scholar
  28. 28.
    Hu, J., Zhou, M., Li, X., Xu, Z.: Online model regression for nonlinear time-varying manufacturing systems. Automatica 78, 163–173 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Kamalapurkar, R., Reish, B., Chowdhary, G., Dixon, W.E.: Concurrent learning for parameter estimation using dynamic state-derivative estimators. IEEE Trans. Autom. Control 62(7), 3594–3601 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Kiumarsi, B., Lewis, F.L., Jiang, Z.P.: H\(\infty \) control of linear discrete-time systems: Off-policy reinforcement learning. Automatica 78, 144–152 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems (2017)Google Scholar
  32. 32.
    Kokar, M.: Machine learning in a dynamic world. In: Proceedings of IEEE international symposium on intelligent control, pp. 500–507. IEEE (1988)Google Scholar
  33. 33.
    Lagoudakis, M.G., Parr, R., Littman, M.L.: Least-squares methods in reinforcement learning for control. In: Hellenic conference on artificial intelligence, pp. 249–260. Springer (2002)Google Scholar
  34. 34.
    Lee, C., Kim, J., Babcock, D., Goodman, R.: Application of neural networks to turbulence control for drag reduction. Phys. Fluids 9(6), 1740–1747 (1997)CrossRefGoogle Scholar
  35. 35.
    Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal control. Wiley , Hoboken (2012)CrossRefzbMATHGoogle Scholar
  36. 36.
    Lewis, F.L., Vrabie, D., Vamvoudakis, K.G.: Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control. Syst. 32(6), 76–105 (2012)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Michalewicz, Z., Janikow, C.Z., Krawczyk, J.B.: A modified genetic algorithm for optimal control problems. Computers & Mathematics with Applications 23(12), 83–94 (1992)CrossRefzbMATHGoogle Scholar
  38. 38.
    Michalski, R.S., Carbonell, J.G., Mitchell, T.M.: Machine learning: an artificial intelligence approach. Springer, Berlin (2013)zbMATHGoogle Scholar
  39. 39.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRefGoogle Scholar
  40. 40.
    Mohri, M., Rostamizadeh, A., Talwalkar, A.: Foundations of machine learning. MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  41. 41.
    Nageshrao, S.P., Lopes, G.A., Jeltsema, D., Babuška, R.: Port-hamiltonian systems in adaptive and learning control: a survey. IEEE Trans. Autom. Control 61(5), 1223–1238 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Nedić, A, Olshevsky, A., Uribe, C.A.: Fast convergence rates for distributed non-bayesian learning. IEEE Trans. Autom. Control 62(11), 5538–5553 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Plaat, A., Kosters, W., van den Herik, J.: Computers and games. Springer, Berlin (2017)zbMATHGoogle Scholar
  44. 44.
    Sklansky, J.: Learning systems for automatic control. IEEE Trans. Autom. Control 11(1), 6–19 (1966)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. 1. MIT Press, Cambridge (1998)Google Scholar
  46. 46.
    Sutton, R.S., Barto, A.G., Williams, R.J.: Reinforcement learning is direct adaptive optimal control. IEEE Control. Syst. 12(2), 19–22 (1992)CrossRefGoogle Scholar
  47. 47.
    Tsypkin, Y.: Self-learning–what is it? IEEE Trans. Autom. Control 13(6), 608–612 (1968)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Vrabie, D., Lewis, F.: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw. 22(3), 237–246 (2009)CrossRefzbMATHGoogle Scholar
  49. 49.
    Vrabie, D., Vamvoudakis, K.G., Lewis, F.L.: Optimal adaptive control and differential games by reinforcement learning principles, vol. 2. IET (2013)Google Scholar
  50. 50.
    Werbos, P.J.: Neural networks for control and system identification. In: Proceedings of the 28th IEEE conference on decision and control, pp. 260–265. IEEE (1989)Google Scholar
  51. 51.
    Kraft, L. G., Campagna, D.: A summary comparison of CMAC neural network and traditional adaptive control systems. Neural Networks for Control, W. T. Miller, R. Sutton, and P. Werbos, MIT Press, Cambridge, MA (1990)Google Scholar
  52. 52.
    Xie, J., Wan, Y., Mills, K., Filliben, J.J., Lewis, F.: A scalable sampling method to high-dimensional uncertainties for optimal and reinforcement learning-based controls. IEEE Control Systems Letters 1(1), 98–103 (2017)CrossRefGoogle Scholar
  53. 53.
    Yang, C., Teng, T., Xu, B., Li, Z., Na, J., Su, C.Y.: Global adaptive tracking control of robot manipulators using neural networks with finite-time learning convergence. Int. J. Control. Autom. Syst. 15 (4), 1916–1924 (2017)CrossRefGoogle Scholar
  54. 54.
    Yang, X., Ruan, X.: Reinforced gradient-type iterative learning control for discrete linear time-invariant systems with parameters uncertainties and external noises. IMA J. Math. Control. Inf. 34(4), 1117–1133 (2016)MathSciNetGoogle Scholar
  55. 55.
    Yang, X., He, H., Liu, D., Zhu, Y.: Adaptive dynamic programming for robust neural control of unknown continuous-time non-linear systems. IET Control Theory Appl. 11(14), 2307–2316 (2017)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.The Department of Electrical EngineeringUniversity of Notre DameNotre DameUSA

Personalised recommendations