Design of Autonomously Learning Controllers Using Fynesse

  • Ralf Schoknecht
  • Martin Spott
  • Martin Riedmiller
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 59)


With the growing number and difficulty of control problems there is an increasing demand for design methods that are easy to use. Fynesse fulfils this requirement: without knowledge of a process model the system learns a control policy. Optimization goals like time-optimal or energy-optimal control as well as restrictions of allowed manipulated variables or system states can be defined in a simple and flexible way. Fynesse only learns on basis of success and failure of former control interactions and, thus, learning can be carried out directly at the real process. A priori knowledge about the control policy as, for example, a fuzzy or linear control law considerably improves the learning process. Moreover, the learned policy can be interpreted as fuzzy control law which allows for easily checking the plausibility.


Target Region Fuzzy Control Fuzzy Controller Local Cost Control Trajectory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. G. Barto, S. J. Bradtke, and S. P. Singh. Learning to act using real-time dynamic programming. Artificial Intelligence, (72): 81–138, 1995.CrossRefGoogle Scholar
  2. 2.
    A. G. Barto, R. S. Sutton, and C. W. Anderson. Neuron-like adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13: 834–846, 1983.Google Scholar
  3. 3.
    R. E. Bellman. Dynamic Programming. Princeton University Press, Princeton, NJ, 1957.MATHGoogle Scholar
  4. 4.
    H.R. Berenji and P. Khedkar. Learning and tuning fuzzy logic controllers through reinforcements. In IEEE Trans. Neural Networks, volume 3, pages 724–740, 1992.Google Scholar
  5. 5.
    D. P. Bertsekas. Dynamic Programming. Prentice-Hall, Englewood Cliffs, New Jersey, 1987.Google Scholar
  6. 6.
    M. Brown and C.J. Harris. A perspective and critique of adaptive neurofuzzy systems used in modelling and control applications. Int. J. Neural Systems, 6 (2): 197–220, 1995.MathSciNetCrossRefGoogle Scholar
  7. 7.
    D. Cayrac, D. Dubois, and H. Prade. Handling uncertainty with possibility theory and fuzzy sets in a satellite fault diagnosis application. IEEE Transactions on Fuzzy Systems, 4 (3): 251–269, 1996.CrossRefGoogle Scholar
  8. 8.
    C. de Boor. On calculating with B-splines. Journal of Approximation Theory, 6: 50–62, 1972.MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    D. Dubois and H. Prade. Possibility theory: an approach to computerized processing of uncertainty. Plenum Press, New York, 1988.MATHGoogle Scholar
  10. 10.
    D. Dubois and H. Prade. Fuzzy sets in approximate reasoning, part 1: Inference with possibility distributions. Fuzzy Sets and Systems, 40: 143–201, 1991.MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    G. E. Farin. Curves and surfaces for computer aided geometric design: a practical guide. Academic Press, Boston, 3rd edition, 1993.Google Scholar
  12. 12.
    M.I. Jordan and R.A. Jacobs. Learning to control an unstable system with forward modeling. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems, volume 2, pages 84–97. Morgan Kaufmann, San Mateo, California, 1989.Google Scholar
  13. 13.
    H. Kiendl. Fuzzy control methodenorientiert. Oldenbourg Verlag, 1997.Google Scholar
  14. 14.
    K. Michels and R. Kruse. Numerical stability analysis for fuzzy control. Journal of Approximate Reasoning, pages 3–24, 1997.Google Scholar
  15. 15.
    R. Munos and A. Moore. Variable Resolution Discretization in Optimal Control. Machine Learning, 1999. submitted.Google Scholar
  16. 16.
    M. Riedmiller. Selbständig lernende neuronale Steuerungen. PhD thesis, Universität Karlsruhe, 1996.Google Scholar
  17. 17.
    M. Riedmiller. High quality thermostat control by reinforcement learning - a case study. In Proceedings of the Conald Workshop 1998, Carnegie-Mellon-University, 1998.Google Scholar
  18. 18.
    M. Riedmiller. Reinforcement learning without an explicit terminal state. In Proc. of the International Joint Conference on Neural Networks, IJCNN ‘88, Anchorage, Alaska, 1998.Google Scholar
  19. 19.
    M. Riedmiller, M. Spott, and J. Weisbrod. First results on the application of the Fynesse control architecture. In IEEE 1997 International Aerospace Conference, volume 2, pages 421–434, Aspen, USA, 1997.Google Scholar
  20. 20.
    R.Schoknecht and M. Riedmiller. Using reinforcement learning for engine control. In Ninth International Conference on Artificial Neural Networks (ICANN ‘89),Edinburgh, 1999. preliminary accepted.Google Scholar
  21. 21.
    M. Spott. Using classic approximation techniques for approximate reasoning. In FUZZ-IEEE ‘88, pages 909–914, Anchorage, USA, 1998.Google Scholar
  22. 22.
    M. Spott. A theory of possibility distributions. Fuzzy Sets and Systems, 102 (2): 135–155, 1999.MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    M. Spott and J. Weisbrod. A new approach to the adaptation of fuzzy relations. In Proc. of EUFIT’96, volume 2, pages 782–786, Aachen, Deutschland, 1996.Google Scholar
  24. 24.
    R. Sun and T. Peterson. A subsymbolic+symbolic model for learning sequential navigation. In Fifth International Conference of the Society for Adaptive Behavior (SAB’98),Zurich, Switzerland, 1998. MIT Press.Google Scholar
  25. 25.
    R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, (3): 9–44, 1988.Google Scholar
  26. 26.
    C. J. Watkins. Learning from Delayed Rewards. Phd thesis, Cambridge University, 1989.Google Scholar
  27. 27.
    J. Weisbrod. Fuzzy control revisited — why is it working? In P. P. Wang, editor, Advances in Fuzzy Theory and Technology, Vol. III, pages 219–244. Bookwrights, Durham (NC), 1995.Google Scholar
  28. 28.
    Lotfi A. Zadeh. Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems, 1: 3–28, 1978.MathSciNetCrossRefGoogle Scholar
  29. 29.
    J. Zhang and A. Knoll. Constructing fuzzy controllers with b-spline models. In FUZZ-IEEE ‘86, New Orleans, USA, 1996.Google Scholar

Copyright information

© Physica-Verlag Heidelberg 2001

Authors and Affiliations

  • Ralf Schoknecht
    • 1
  • Martin Spott
    • 2
  • Martin Riedmiller
    • 1
  1. 1.Institut für Logik, Komplexität und DeduktionssystemeUniversität KarlsruheKarlsruheDeutschland
  2. 2.Institut für Programmstrukturen und DatenorganisationUniversität KarlsruheKarlsruheDeutschland

Personalised recommendations