Synthesis of Optimal Control Policies for Some Infinite-State Transition Systems

  • Michel Sintzoff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5133)


We develop a symbolic, logic-based technique for constructing optimal control policies in some transition systems where state spaces are large or infinite. These systems are presented as iterations of finite sets of guarded assignments which have costs. The optimality objective is to minimize the total costs of system executions reaching the set characterized by a given target predicate. Guards are predicates and control policies are expressed by tuples of guards. The optimal control policy refines the control policy of the given system. It is generated from the target predicate by an iteration based on backwards induction. This iterative procedure amounts to a variant of the symbolic algorithm generating the reachability precondition; the latter characterizes the states from which some system execution reaches the target set. The main difference is the introduction of greedy and cost-dependent iteration steps.


Transition System Optimal Policy Control Policy Action Program Symbolic Generator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abrial, J.-R.: Modeling in Event-B: System and Software Engineering. Cambridge Univ. Press, Cambridge (to be published, 2008)Google Scholar
  2. 2.
    Akin, E.: The General Topology of Dynamical Systems. Amer. Math. Soc., Providence (1998)Google Scholar
  3. 3.
    Back, R.-J., von Wright, J.: Refinement Calculus. Springer, New York (1998)zbMATHGoogle Scholar
  4. 4.
    Başa, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. SIAM, Philadelphia (1999)Google Scholar
  5. 5.
    Bellman, R.: Dynamic Programming. Princeton Univ. Press, Princeton (1957)Google Scholar
  6. 6.
    Bertsekas, D.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (2000)Google Scholar
  7. 7.
    Boutilier, C., Reiter, R., Price, B.: Symbolic dynamic programming for first-order MDPs. In: Proc. 7th Int. Joint Conf. Artificial Intelligence, pp. 690–697. M. Kaufmann, San Francisco (2001)Google Scholar
  8. 8.
    Bouyer, P., Cassez, F., Fleury, E., Larsen, K.: Synthesis of optimal strategies using HyTech. Electronic Notes Theor. Computer Sci. 119, 11–31 (2005)CrossRefGoogle Scholar
  9. 9.
    Bouyer, P., Brihaye, T., Chevalier, M.: Weighted o-minimal hybrid systems are more decidable than weighted timed automata! In: Artemov, S.N., Nerode, A. (eds.) LFCS 2007. LNCS, vol. 4514, pp. 69–83. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. MIT Press, Cambridge (1999)Google Scholar
  11. 11.
    de Bakker, J.W., de Roever, W.P.: A calculus for recursive program schemes. In: Nivat, M. (ed.) Proc. 1st Int. Conf. Automata, Languages and Programming, pp. 167–196. North-Holland, Amsterdam (1973)Google Scholar
  12. 12.
    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271 (1959)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Dijkstra, E.W.: A Discipline of Programming. Prentice-Hall, Englewood Cliffs (1976)zbMATHGoogle Scholar
  14. 14.
    Floyd, R.: Algorithm 97 (Shortest path). Commun. ACM 5, 345 (1962)CrossRefGoogle Scholar
  15. 15.
    Floyd, R.: Assigning meanings to programs. In: Schwartz, J.T. (ed.) Proc. Symp. Appl. Mathematics, vol. 19, pp. 19–31. Amer. Math. Soc., Providence (1967)Google Scholar
  16. 16.
    Henzinger, T.A., Majumdar, R., Raskin, J.-F.: A classification of symbolic transition systems. ACM Trans. Computational Logic 6, 1–32 (2005)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Kupferman, O., Vardi, M.Y.: An Automata-theoretic Approach to Reasoning about Infinite-state Systems. In: Proc. 12th Int. Conf. Computer Aided Verification. LNCS, vol. 1855, pp. 36–52. Springer, Berlin (2006)CrossRefGoogle Scholar
  18. 18.
    Lind, D., Marcus, B.: An Introduction to Symbolic Dynamics and Coding. Cambridge Univ. Press, Cambridge (1995)zbMATHGoogle Scholar
  19. 19.
    Schmidt, D.A.: Structure-preserving binary relations for program abstraction. In: Mogensen, T.Æ., Schmidt, D.A., Sudborough, I.H. (eds.) The Essence of Computation. LNCS, vol. 2566, pp. 245–265. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  20. 20.
    Sintzoff, M.: Iterative Synthesis of Control Guards Ensuring Invariance and Inevitability in Discrete-decision Games. In: Owe, O., Krogdahl, S., Lyche, T. (eds.) From Object-Orientation to Formal Methods. LNCS, vol. 2635, pp. 272–301. Springer, Heidelberg (2004)Google Scholar
  21. 21.
    Sontag, E.D.: Mathematical Control Theory. Springer, New-York (1990)zbMATHGoogle Scholar
  22. 22.
    van Lamsweerde, A., Sintzoff, M.: Formal derivation of strongly concurrent programs. Acta Informatica 12, 1–31 (1979)zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Vinter, R.: Optimal Control. Birkhäuser, Boston (2000)zbMATHGoogle Scholar
  24. 24.
    Warshall, S.: A theorem on boolean matrices. J. ACM 9, 11–12 (1962)zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Wiggins, S.: Introduction to Applied Nonlinear Dynamical Systems and Chaos. Springer, New York (1990)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Michel Sintzoff
    • 1
  1. 1.Department of Computing Science and EngineeringUniversité catholique de Louvain 

Personalised recommendations