Abstract
This chapter presents analytic methods for stochastic reachability using the connections with well-studied stochastic control problems (like the optimal stopping problem, obstacle problem and exit problem). These methods have to be adapted such that we capture the interaction between continuous dynamics and discrete dynamics, which is characteristic to a stochastic hybrid system. This interaction is illustrated mainly by the structure of the infinitesimal generator of the stochastic hybrid process, which is an integro-differential operator that satisfies some boundary conditions. For the characterisation of reach set probabilities, variational inequalities are derived. Variational inequalities based on energy forms, Dirichlet boundary problems and Hamilton-Jacobi-Bellman equations provide solutions for the estimation of reach set probabilities. The novelty of all these approaches consists of the ability of using the infinitesimal generator associated to a stochastic hybrid process. Simpler solutions of such problems might be obtained by ways to “approximate” this generator by operators that have richer properties and are easier to be handled in dynamic programming associated equations.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Approximation of general stochastic hybrid systems by switching diffusions with random hybrid jumps. In: Egerstedt, M., Mishra, B. (eds.) Hybrid Systems: Computation and Control. Lecture Notes in Computer Science, vol. 4981, pp. 598–601. Springer, Berlin (2008)
Barles, C., Chasseigne, E., Imbert, C.: On the Dirichlet problem for second-order elliptic integro-differential equations. Preprint (2007)
Bassan, B., Ceci, C.: Regularity of the value function and viscosity solutions in optimal stopping problems for general Markov processes. Stoch. Stoch. Rep. 74(3–4), 633–649 (2002)
Bismut, J.-M., Skalli, B.: Temps d’arrêt optimal, théorie générale des processus et processus de Markov. Probab. Theory Relat. Fields 36(4), 301–313 (1977)
Blumenthal, R.M., Getoor, R.K.: Markov Processes and Potential Theory. Academic Press, New York (1968)
Bujorianu, M.L., Lygeros, J.: General stochastic hybrid systems: modelling and optimal control. In: Proc. of 43rd IEEE Conference in Decision and Control, pp. 182–187 (2004)
Bujorianu, M.L., Lygeros, J.: New insights on stochastic reachability. In: Proc. of 46th IEEE Conference in Decision and Control, pp. 6172–6177 (2007)
Costa, O.L.V., Dufour, F.: On the Poisson equation for piecewise-deterministic Markov processes. SIAM J. Control Optim. 42(3), 985–1001 (2003)
Costa, O.L.V., Raymundo, C.A.B., Dufour, F.: Optimal stopping with continuous control of piecewise deterministic Markov processes. Stoch. Stoch. Rep. 70(1–2), 41–73 (2000)
Costa, O.L.V., Davis, M.H.A.: Approximations for optimal stopping of a piecewise-deterministic process. Math. Control Signals Syst. 1(2), 123–146 (1988)
Davis, M.H.A., Vellekoop, M.H.: Permanent health insurance: a case study in piecewise-deterministic Markov modelling. Mitt. - Schweiz. Ver. Versicher.math. 2, 177–212 (1995)
Davis, M.H.A.: Markov Models and Optimization. Chapman & Hall, London (1993)
Dynkin, E.B.: Optimal choice of the stopping moment of a Markov process. Dokl. Akad. Nauk SSSR 150, 238–240 (1963)
Dynkin, E.B., Yushkevich, A.A.: Markov Processes: Theorems and Problems. Plenum, New York (1969)
El Karoui, N., Lepeltier, J.-P., Millet, A.: A probabilistic approach to the reduite in optimal stopping. Probab. Math. Stat. 13(1), 97–121 (1992)
Farid, M., Davis, M.H.A.: Optimal consumption and exploration: a case study in piecewise-deterministic Markov modelling. In: Optimal Control and Differential Games, Vienna, 1997. Ann. Oper. Res., vol. 88, pp. 121–137 (1999)
Fleming, W., Soner, H.: Controlled Markov Processes and Viscosity Solutions. Springer, New York (1993)
Fukushima, M.: Dirichlet Forms and Markov Processes. North-Holland, Amsterdam (1980)
Gatarek, D.: On first order quasi-variational inequalities with integral terms. Appl. Math. Optim. 24, 85–98 (1992)
Garroni, M.G., Menaldi, J.L.: Second Order Elliptic Integro-Differential Problems. Chapman & Hall/CRC Press, London/Boca Raton (2002)
Getoor, R.K.: Markov Processes: Ray Processes and Right Processes. Lecture Notes in Mathematics, vol. 440. Springer, Berlin (1975)
Getoor, R.K., Glover, J.: Riesz decompositions in Markov process theory. Trans. Am. Math. Soc. 285(1), 107–132 (1984)
Getoor, R.K., Steffens, J.: The energy functional, balayage, and capacity. Ann. Henri Poincaré 23(2), 321–357 (1987)
Glover, J.: Representing last exit potentials as potentials of measures. Z. Wahrscheinlichkeitstheor. Verw. Geb. 61(1), 17–30 (1982)
Graversen, S.E.: A Riesz decomposition theorem. Nagoya Math. J. 114, 123–133 (1989)
Gugerli, U.S.: Optimal stopping of a piecewise-deterministic Markov process. Stochastics 19(4), 221–236 (1986)
Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. Academic Press, London (1980)
Koutsoukos, X., Riley, D.: Computational methods for verification of stochastic hybrid systems. IEEE Trans. Syst. Man Cybern., Part A, Syst. Hum. 38(2), 385–396 (2008)
Krystul, J., Blom, H.A.P.: Sequential Monte Carlo simulation of rare event probability in stochastic hybrid systems. Preprint, 16th IFAC World Congress (2005)
Kushner, H.J.: Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Academic Press, New York (1977)
Ishii, H.: Perron’s method for Hamilton–Jacobi equations. Duke Math. J. 55(2), 369–384 (1987)
Lenhard, S.M., Yamada, N.: Perron’s method for viscosity solutions associated with piecewise deterministic processes. Funkc. Ekvacioj 34, 173–186 (1991)
Lenhart, S., Liao, Y.C.: Integro-differential equations associated with optimal stopping time of a piecewise-deterministic process. Stochastics 15(3), 183–207 (1985)
Ma, M., Rockner, M.: The Theory of (Non-Symmetric) Dirichlet Forms and Markov Processes. Springer, Berlin (1990)
Mataloni, S.: Representation formulas for non-symmetric Dirichlet forms. Z. Anal. Anwend. 18(4), 1039–1064 (1999)
Mertens, J.F.: Strongly supermedian functions and optimal stopping. Z. Wahrscheinlichkeitstheor. Verw. Geb. 22, 45–68 (1972)
Mordecki, E., Salminen, P.: Optimal stopping of Hunt and Lévy processes. Stochastics 79(3–4), 233–251 (2007)
Nagai, H.: On an optimal stopping problem and a variational inequalities. J. Math. Soc. Jpn. 30(2), 303–312 (1978)
Peskir, G., Shiryayev, A.N.: Optimal Stopping and Free-Boundary Problems. Birkhäuser, Basel (2006)
Prandini, M., Hu, J.: A Stochastic approximation method for reachability computations. In: [28], pp. 107–139
Salminen, P.: Optimal stopping of one-dimensional diffusions. Math. Nachr. 124, 85–101 (1985)
Shiryayev, A.N.: Optimal Stopping Rules. Springer, Berlin (1976)
Weis, L., Werner, D.: Reference measures and the fine topology. Preprint (1999)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag London Limited
About this chapter
Cite this chapter
Bujorianu, L.M. (2012). Analytic Methods for Stochastic Reachability. In: Stochastic Reachability Analysis of Hybrid Systems. Communications and Control Engineering. Springer, London. https://doi.org/10.1007/978-1-4471-2795-6_7
Download citation
DOI: https://doi.org/10.1007/978-1-4471-2795-6_7
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-2794-9
Online ISBN: 978-1-4471-2795-6
eBook Packages: EngineeringEngineering (R0)