Abstract
In this paper, we propose a numerical method for bounds computations of discrete-time Markov chains with different state spaces. This method is based on the necessary and sufficient conditions for the comparison of one-dimensional (also known as the point-wise comparison) of discrete-time Markov chains given in our previous work [3]. For achieving our objective, we proceed as follows. Firstly, we transform the comparison criterion under the form of a complete linear system of inequalities. Secondly, we use our implementation on Scilab software of Gamma-algorithm to determine the set of all possible bounds of a given Markov chain.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abu-Amsha, O., Vincent, J.M.: An Algorithm to Bound Functionals of Markov Chains with Large State Space. In: 4th INFORMS Telecommunications Conference, Boca Raton, March 8-11 (1998)
Ahmane, M., Ledoux, J., Truffet, L.: Criteria for the Comparison of Discrete-Time Markov Chains. In: The Thirteenth International Workshop on Matrices and Statistics in celebration of Ingram Olkin’s 80th Birthday, IWMS 2004, Bedlewo, Poland, August 18-21 (2004)
Ahmane, M., Ledoux, J., Truffet, L.: Positive invariance of polyherons and comparison of Markov reward models with different state spaces. In: POsitive Systems: Theory and Applications (POSTA 2006). LNCIS, vol. 341, pp. 153–160. Springer, Heidelberg (2006) (invited paper)
Castillo, E., Jubete, F.: The -algorithm and some applications. University of Cantabria, Santander, Spain (March 2003)
Castillo, E., Jubete, F., Pruneda, E., Solares, C.: Obtaining simultaneous solutions of linear subsystems of inequalities and duals. Linear Algebra and its Applications 346, 131–154 (2002)
Doisy, M.: A Coupling Technique For Comparison of Functions of Markov Processes. Appl. Math. Decis. Sci. 4, 39–64 (2000)
Jubete, F.: El cono polidrico convexo. Su indenciaen el algebra lineal y la programacion no lineal. Editorial CIS, Santander, Spain (1991)
Kester, A.J.M.: Preservation of Cone Characterizing Properties in Markov Chains. Phd thesis, Univ. of Rochester, New York (1977)
Keilson, J., Kester, A.: Monotone Matrices and Monotone Markov Processes. Stochastic Process. Appl. 5, 231–241 (1977)
Kemeny, J.G., Snell, J.L.: Finite Markov Chains. Springer, Heidelberg (1976)
Kijima, M.: Markov Processes for Stochastic Modeling. Chapman-Hall, Boca Raton (1997)
Ledoux, J., Truffet, L.: Markovian Bounds on Functions of Finite Markov Chains. Adv. in Appl. Probab. 33, 505–519 (2001)
Li, H., Shaked, M.: Stochastic Convexity and Concavity of Markov Processes. Math. Oper. Res. 29, 477–493 (1994)
Massey, W.A.: Stochastics Orderings for Markov Processes on Partially Ordered Spaces. Math. Oper. Res. 11, 350–367 (1987)
Muller, A., Stoyan, D.: Comparison methods for stochastic models and risks. J. Wiley and Sons, Chichester (2002)
Pekergin, N.: Stochastic Performance Bounds by State Space Reduction. Performance Eval. 117, 36–37 (1999)
Truffet, L.: Geometrical Bounds on an Output Stream of a Queue in ATM Switch: Application to the Dimensioning Problem. In: Kouvatsos, D. (ed.) ATM Networks: Performance Modelling and Analysis, vol. 2. Chapman-Hall, London (1996)
Whitt, W.: Stochastic comparisons for non-Markov processes. Math. Oper. Res. 11, 608–618 (1986)
Ziegler, G.M.: Lectures notes en Polytopes. Graduate Texts in Mathematics. Springer, Berlin (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ahmane, M., Truffet, L. (2009). Numerical Method for Bounds Computations of Discrete-Time Markov Chains with Different State Spaces. In: Al-Begain, K., Fiems, D., Horváth, G. (eds) Analytical and Stochastic Modeling Techniques and Applications. ASMTA 2009. Lecture Notes in Computer Science, vol 5513. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02205-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-02205-0_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02204-3
Online ISBN: 978-3-642-02205-0
eBook Packages: Computer ScienceComputer Science (R0)