Skip to main content
SpringerLink
Log in

Orderfield property of mixtures of stochastic games

  • Published:
Sankhya A Aims and scope Submit manuscript

Abstract

We consider certain mixtures, Γ, of classes of stochastic games and provide sufficient conditions for these mixtures to possess the orderfield property. For 2-player zero-sum and non-zero sum stochastic games, we prove that if we mix a set of states S 1 where the transitions are controlled by one player with a set of states S 2 constituting a sub-game having the orderfield property (where S 1 ∩ S 2 = ∅), the resulting mixture Γ with states S = S 1 ∪ S 2 has the orderfield property if there are no transitions from S 2 to S 1. This is true for discounted as well as undiscounted games. This condition on the transitions is sufficient when S 1 is perfect information or SC (Switching Control) or ARAT (Additive Reward Additive Transition). In the zero-sum case, S 1 can be a mixture of SC and ARAT as well. On the other hand,when S 1 is SER-SIT (Separable Reward — State Independent Transition), we provide a counter example to show that this condition is not sufficient for the mixture Γ to possess the orderfield property. In addition to the condition that there are no transitions from S 2 to S 1, if the sum of all transition probabilities from S 1 to S 2 is independent of the actions of the players, then Γ has the orderfield property even when S 1 is SER-SIT. When S 1 and S 2 are both SERSIT, their mixture Γ has the orderfield property even if we allow transitions from S 2 to S 1. We also extend these results to some multi-player games namely, mixtures with one player control Polystochastic games. In all the above cases, we can inductively mix many such games and continue to retain the orderfield property.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bewley, T. and Kohlberg, E. (1976). The asymptotic theory of stochastic games. Math. Oper. Res., 1, 197–208.

    Article  MATH  MathSciNet  Google Scholar 

  • Blackwell, D. and Ferguson, T.S. (1968). The big match. Ann. Math. Statist., 39, 159–168.

    Article  MATH  MathSciNet  Google Scholar 

  • Condon, A. (1992). The complexity of stochastic games. Inform. and Comput., 96, 203–224.

    Article  MATH  MathSciNet  Google Scholar 

  • Cottle, R.W., Pang, J.S. and Stone, R.E. (1992). The Linear Complementarity Problem. Academic Press, New York.

    MATH  Google Scholar 

  • Filar, J.A. (1981). Orderfield property of stochastic games when the player who controls the transition changes from state to state. J. Optim. Theory Appl., 34, 505–517.

    Article  MathSciNet  Google Scholar 

  • Filar, J.A. and Vrieze, O.J. (1997). Competitive Markov Decision Processes. Springer, New York.

    MATH  Google Scholar 

  • Filar, J.A., Schultz, T., Thuijsman, F. and Vrieze, O.J. (1991). Nonlinear programming and stationary equilibria in stochastic games. Math. Program., 50, 227–237.

    Article  MATH  MathSciNet  Google Scholar 

  • Fink, A.M. (1964). Equilibrium in a stochastic n-person game. J. Sci. Hiroshima Univ. Ser. A-I Math., 28, 89–93.

    MATH  MathSciNet  Google Scholar 

  • Flesch, J., Thuijsman, F. and Vrieze, O.J. (2007). Stochastic games with additive transitions. European J. Oper. Res., 179, 483–497.

    Article  MATH  MathSciNet  Google Scholar 

  • Gillette, D. (1957). Stochastic games with zero-stop probabilities. In Contributions to the Theory of Games, Vol. III, (M. Dresher, A.W. Tucker and P. Wolfe, eds.). Ann. Math. Studies, 39. Princeton University Press, Princeton, NJ, 179–188.

    Google Scholar 

  • Howson, J.T., Jr. (1972). Equilibria of polymatrix games. Management Sci., 18, 312–318.

    Article  MATH  MathSciNet  Google Scholar 

  • Janovskaya, E.B. (1968). Equilibrium points of polymatrix games. Litovsk. Mat. Sb., 8, 381–384. (In Russian).

    MATH  MathSciNet  Google Scholar 

  • Kaplansky, I. (1945). A contribution to von Neumann’s theory of games. Ann. of Math. (2), 46, 474–479.

    Article  MathSciNet  Google Scholar 

  • Lemke, C. (1965). Bimatrix equilibrium points and mathematical programming. Management Sci., 11, 681–689.

    Article  MathSciNet  Google Scholar 

  • Maitra, A. and Parthasarathy, T. (1970). On stochastic games. J. Optim. Theory Appl., 5, 289–300.

    Article  MATH  MathSciNet  Google Scholar 

  • Maitra, A.P. and Sudderth, W. (1996). Discrete Gambling and Stochastic Games. Springer-Verlag, New York.

    MATH  Google Scholar 

  • Mertens, J.F. (2002). Stochastic games. In Handbook of Game Theory with Economic Applications, 3, (R. Aumann and S. Hart, eds.). Elsevier, 1809–1832.

  • Mertens, J.F. and Neyman, A. (1981). Stochastic games. Internat. J. Game Theory, 10, 53–66.

    Article  MATH  MathSciNet  Google Scholar 

  • Mohan, S.R., Neogy, S.K. and Parthasarathy, T. (1997). Linear complementarity and discounted polystochastic game when one player controls transitions. In Complementarity and Variational Problems (Baltimore, MD, 1995), (M.C. Ferris, J.-S. Pang, eds.). SIAM, Philadelphia, 284–294.

    Google Scholar 

  • Mohan, S.R., Neogy S.K. and Parthasarathy, T. (2001). Pivoting algorithms for some classes of stochastic games: A survey. Int. Game Theory Rev., 3, 253–281.

    Article  MATH  MathSciNet  Google Scholar 

  • Mohan, S.R., Neogy S.K., Parthasarathy, T. and Sinha, S. (1999). Vertical linear complementarity and discounted zero-sum stochastic games with ARAT structure. Math. Program., Ser. A, 86, 637–648.

    Article  MATH  MathSciNet  Google Scholar 

  • Nash, J.F. (1951). Non-cooperative Games. Ann. of Math. (2), 54, 286–295.

    Article  MathSciNet  Google Scholar 

  • Neogy, S.K., Das, A.K., Sinha, S. and Gupta, A. (2008). On a mixture class of stochastic game with ordered field property. In Mathematical Programming and Game Theory for Decision Making, (S.K. Neogy, R.B. Bapat, A.K. Das and T. Parthasarathy, eds.). Stat. Sci. Interdiscip. Res., 1. World Scientific, Singapore, 451–477.

    Chapter  Google Scholar 

  • Nowak, A.S. and Raghavan, T.E.S. (1993). A finite step algorithm via a bimatrix game to a single controller non-zero sum stochastic game. Math. Program., 59, 249–259.

    Article  MathSciNet  Google Scholar 

  • Parthasarathy, T. and Raghavan, T.E.S. (1981). An orderfield property for stochastic games when one player controls transition probabilities. J. Optim. Theory Appl., 33, 375–392.

    Article  MATH  MathSciNet  Google Scholar 

  • Parthasarathy, T., Tijs, S.J. and Vrieze, O.J. (1984). Stochastic games with state independent transitions and separable rewards. In Selected Topics in Operations Research and Mathematical Economics (Karlsruhe, 1983), (G. Hammer and D. Pallaschke, eds.). Lecture Notes in Econom. and Math. Systems, 226, Springer, Berlin, 262–271.

    Google Scholar 

  • Raghavan, T.E.S. (2003). Finite-step algorithms for single controller and perfect information stochastic games. In Stochastic games and applications (Stony Brook, NY, 1999), (Abraham Neyman and Sylvain Sorin, eds.). NATO Sci. Ser. C Maths. Phys. Sci., 570. Kluwer Academic Publishers Group, Dordrecht, 227–251.

    Google Scholar 

  • Raghavan, T.E.S. and Filar, J.A. (1991). Algorithms for stochastic games — A survey. Z. Oper. Res., 35, 437–472.

    MATH  MathSciNet  Google Scholar 

  • Raghavan, T.E.S. and Syed, Z. (2002). Computing stationary Nash equilibria of undiscounted single-controller stochastic games. Math. Oper. Res., 27, 384–400.

    Article  MATH  MathSciNet  Google Scholar 

  • Raghavan, T.E.S. and Syed, Z. (2003). A policy-improvement type algorithm for solving zero-sum two-person stochastic games of perfect information. Math. Program., Ser. A, 95, 513–532.

    Article  MATH  MathSciNet  Google Scholar 

  • Raghavan, T.E.S., Tijs, S.J. and Vrieze, O.J. (1986). Stochastic games with additive rewards and additive transitions. J. Optim. Theory Appl., 47, 451–464.

    Article  MathSciNet  Google Scholar 

  • Schultz, T.A. (1992). Linear complementarity and discounted switching controller stochastic games. J. Optim. Theory Appl., 73, 89–99.

    Article  MATH  MathSciNet  Google Scholar 

  • Shapley, L. (1953). Stochastic games. Proc. Natl. Acad. Sci., 39, 1095–1100.

    Article  MATH  MathSciNet  Google Scholar 

  • Sinha, S. (1989). A contribution to the theory of stochastic games. Ph.D. thesis, Indian Statistical Institute, New Delhi, India.

    Google Scholar 

  • Sobel, M.J. (1971). Noncooperative stochastic games. Ann. Math. Statist., 42, 1930–1935.

    Article  MATH  MathSciNet  Google Scholar 

  • Sobel, M.J. (1981). Myopic solutions of Markov decision processes and stochastic games. Oper. Res., 29, 995–1009.

    Article  MATH  MathSciNet  Google Scholar 

  • Syed, Z. (1999). Algorithms for stochastic games and related topics. Ph. D. thesis, University of Illinois at Chicago.

  • Takahashi, M. (1964). Equilibrium points of stochastic noncooperative n-person games. J. Sci. Hiroshima Univ. Ser. A-I Math., 28, 95–99.

    MATH  MathSciNet  Google Scholar 

  • Thuijsman, F. and Raghavan, T.E.S. (1997). Perfect information stochastic games and related classes. Internat. J. Game Theory, 26, 403–408.

    MATH  MathSciNet  Google Scholar 

  • Vrieze, O.J. (1981). Linear programming and undiscounted stochastic game in which one player controls transitions. OR Spectrum, 3, 29–35.

    MATH  Google Scholar 

  • Vrieze, O.J., Tijs, S.H., Raghavan, T.E.S. and Filar, J.A. (1983). A finite algorithm for the switching control stochastic game. OR Spectrum, 5, 15–24.

    MATH  Google Scholar 

  • Weyl, H. (1950). Elementary proof of a minimax theorem due to von Neumann. In Contributions to the Theory of Games, Vol. I, (H.W. Kuhn and A.W. Tucker, eds.). Ann. Math. Studies, 24. Princeton University Press, Princeton, NJ, 19–25.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chennai Mathematical Institute, Chennai, India

    Nagarajan Krishnamurthy

  2. Indian Statistical Institute, Chennai, India

    T. Parthasarathy & G. Ravindran

  3. Computer Science, Chennai Mathematical Institute, Plot H1, SIPCOT IT Park, Siruseri, 603103, India

    Nagarajan Krishnamurthy

  4. Indian Statistical Institute, 110, Nelson Manickam Road, Chennai, 600029, India

    T. Parthasarathy & G. Ravindran

Authors
  1. Nagarajan Krishnamurthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Parthasarathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. Ravindran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nagarajan Krishnamurthy.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Krishnamurthy, N., Parthasarathy, T. & Ravindran, G. Orderfield property of mixtures of stochastic games. Sankhya 72, 246–275 (2010). https://doi.org/10.1007/s13171-010-0012-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-010-0012-7

AMS (2000) subject classification

Keywords and phrases

Search

Navigation