Abstract
We consider a problem of optimal control of a two-state Markov process. The objective is to minimize a total discounted cost over an infinite horizon, when the capabilities of the control effort are different in the two states. The necessary optimality conditions allow studying state-costate dynamics over the regular and singular control regimes. By making use of the properties of the costate process we prove the optimality of a threshold policy and calculate the value of the threshold in some specific cases of the cost function, as well as in a case where a probabilistic constraint is imposed on the state variable. The distribution function of the state variable and the thresholds are expressed as a series of the modified Bessel functions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akella, R., & Kumar, P. R. (1986). Optimal Control of Production Rate in a Failure-Prone Manufacturing System. IEEE Transactions on Automatic Control, 31, 116–126.
Barucci, E., & Marazzina, D. (2012). Optimal investment, stochastic labor income and retirement. Applied Mathematics and Computation, 218, 5588–5604.
Becker, G. S. (1965). Theory of the Allocation of Time. The Economic Journal, 75, 493–517.
Bodie, Z., Merton, R. C., & Samuelson, W. F. (1992). Labor supply flexibility and portfolio choice in a life cycle model. Journal of Economic Dynamics and Control, 16, 427–449.
Domeij, D., & Floden, M. (2006). The labor-supply elasticity and borrowing constraints: Why estimates are biased. Review of Economic Dynamics, 9, 242–262.
Hartl, R. F., Sethi, S. P., & Vickson, R. G. (1995). A survey of the maximum principles for optimal control problems with state constraints. SIAM Review, 37, 181–218.
Hirsch, F., & Yor, M. (2012). On temporally completely monotone functions for Markov processes. Probability Surveys, 9, 253–286.
Khmelnitsky, E., Presman, E., & Sethi, S. P. (2011). Optimal production control of a failure-prone machine. Annals of Operations Research, 182, 67–86.
Kovchegov, Y., Meredith, N., & Nir, E. (2010). Occupation times and Bessel densities. Statistics and Probability Letters, 80, 104–110.
Maimon, O., Khmelnitsky, E., & Kogan, K. (1998). Optimal Flow Control in Manufacturing Systems: Production Planning and Scheduling. Dordrecht: Kluwer Academic Publishers.
Presman, E., Sethi, S., & Zhang, Q. (1995). Optimal Feedback Production Planning in a Stochastic N-machine Flowshop. Automatica, 31(9), 1325–1332.
Prudnikov, A. P., Brychkov, Yu A, & Marichev, O. I. (1988). Integrals and Series, Volume 2: Special Functions. New York: Gordon and Breach Science Publications.
Sethi, S.P. and G.L. Thompson, 2000, Optimal Control Theory: Applications to Management Science, 2\(^{nd}\) ed., Kluwer Academic Publishers, Dordrecht, The Netherlands.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Khmelnitsky, E. (2014). An Optimal Threshold Policy in Applications of a Two-State Markov Process. In: El Ouardighi, F., Kogan, K. (eds) Models and Methods in Economics and Management Science. International Series in Operations Research & Management Science, vol 198. Springer, Cham. https://doi.org/10.1007/978-3-319-00669-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-00669-7_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-00668-0
Online ISBN: 978-3-319-00669-7
eBook Packages: Business and EconomicsBusiness and Management (R0)