On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes
- 128 Downloads
Abstract
In this paper, we obtain new drift-type conditions for nonexplosion and explosion for nonhomogeneous Markov pure jump processes in Borel state spaces. The conditions are sharp; e.g., the one for nonexplosion is necessary if the state space is in addition locally compact and the Q-function satisfies weak Feller-type and local boundedness conditions. We comment on the relations of our conditions with the existing ones in the literature and demonstrate some possible applications.
Keywords
Dynkin’s formula Nonhomogeneous Markov pure jump process NonexplosionMathematics Subject Classification (2010)
60J75 90C40Notes
Acknowledgements
I would like to thank Professor Mufa Chen (Beijing Normal University) for providing the scan copy of the relevant pages in [30] and the paper [29]. I also thank the referees for their helpful comments and remarks. This work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).
References
- 1.Anderson, W.: Continuous-Time Markov Chains. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
- 2.Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, Berlin (2011)CrossRefzbMATHGoogle Scholar
- 3.Bertsekas, D., Shreve, S.: Stochastic Optimal Control. Academic Press, New York (1978)zbMATHGoogle Scholar
- 4.Blok, H., Spieksma, F.: Countable state Markov decision processes with unbounded jump rates and discounted optimality equation and approximations. Adv. Appl. Probab. 47, 1088–1107 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
- 5.Chen, M.: Coupling for jump processes. Acta Math. Sin. (Engl. Ser.) 2, 123–136 (1986)MathSciNetzbMATHGoogle Scholar
- 6.Chen, M.: A comment on the book “Continuous-time Markov Chains” by W. J. Anderson. Chin. J. Appl. Probab. Stat. 12, 55–59 (1996). Updated version also available at arXiv:1412.5856
- 7.Chen, M.: From Markov Chains to Non-Equilibrium Particle Systems. World Scientific, Singapore (2004)CrossRefzbMATHGoogle Scholar
- 8.Chen, M.: Eigenvalues, Inequalities, and Ergodic Theory. Springer, London (2005)zbMATHGoogle Scholar
- 9.Chen, M.: Practical criterion for uniqueness of q-processes. Chin. J. Appl. Probab. Stat. 31, 213–224 (2015). arXiv:1503.02119
- 10.Chow, P., Khasminskii, R.: Methods of Lyapunov functions for analysis of absorption and explosion in Markov chains. Probl. Inf. Transm. 47, 232–250 (2011). (Translation of the original Russian text at Problemy Peredachi Informatsii 47, 19–38 (2011))Google Scholar
- 11.Eibeck, A., Wagner, W.: Stochastic interacting particle systems and nonlinear kinetic equations. Ann. Appl. Probab. 13, 845–889 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
- 12.Feinberg, E., Mandava, M., Shiryaev, A.: On solutions of Kolmogorov’s equations for nonhomogeneous jump Markov processes. J. Math. Anal. Appl. 411, 261–270 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
- 13.Feinberg, E., Mandava, M., Shiryaev, A.: Kolmogorov’s equations for jump Markov processes with unbounded jump rates (2016). arXiv:1603.02367
- 14.Feller, W.: On the integro-differential equations of purely discontinuous Markoff processes. Trans. Am. Math. Soc. 48, 488–515 (1940)MathSciNetCrossRefzbMATHGoogle Scholar
- 15.Guo, X.: Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Oper. Res. 32, 73–87 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
- 16.Hu, D.: Analytic Theory of Markov Processes in General State Spaces. Wuhan University Press, Wuhan (2013). Republication of the edition originally published in 1985 (in Chinese) Google Scholar
- 17.Jacod, J.: Multivariate point processes: predictable projection, Radon–Nykodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie verw. Gebite. 31, 235–253 (1975)CrossRefzbMATHGoogle Scholar
- 18.Kitaev, M., Rykov, V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)zbMATHGoogle Scholar
- 19.Menshikov, M., Petritis, D.: Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach. Stoch. Proc. Appl. 124, 2388–2414 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
- 20.Meyn, S., Tweedie, R.: Markov Chains and Stochastic Stability. Springer, London (1993)CrossRefzbMATHGoogle Scholar
- 21.Meyn, S., Tweedie, R.: Stability of Markov processes III: Forster–Lyapunov criteria for continous time processes. Adv. Appl. Probab. 25, 518–548 (1993)CrossRefzbMATHGoogle Scholar
- 22.Piunovskiy, A., Zhang, Y.: Discounted continuous-time markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-Q. J. Oper. Res. 12, 49–75 (2014). The extended version is available at arXiv:1103.0134
- 23.Reuter, G.: Denumerable Markov processes and the associated contraction semigroups on \(l\). Acta Math. 97, 1–46 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
- 24.Serfozo, R.: Convergence of Lebesgue integrals with varying measures. Sankhya 44, 380–402 (1982)MathSciNetzbMATHGoogle Scholar
- 25.Spieksma, F.: Countable state Markov processes: non-explosiveness and moment function. Probab. Eng. Inf. Sci. 29, 623–637 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
- 26.Spieksma, F.: Kolmogorov forward equation and explosiveness in countable state Markov processes. Ann. Oper. Res. 241, 3–22 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
- 27.Wagner, W.: Explosion phenomena in stochastic coagulation-gragmentation models. Ann. Appl. Probab. 15, 2081–2112 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
- 28.Ye, L., Guo, X.: Construction and regularity of transition functions on Polish spaces under measurability conditions. Acta Math. Appl. Sin. (Engl. Ser.) 29, 1–14 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
- 29.Zheng, J., Zheng, X.: A martingale approadch to \(q\)-processes. Kexue Tongbao (Chin. Sci. Bull.) 32, 1457–1459 (1987)zbMATHGoogle Scholar
- 30.Zheng, J.: Phase Transitions of Ising Model on Lattice Fractals, Martingale Approach for \(q\)-Processes. Ph.D. thesis. Beijing Normal University (1993) (in Chinese) Google Scholar