Abstract
In this paper, we obtain new drift-type conditions for nonexplosion and explosion for nonhomogeneous Markov pure jump processes in Borel state spaces. The conditions are sharp; e.g., the one for nonexplosion is necessary if the state space is in addition locally compact and the Q-function satisfies weak Feller-type and local boundedness conditions. We comment on the relations of our conditions with the existing ones in the literature and demonstrate some possible applications.
Similar content being viewed by others
References
Anderson, W.: Continuous-Time Markov Chains. Springer, New York (1991)
Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, Berlin (2011)
Bertsekas, D., Shreve, S.: Stochastic Optimal Control. Academic Press, New York (1978)
Blok, H., Spieksma, F.: Countable state Markov decision processes with unbounded jump rates and discounted optimality equation and approximations. Adv. Appl. Probab. 47, 1088–1107 (2015)
Chen, M.: Coupling for jump processes. Acta Math. Sin. (Engl. Ser.) 2, 123–136 (1986)
Chen, M.: A comment on the book “Continuous-time Markov Chains” by W. J. Anderson. Chin. J. Appl. Probab. Stat. 12, 55–59 (1996). Updated version also available at arXiv:1412.5856
Chen, M.: From Markov Chains to Non-Equilibrium Particle Systems. World Scientific, Singapore (2004)
Chen, M.: Eigenvalues, Inequalities, and Ergodic Theory. Springer, London (2005)
Chen, M.: Practical criterion for uniqueness of q-processes. Chin. J. Appl. Probab. Stat. 31, 213–224 (2015). arXiv:1503.02119
Chow, P., Khasminskii, R.: Methods of Lyapunov functions for analysis of absorption and explosion in Markov chains. Probl. Inf. Transm. 47, 232–250 (2011). (Translation of the original Russian text at Problemy Peredachi Informatsii 47, 19–38 (2011))
Eibeck, A., Wagner, W.: Stochastic interacting particle systems and nonlinear kinetic equations. Ann. Appl. Probab. 13, 845–889 (2003)
Feinberg, E., Mandava, M., Shiryaev, A.: On solutions of Kolmogorov’s equations for nonhomogeneous jump Markov processes. J. Math. Anal. Appl. 411, 261–270 (2014)
Feinberg, E., Mandava, M., Shiryaev, A.: Kolmogorov’s equations for jump Markov processes with unbounded jump rates (2016). arXiv:1603.02367
Feller, W.: On the integro-differential equations of purely discontinuous Markoff processes. Trans. Am. Math. Soc. 48, 488–515 (1940)
Guo, X.: Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Oper. Res. 32, 73–87 (2007)
Hu, D.: Analytic Theory of Markov Processes in General State Spaces. Wuhan University Press, Wuhan (2013). Republication of the edition originally published in 1985 (in Chinese)
Jacod, J.: Multivariate point processes: predictable projection, Radon–Nykodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie verw. Gebite. 31, 235–253 (1975)
Kitaev, M., Rykov, V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)
Menshikov, M., Petritis, D.: Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach. Stoch. Proc. Appl. 124, 2388–2414 (2014)
Meyn, S., Tweedie, R.: Markov Chains and Stochastic Stability. Springer, London (1993)
Meyn, S., Tweedie, R.: Stability of Markov processes III: Forster–Lyapunov criteria for continous time processes. Adv. Appl. Probab. 25, 518–548 (1993)
Piunovskiy, A., Zhang, Y.: Discounted continuous-time markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-Q. J. Oper. Res. 12, 49–75 (2014). The extended version is available at arXiv:1103.0134
Reuter, G.: Denumerable Markov processes and the associated contraction semigroups on \(l\). Acta Math. 97, 1–46 (1957)
Serfozo, R.: Convergence of Lebesgue integrals with varying measures. Sankhya 44, 380–402 (1982)
Spieksma, F.: Countable state Markov processes: non-explosiveness and moment function. Probab. Eng. Inf. Sci. 29, 623–637 (2015)
Spieksma, F.: Kolmogorov forward equation and explosiveness in countable state Markov processes. Ann. Oper. Res. 241, 3–22 (2016)
Wagner, W.: Explosion phenomena in stochastic coagulation-gragmentation models. Ann. Appl. Probab. 15, 2081–2112 (2005)
Ye, L., Guo, X.: Construction and regularity of transition functions on Polish spaces under measurability conditions. Acta Math. Appl. Sin. (Engl. Ser.) 29, 1–14 (2013)
Zheng, J., Zheng, X.: A martingale approadch to \(q\)-processes. Kexue Tongbao (Chin. Sci. Bull.) 32, 1457–1459 (1987)
Zheng, J.: Phase Transitions of Ising Model on Lattice Fractals, Martingale Approach for \(q\)-Processes. Ph.D. thesis. Beijing Normal University (1993) (in Chinese)
Acknowledgements
I would like to thank Professor Mufa Chen (Beijing Normal University) for providing the scan copy of the relevant pages in [30] and the paper [29]. I also thank the referees for their helpful comments and remarks. This work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
In this appendix, we present some auxiliary results and the proofs of some lemmas stated in the previous sections.
Proposition 6.1
Let \(\mu _n\) and \(\mu \) be \([0,\infty ]\)-valued measures on the Borel space X, and \(f_n\) (resp., f) be \((-\infty ,\infty )\)-valued \(\mu _n\)-integrable (resp., measurable) functions thereon, and \(g_n\) be nonnegative measurable functions on X. Suppose \(\mathop {\underline{\lim }}_{n\rightarrow \infty }\mu _n(\Gamma )\ge \mu (\Gamma )\) for each \(\Gamma \in \mathcal{B}(X)\), and \(f_n(x)\rightarrow f(x)\) for each \(x\in X.\) If \(|f_n(x)|\le g_n(x)\) for each \(x\in X\) and \(n\ge 1,\) and
then f is \(\mu \)-integrable, and
Proof
See Theorem 2.4 of [24] \(\square \)
Proof of Lemma 2.1
Let us firstly justify that (b), (c) and (d) are equivalent. Clearly, (d) implies (b), which in turn implies (c). That (c) implies (b) is true because clearly \(t\rightarrow P(0,x,t,S)\) is monotone nonincreasing in \(t\ge 0\). Now suppose (b) is true. By the Chapman–Kolmogorov equation, we see that for each \(x\in S\) and \(0\le s\le t\),
for almost all y with respect to \(P_q(0,x,s,\hbox {d}y)\). Now suppose for contradiction that (d) does not hold, so that there exist some \(0\le s<t\) and \(x\in S\) such that
Then
where the first equality and the last equality are by assumption that (b) holds, the second equality is by the Chapman–Kolmogorov equation, the third equality is by (40), and the inequality is by (41). This is a desired contradiction, and thus, assertions (b), (c) and (d) are equivalent.
It is clear that (a) implies (b), c.f. Proposition 2.3 and (6). That (b) implies (a) follows from the relation \( \{t_\infty =\infty \}=\bigcap _{n=1}^\infty \{X_n\in S\}, \) c.f. (6). The proof is completed. \(\square \)
Proof of Lemma 4.1
It is clear that \(\int _S g(y)q(\hbox {d}y|x,s)\) is well defined. For the second part of the statement, note that for each \(x\in S\) and \(s\ge 0\),
where the second inequality is by the fact that f is a c-drift function. Thus,
the integral on the left-hand side being well defined. Similarly, one can check that
as required. \(\square \)
Proof of Lemma 4.2
The first assertion follows from the proof of the relevant assertion in Theorem 3.1 of [15]. The second assertion follows from the first assertion because for each \(x\in S\) such that \(\int _{S}f(y)q(\hbox {d}y|x,s)\ge 0,\)
whereas for each \(x\in S\) such that \(\int _{S}f(y)q(\hbox {d}y|x,s)< 0,\)
The statement is proved. \(\square \)
Lemma 6.1
Let f be a c-drift function with respect to the Q-function q on S. Suppose
Then for each \(x\in S,\) \(0\le s\le t\), and f-bounded (measurable) function g on S,
In particular,
Proof
The first assertion immediately follows from Proposition 2.2 and Lemma 4.2. The last assertion follows because \(f\equiv 1\) is a c-drift function, and one can put \(g\equiv 1\) in the first assertion. \(\square \)
Proof of Lemma 4.3
We only need justify that (22) implies (23) as follows. Let some \(x \in S\) and \(t\in [0,\infty )\) be arbitrarily fixed. Since (22) holds by assumption,
by Lemma 4.2. Consequently, (24) holds. According to the Fubini theorem,
Now for each \(0\le u\le t,\) (22) implies
All the expressions are finite. Now (23) follows. \(\square \)
Lemma 6.2
For a c-drift function f with respect to the Q-function on S that satisfies relation (22), it holds that for each \(d\in (-\infty ,\infty )\), \(x\in S\) and \(t\ge 0\),
Proof
Keeping in mind Lemmas 4.2 and 4.3 (and especially (24)), direct calculations give
where the first and the third equalities are by (23). All the expressions are finite. The statement now follows. \(\square \)
Lemma 6.3
Consider a c-drift function f with respect to the Q-function q on S. For each \(x\in S,\) \(s,t\ge 0\), \(s\le t\) and \(\Gamma \in \mathcal{B}(S)\), the following relation holds;
Proof
By Feller’s construction, c.f., (2) and (3), it suffices to show
as follows. Let \(x\in S,~s,t\ge 0,~ s\le t,~\Gamma \in \mathcal{B}(S)\) be arbitrarily fixed. Consider \(n=0.\) Then
Now assume that (43) holds for \(n=k\). Then
where the first equality is by (44), the third equality is by (2), and the last equality is by (2), the inductive supposition and (7). Now (43) holds for \(n=k+1\), and thus for all \(n=0,1,\ldots \) by induction. \(\square \)
Rights and permissions
About this article
Cite this article
Zhang, Y. On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes. J Theor Probab 31, 1322–1355 (2018). https://doi.org/10.1007/s10959-017-0763-3
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10959-017-0763-3