Skip to main content
Log in

On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes

  • Published:
Journal of Theoretical Probability Aims and scope Submit manuscript

Abstract

In this paper, we obtain new drift-type conditions for nonexplosion and explosion for nonhomogeneous Markov pure jump processes in Borel state spaces. The conditions are sharp; e.g., the one for nonexplosion is necessary if the state space is in addition locally compact and the Q-function satisfies weak Feller-type and local boundedness conditions. We comment on the relations of our conditions with the existing ones in the literature and demonstrate some possible applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Anderson, W.: Continuous-Time Markov Chains. Springer, New York (1991)

    Book  MATH  Google Scholar 

  2. Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  3. Bertsekas, D., Shreve, S.: Stochastic Optimal Control. Academic Press, New York (1978)

    MATH  Google Scholar 

  4. Blok, H., Spieksma, F.: Countable state Markov decision processes with unbounded jump rates and discounted optimality equation and approximations. Adv. Appl. Probab. 47, 1088–1107 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, M.: Coupling for jump processes. Acta Math. Sin. (Engl. Ser.) 2, 123–136 (1986)

    MathSciNet  MATH  Google Scholar 

  6. Chen, M.: A comment on the book “Continuous-time Markov Chains” by W. J. Anderson. Chin. J. Appl. Probab. Stat. 12, 55–59 (1996). Updated version also available at arXiv:1412.5856

  7. Chen, M.: From Markov Chains to Non-Equilibrium Particle Systems. World Scientific, Singapore (2004)

    Book  MATH  Google Scholar 

  8. Chen, M.: Eigenvalues, Inequalities, and Ergodic Theory. Springer, London (2005)

    MATH  Google Scholar 

  9. Chen, M.: Practical criterion for uniqueness of q-processes. Chin. J. Appl. Probab. Stat. 31, 213–224 (2015). arXiv:1503.02119

  10. Chow, P., Khasminskii, R.: Methods of Lyapunov functions for analysis of absorption and explosion in Markov chains. Probl. Inf. Transm. 47, 232–250 (2011). (Translation of the original Russian text at Problemy Peredachi Informatsii 47, 19–38 (2011))

  11. Eibeck, A., Wagner, W.: Stochastic interacting particle systems and nonlinear kinetic equations. Ann. Appl. Probab. 13, 845–889 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  12. Feinberg, E., Mandava, M., Shiryaev, A.: On solutions of Kolmogorov’s equations for nonhomogeneous jump Markov processes. J. Math. Anal. Appl. 411, 261–270 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  13. Feinberg, E., Mandava, M., Shiryaev, A.: Kolmogorov’s equations for jump Markov processes with unbounded jump rates (2016). arXiv:1603.02367

  14. Feller, W.: On the integro-differential equations of purely discontinuous Markoff processes. Trans. Am. Math. Soc. 48, 488–515 (1940)

    Article  MathSciNet  MATH  Google Scholar 

  15. Guo, X.: Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Oper. Res. 32, 73–87 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hu, D.: Analytic Theory of Markov Processes in General State Spaces. Wuhan University Press, Wuhan (2013). Republication of the edition originally published in 1985 (in Chinese)

  17. Jacod, J.: Multivariate point processes: predictable projection, Radon–Nykodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie verw. Gebite. 31, 235–253 (1975)

    Article  MATH  Google Scholar 

  18. Kitaev, M., Rykov, V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)

    MATH  Google Scholar 

  19. Menshikov, M., Petritis, D.: Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach. Stoch. Proc. Appl. 124, 2388–2414 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  20. Meyn, S., Tweedie, R.: Markov Chains and Stochastic Stability. Springer, London (1993)

    Book  MATH  Google Scholar 

  21. Meyn, S., Tweedie, R.: Stability of Markov processes III: Forster–Lyapunov criteria for continous time processes. Adv. Appl. Probab. 25, 518–548 (1993)

    Article  MATH  Google Scholar 

  22. Piunovskiy, A., Zhang, Y.: Discounted continuous-time markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-Q. J. Oper. Res. 12, 49–75 (2014). The extended version is available at arXiv:1103.0134

  23. Reuter, G.: Denumerable Markov processes and the associated contraction semigroups on \(l\). Acta Math. 97, 1–46 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  24. Serfozo, R.: Convergence of Lebesgue integrals with varying measures. Sankhya 44, 380–402 (1982)

    MathSciNet  MATH  Google Scholar 

  25. Spieksma, F.: Countable state Markov processes: non-explosiveness and moment function. Probab. Eng. Inf. Sci. 29, 623–637 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  26. Spieksma, F.: Kolmogorov forward equation and explosiveness in countable state Markov processes. Ann. Oper. Res. 241, 3–22 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  27. Wagner, W.: Explosion phenomena in stochastic coagulation-gragmentation models. Ann. Appl. Probab. 15, 2081–2112 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  28. Ye, L., Guo, X.: Construction and regularity of transition functions on Polish spaces under measurability conditions. Acta Math. Appl. Sin. (Engl. Ser.) 29, 1–14 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Zheng, J., Zheng, X.: A martingale approadch to \(q\)-processes. Kexue Tongbao (Chin. Sci. Bull.) 32, 1457–1459 (1987)

    MATH  Google Scholar 

  30. Zheng, J.: Phase Transitions of Ising Model on Lattice Fractals, Martingale Approach for \(q\)-Processes. Ph.D. thesis. Beijing Normal University (1993) (in Chinese)

Download references

Acknowledgements

I would like to thank Professor Mufa Chen (Beijing Normal University) for providing the scan copy of the relevant pages in [30] and the paper [29]. I also thank the referees for their helpful comments and remarks. This work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yi Zhang.

Appendix

Appendix

In this appendix, we present some auxiliary results and the proofs of some lemmas stated in the previous sections.

Proposition 6.1

Let \(\mu _n\) and \(\mu \) be \([0,\infty ]\)-valued measures on the Borel space X,  and \(f_n\) (resp., f) be \((-\infty ,\infty )\)-valued \(\mu _n\)-integrable (resp., measurable) functions thereon, and \(g_n\) be nonnegative measurable functions on X. Suppose \(\mathop {\underline{\lim }}_{n\rightarrow \infty }\mu _n(\Gamma )\ge \mu (\Gamma )\) for each \(\Gamma \in \mathcal{B}(X)\), and \(f_n(x)\rightarrow f(x)\) for each \(x\in X.\) If \(|f_n(x)|\le g_n(x)\) for each \(x\in X\) and \(n\ge 1,\) and

$$\begin{aligned} \mathop {\overline{\lim }}_{n\rightarrow \infty } \int _X g_n(y)\mu _n(\mathrm{d}y)\le \int _X \mathop {\underline{\lim }}_{n\rightarrow \infty }g_n(y)\mu (\mathrm{d}y)<\infty , \end{aligned}$$

then f is \(\mu \)-integrable, and

$$\begin{aligned}&\lim _{n\rightarrow \infty } \int _X f_n(y)\mu _n(\mathrm{d}y)=\int _X f(y)\mu (\mathrm{d}y)\in (-\infty ,\infty ). \end{aligned}$$

Proof

See Theorem 2.4 of [24] \(\square \)

Proof of Lemma 2.1

Let us firstly justify that (b), (c) and (d) are equivalent. Clearly, (d) implies (b), which in turn implies (c). That (c) implies (b) is true because clearly \(t\rightarrow P(0,x,t,S)\) is monotone nonincreasing in \(t\ge 0\). Now suppose (b) is true. By the Chapman–Kolmogorov equation, we see that for each \(x\in S\) and \(0\le s\le t\),

$$\begin{aligned} P_q(s,y,t,S)=1 \end{aligned}$$
(40)

for almost all y with respect to \(P_q(0,x,s,\hbox {d}y)\). Now suppose for contradiction that (d) does not hold, so that there exist some \(0\le s<t\) and \(x\in S\) such that

$$\begin{aligned} P_q(s,x,t,S)<1. \end{aligned}$$
(41)

Then

$$\begin{aligned} 1= & {} P_q(0,x,t,S)=P_q(0,x,s,\{x\})P(s,x,t,S)+\int _{S\setminus \{x\}} P_q(0,x,s,\hbox {d}y)P_q(s,y,t,S)\\= & {} P_q(0,x,s,\{x\})P(s,x,t,S)+ P_q(0,x,s,S\setminus \{x\})\\&\quad <P_q(0,x,s,\{x\})+ P_q(0,x,s,S\setminus \{x\})=1, \end{aligned}$$

where the first equality and the last equality are by assumption that (b) holds, the second equality is by the Chapman–Kolmogorov equation, the third equality is by (40), and the inequality is by (41). This is a desired contradiction, and thus, assertions (b), (c) and (d) are equivalent.

It is clear that (a) implies (b), c.f. Proposition 2.3 and (6). That (b) implies (a) follows from the relation \( \{t_\infty =\infty \}=\bigcap _{n=1}^\infty \{X_n\in S\}, \) c.f. (6). The proof is completed. \(\square \)

Proof of Lemma 4.1

It is clear that \(\int _S g(y)q(\hbox {d}y|x,s)\) is well defined. For the second part of the statement, note that for each \(x\in S\) and \(s\ge 0\),

$$\begin{aligned} \int _S g(y)\tilde{q}(\hbox {d}y|x,s)-g(x)q_x(s)\ge & {} -||g||_f \int _S f(y)\tilde{q}(\hbox {d}y|x,s)-||g||_f f(x)q_x(s)\\= & {} -||g||_f \int _S f(y)q(\hbox {d}y|x,s)- 2||g||_ff(x)q_x(s) \\\ge & {} -||g||_f cf(x)- 2||g||_ff(x)q_x(s)\\= & {} -||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

where the second inequality is by the fact that f is a c-drift function. Thus,

$$\begin{aligned} \int _S g(y)q(\hbox {d}y|x,s)\ge -||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

the integral on the left-hand side being well defined. Similarly, one can check that

$$\begin{aligned} \int _S g(y)q(\hbox {d}y|x,s)\le ||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

as required. \(\square \)

Proof of Lemma 4.2

The first assertion follows from the proof of the relevant assertion in Theorem 3.1 of [15]. The second assertion follows from the first assertion because for each \(x\in S\) such that \(\int _{S}f(y)q(\hbox {d}y|x,s)\ge 0,\)

$$\begin{aligned} \left( \int _S f(y)q(\hbox {d}y|x,s)\right) ^+=\int _{S}f(y)q(\hbox {d}y|x,s) \le cf(x),\quad \forall ~s\ge 0, \end{aligned}$$

whereas for each \(x\in S\) such that \(\int _{S}f(y)q(\hbox {d}y|x,s)< 0,\)

$$\begin{aligned} \left( \int _S f(y)q(\hbox {d}y|x,s)\right) ^+=0 \le cf(x),\quad \forall ~s\ge 0. \end{aligned}$$

The statement is proved. \(\square \)

Lemma 6.1

Let f be a c-drift function with respect to the Q-function q on S. Suppose

$$\begin{aligned} \sup _{x\in S,~s\ge 0}\{q_x(s)\}<\infty . \end{aligned}$$

Then for each \(x\in S,\) \(0\le s\le t\), and f-bounded (measurable) function g on S,

$$\begin{aligned} \int _S g(z)P_q(s,x,t,\mathrm{d}z)=g(x)+\int _s^t \left( \int _Sg(z) q(\mathrm{d}z|y,u)\right) P_q(s,x,u,\mathrm{d}y)\mathrm{d}u. \end{aligned}$$

In particular,

$$\begin{aligned} P_q(s,x,t,S)=1,\quad \forall ~x\in S, ~s,t\in [0,\infty ),~s\le t. \end{aligned}$$
(42)

Proof

The first assertion immediately follows from Proposition 2.2 and Lemma 4.2. The last assertion follows because \(f\equiv 1\) is a c-drift function, and one can put \(g\equiv 1\) in the first assertion. \(\square \)

Proof of Lemma 4.3

We only need justify that (22) implies (23) as follows. Let some \(x \in S\) and \(t\in [0,\infty )\) be arbitrarily fixed. Since (22) holds by assumption,

$$\begin{aligned} \int _0^t \mathbb {E}^q_x\left[ \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) ^-\right] \hbox {d}s= & {} \int _0^t \mathbb {E}^q_x\left[ \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) ^+\right] \hbox {d}s\\&-\mathbb {E}^q_x[f(X_t)]+f(x)<\infty \end{aligned}$$

by Lemma 4.2. Consequently, (24) holds. According to the Fubini theorem,

$$\begin{aligned} \int _0^t \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s= \mathbb {E}^q_x\left[ \int _0^t \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) \hbox {d}s\right] \in (-\infty ,\infty ). \end{aligned}$$

Now for each \(0\le u\le t,\) (22) implies

$$\begin{aligned} \mathbb {E}_x^q[f(X_t)]-f(x)= & {} \mathbb {E}_x^q\left[ \int _0^t\int _S f(y)q(\hbox {d}y|X_s,s)\hbox {d}s\right] \\= & {} \mathbb {E}_x^q\left[ \int _0^u\int _S f(y)q(\hbox {d}y|X_s,s)\hbox {d}s\right] +\mathbb {E}_x^q\left[ \int _u^t\int _S f(y)q(dy|X_s,s)\hbox {d}s\right] \\= & {} \mathbb {E}_x^q[f(X_u)]-f(x)+\int _u^t\mathbb {E}_x^q\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s. \end{aligned}$$

All the expressions are finite. Now (23) follows. \(\square \)

Lemma 6.2

For a c-drift function f with respect to the Q-function on S that satisfies relation (22), it holds that for each \(d\in (-\infty ,\infty )\), \(x\in S\) and \(t\ge 0\),

$$\begin{aligned} \hbox {e}^{\mathrm{d}t} \mathbb {E}^q_x[f(X_t)]=f(x)+\int _0^t \mathrm{e}^{\mathrm{d}u} \left( \mathbb {E}^q_x\left[ \int _S f(y)q(\mathrm{d}y|X_u,u)\right] +d \mathbb {E}^q_x[f(X_u)]\right) \mathrm{d}u. \end{aligned}$$

Proof

Keeping in mind Lemmas 4.2 and 4.3 (and especially (24)), direct calculations give

$$\begin{aligned}&\int _0^t \hbox {e}^{\mathrm{d}s}\mathbb {E}_x^q\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s\\&\quad =\int _0^t \int _0^s \left( d \hbox {e}^{\mathrm{d}u}\hbox {d}u\right) \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s +\mathbb {E}^q_x[f(X_t)]-f(x)\\&\quad = \int _0^t d \hbox {e}^{\mathrm{d}u} \left( \int _u^t \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s\right) \hbox {d}u+\mathbb {E}^q_x[f(X_t)]-f(x)\\&\quad = \hbox {e}^{\mathrm{d}t} \mathbb {E}^q_x[f(X_t)](\hbox {e}^{\mathrm{d}t}-1)- \int _0^t d\hbox {e}^{\mathrm{d}u} \mathbb {E}^q_x[f(X_u)]\hbox {d}u+\mathbb {E}^q_x[f(X_t)]-f(x), \end{aligned}$$

where the first and the third equalities are by (23). All the expressions are finite. The statement now follows. \(\square \)

Lemma 6.3

Consider a c-drift function f with respect to the Q-function q on S. For each \(x\in S,\) \(s,t\ge 0\), \(s\le t\) and \(\Gamma \in \mathcal{B}(S)\), the following relation holds;

$$\begin{aligned} P_{q^f}(s,x,t,\Gamma )=\frac{\mathrm{e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) P_q(s,x,t,\mathrm{d}y). \end{aligned}$$

Proof

By Feller’s construction, c.f., (2) and (3), it suffices to show

$$\begin{aligned} \sum _{m=0}^n P_{q^f}^{(m)}(s,x,t,\Gamma )= & {} \frac{\mathrm{e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^n P^{(m)}_q(s,x,t,\hbox {d}y)\right) ,~x\in S,\nonumber \\&\qquad ~s,t\ge 0,~ s\le t,~\Gamma \in \mathcal{B}(S), \end{aligned}$$
(43)

as follows. Let \(x\in S,~s,t\ge 0,~ s\le t,~\Gamma \in \mathcal{B}(S)\) be arbitrarily fixed. Consider \(n=0.\) Then

$$\begin{aligned} \frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) P^{(0)}_q(s,x,t,\hbox {d}y)= & {} \frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \hbox {e}^{-\int _s^t q_x(v)\hbox {d}v} \delta _x(\hbox {d}y) \nonumber \\= & {} \delta _x(\Gamma ) \hbox {e}^{-\int _s^t q_x(v)\mathrm{d}v-c(t-s)}\nonumber \\= & {} P_{q^f}^{(0)}(s,x,t,\Gamma ). \end{aligned}$$
(44)

Now assume that (43) holds for \(n=k\). Then

$$\begin{aligned}&\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^{k+1} P^{(m)}_q(s,x,t,\hbox {d}y)\right) \\&\quad =P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=1}^{k+1} P^{(m)}_q(s,x,t,\hbox {d}y)\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^{k} P^{(m+1)}_q(s,x,t,\hbox {d}y)\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \\&\qquad \quad \left( \sum _{m=0}^{k}\int _s^t \hbox {e}^{-\int _s^u q_x(v)\hbox {d}v} \left( \int _S \tilde{q}(dz|x,u) P^{(m)}_q(u,z,t,\hbox {d}y)\right) \hbox {d}u\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\int _{s}^t \hbox {e}^{-\int _s^u (q_x(v)+c)\hbox {d}v} \int _S \frac{f(z)}{f(x)} \tilde{q}(dz|x,u) \\&\quad \qquad \left( \int _\Gamma \frac{f(y)}{f(z)} \hbox {e}^{-c(t-u)}\sum _{m=0}^k P^{(m)}_q(u,z,t,\hbox {d}y) \right) \hbox {d}u\\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\sum _{m=0}^k P_{q^f}^{(m+1)}(s,x,t,\Gamma ), \end{aligned}$$

where the first equality is by (44), the third equality is by (2), and the last equality is by (2), the inductive supposition and (7). Now (43) holds for \(n=k+1\), and thus for all \(n=0,1,\ldots \) by induction. \(\square \)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y. On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes. J Theor Probab 31, 1322–1355 (2018). https://doi.org/10.1007/s10959-017-0763-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10959-017-0763-3

Keywords

Mathematics Subject Classification (2010)

Navigation