On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes

Yi Zhang

doi:10.1007/s10959-017-0763-3

Journal of Theoretical Probability

September 2018, Volume 31, Issue 3, pp 1322–1355 | Cite as

On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes

Authors
Authors and affiliations

Yi Zhang

Article

First Online: 25 April 2017

100 Downloads

Abstract

In this paper, we obtain new drift-type conditions for nonexplosion and explosion for nonhomogeneous Markov pure jump processes in Borel state spaces. The conditions are sharp; e.g., the one for nonexplosion is necessary if the state space is in addition locally compact and the Q-function satisfies weak Feller-type and local boundedness conditions. We comment on the relations of our conditions with the existing ones in the literature and demonstrate some possible applications.

Keywords

Dynkin’s formula Nonhomogeneous Markov pure jump process Nonexplosion

Mathematics Subject Classification (2010)

60J75 90C40

This is a preview of subscription content, to check access.

Notes

Acknowledgements

I would like to thank Professor Mufa Chen (Beijing Normal University) for providing the scan copy of the relevant pages in [30] and the paper [29]. I also thank the referees for their helpful comments and remarks. This work was carried out with a financial grant from the Research Fund for Coal and Steel of the European Commission, within the INDUSE-2-SAFETY project (Grant No. RFSR-CT-2014-00025).

Appendix

In this appendix, we present some auxiliary results and the proofs of some lemmas stated in the previous sections.

Proposition 6.1

Let $\mu _n$ and $\mu $ be $[0,\infty ]$-valued measures on the Borel space X, and $f_n$ (resp., f) be $(-\infty ,\infty )$-valued $\mu _n$-integrable (resp., measurable) functions thereon, and $g_n$ be nonnegative measurable functions on X. Suppose $\mathop {\underline{\lim }}_{n\rightarrow \infty }\mu _n(\Gamma )\ge \mu (\Gamma )$ for each $\Gamma \in \mathcal{B}(X)$, and $f_n(x)\rightarrow f(x)$ for each $x\in X.$ If $|f_n(x)|\le g_n(x)$ for each $x\in X$ and $n\ge 1,$ and

$$\begin{aligned} \mathop {\overline{\lim }}_{n\rightarrow \infty } \int _X g_n(y)\mu _n(\mathrm{d}y)\le \int _X \mathop {\underline{\lim }}_{n\rightarrow \infty }g_n(y)\mu (\mathrm{d}y)<\infty , \end{aligned}$$

then f is $\mu $-integrable, and

$$\begin{aligned}&\lim _{n\rightarrow \infty } \int _X f_n(y)\mu _n(\mathrm{d}y)=\int _X f(y)\mu (\mathrm{d}y)\in (-\infty ,\infty ). \end{aligned}$$

Proof

See Theorem 2.4 of [24] $\square $

Proof of Lemma 2.1

Let us firstly justify that (b), (c) and (d) are equivalent. Clearly, (d) implies (b), which in turn implies (c). That (c) implies (b) is true because clearly $t\rightarrow P(0,x,t,S)$ is monotone nonincreasing in $t\ge 0$. Now suppose (b) is true. By the Chapman–Kolmogorov equation, we see that for each $x\in S$ and $0\le s\le t$,

$$\begin{aligned} P_q(s,y,t,S)=1 \end{aligned}$$

(40)

for almost all y with respect to $P_q(0,x,s,\hbox {d}y)$. Now suppose for contradiction that (d) does not hold, so that there exist some $0\le s<t$ and $x\in S$ such that

$$\begin{aligned} P_q(s,x,t,S)<1. \end{aligned}$$

(41)

Then

$$\begin{aligned} 1= & {} P_q(0,x,t,S)=P_q(0,x,s,\{x\})P(s,x,t,S)+\int _{S\setminus \{x\}} P_q(0,x,s,\hbox {d}y)P_q(s,y,t,S)\\= & {} P_q(0,x,s,\{x\})P(s,x,t,S)+ P_q(0,x,s,S\setminus \{x\})\\&\quad <P_q(0,x,s,\{x\})+ P_q(0,x,s,S\setminus \{x\})=1, \end{aligned}$$

where the first equality and the last equality are by assumption that (b) holds, the second equality is by the Chapman–Kolmogorov equation, the third equality is by (40), and the inequality is by (41). This is a desired contradiction, and thus, assertions (b), (c) and (d) are equivalent.

It is clear that (a) implies (b), c.f. Proposition 2.3 and (6). That (b) implies (a) follows from the relation $ \{t_\infty =\infty \}=\bigcap _{n=1}^\infty \{X_n\in S\}, $ c.f. (6). The proof is completed. $\square $

Proof of Lemma 4.1

It is clear that $\int _S g(y)q(\hbox {d}y|x,s)$ is well defined. For the second part of the statement, note that for each $x\in S$ and $s\ge 0$,

$$\begin{aligned} \int _S g(y)\tilde{q}(\hbox {d}y|x,s)-g(x)q_x(s)\ge & {} -||g||_f \int _S f(y)\tilde{q}(\hbox {d}y|x,s)-||g||_f f(x)q_x(s)\\= & {} -||g||_f \int _S f(y)q(\hbox {d}y|x,s)- 2||g||_ff(x)q_x(s) \\\ge & {} -||g||_f cf(x)- 2||g||_ff(x)q_x(s)\\= & {} -||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

where the second inequality is by the fact that f is a c-drift function. Thus,

$$\begin{aligned} \int _S g(y)q(\hbox {d}y|x,s)\ge -||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

the integral on the left-hand side being well defined. Similarly, one can check that

$$\begin{aligned} \int _S g(y)q(\hbox {d}y|x,s)\le ||g||_f f(x)(c+2q_x(s)), \end{aligned}$$

as required. $\square $

Proof of Lemma 4.2

The first assertion follows from the proof of the relevant assertion in Theorem 3.1 of [15]. The second assertion follows from the first assertion because for each $x\in S$ such that $\int _{S}f(y)q(\hbox {d}y|x,s)\ge 0,$

$$\begin{aligned} \left( \int _S f(y)q(\hbox {d}y|x,s)\right) ^+=\int _{S}f(y)q(\hbox {d}y|x,s) \le cf(x),\quad \forall ~s\ge 0, \end{aligned}$$

whereas for each $x\in S$ such that $\int _{S}f(y)q(\hbox {d}y|x,s)< 0,$

$$\begin{aligned} \left( \int _S f(y)q(\hbox {d}y|x,s)\right) ^+=0 \le cf(x),\quad \forall ~s\ge 0. \end{aligned}$$

The statement is proved. $\square $

Lemma 6.1

Let f be a c-drift function with respect to the Q-function q on S. Suppose

$$\begin{aligned} \sup _{x\in S,~s\ge 0}\{q_x(s)\}<\infty . \end{aligned}$$

Then for each $x\in S,$ $0\le s\le t$, and f-bounded (measurable) function g on S,

$$\begin{aligned} \int _S g(z)P_q(s,x,t,\mathrm{d}z)=g(x)+\int _s^t \left( \int _Sg(z) q(\mathrm{d}z|y,u)\right) P_q(s,x,u,\mathrm{d}y)\mathrm{d}u. \end{aligned}$$

In particular,

$$\begin{aligned} P_q(s,x,t,S)=1,\quad \forall ~x\in S, ~s,t\in [0,\infty ),~s\le t. \end{aligned}$$

(42)

Proof

The first assertion immediately follows from Proposition 2.2 and Lemma 4.2. The last assertion follows because $f\equiv 1$ is a c-drift function, and one can put $g\equiv 1$ in the first assertion. $\square $

Proof of Lemma 4.3

We only need justify that (22) implies (23) as follows. Let some $x \in S$ and $t\in [0,\infty )$ be arbitrarily fixed. Since (22) holds by assumption,

$$\begin{aligned} \int _0^t \mathbb {E}^q_x\left[ \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) ^-\right] \hbox {d}s= & {} \int _0^t \mathbb {E}^q_x\left[ \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) ^+\right] \hbox {d}s\\&-\mathbb {E}^q_x[f(X_t)]+f(x)<\infty \end{aligned}$$

by Lemma 4.2. Consequently, (24) holds. According to the Fubini theorem,

$$\begin{aligned} \int _0^t \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s= \mathbb {E}^q_x\left[ \int _0^t \left( \int _S f(y)q(\hbox {d}y|X_s,s)\right) \hbox {d}s\right] \in (-\infty ,\infty ). \end{aligned}$$

Now for each $0\le u\le t,$ (22) implies

$$\begin{aligned} \mathbb {E}_x^q[f(X_t)]-f(x)= & {} \mathbb {E}_x^q\left[ \int _0^t\int _S f(y)q(\hbox {d}y|X_s,s)\hbox {d}s\right] \\= & {} \mathbb {E}_x^q\left[ \int _0^u\int _S f(y)q(\hbox {d}y|X_s,s)\hbox {d}s\right] +\mathbb {E}_x^q\left[ \int _u^t\int _S f(y)q(dy|X_s,s)\hbox {d}s\right] \\= & {} \mathbb {E}_x^q[f(X_u)]-f(x)+\int _u^t\mathbb {E}_x^q\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s. \end{aligned}$$

All the expressions are finite. Now (23) follows. $\square $

Lemma 6.2

For a c-drift function f with respect to the Q-function on S that satisfies relation (22), it holds that for each $d\in (-\infty ,\infty )$, $x\in S$ and $t\ge 0$,

$$\begin{aligned} \hbox {e}^{\mathrm{d}t} \mathbb {E}^q_x[f(X_t)]=f(x)+\int _0^t \mathrm{e}^{\mathrm{d}u} \left( \mathbb {E}^q_x\left[ \int _S f(y)q(\mathrm{d}y|X_u,u)\right] +d \mathbb {E}^q_x[f(X_u)]\right) \mathrm{d}u. \end{aligned}$$

Proof

Keeping in mind Lemmas 4.2 and 4.3 (and especially (24)), direct calculations give

$$\begin{aligned}&\int _0^t \hbox {e}^{\mathrm{d}s}\mathbb {E}_x^q\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s\\&\quad =\int _0^t \int _0^s \left( d \hbox {e}^{\mathrm{d}u}\hbox {d}u\right) \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s +\mathbb {E}^q_x[f(X_t)]-f(x)\\&\quad = \int _0^t d \hbox {e}^{\mathrm{d}u} \left( \int _u^t \mathbb {E}^q_x\left[ \int _S f(y)q(\hbox {d}y|X_s,s)\right] \hbox {d}s\right) \hbox {d}u+\mathbb {E}^q_x[f(X_t)]-f(x)\\&\quad = \hbox {e}^{\mathrm{d}t} \mathbb {E}^q_x[f(X_t)](\hbox {e}^{\mathrm{d}t}-1)- \int _0^t d\hbox {e}^{\mathrm{d}u} \mathbb {E}^q_x[f(X_u)]\hbox {d}u+\mathbb {E}^q_x[f(X_t)]-f(x), \end{aligned}$$

where the first and the third equalities are by (23). All the expressions are finite. The statement now follows. $\square $

Lemma 6.3

Consider a c-drift function f with respect to the Q-function q on S. For each $x\in S,$ $s,t\ge 0$, $s\le t$ and $\Gamma \in \mathcal{B}(S)$, the following relation holds;

$$\begin{aligned} P_{q^f}(s,x,t,\Gamma )=\frac{\mathrm{e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) P_q(s,x,t,\mathrm{d}y). \end{aligned}$$

Proof

By Feller’s construction, c.f., (2) and (3), it suffices to show

$$\begin{aligned} \sum _{m=0}^n P_{q^f}^{(m)}(s,x,t,\Gamma )= & {} \frac{\mathrm{e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^n P^{(m)}_q(s,x,t,\hbox {d}y)\right) ,~x\in S,\nonumber \\&\qquad ~s,t\ge 0,~ s\le t,~\Gamma \in \mathcal{B}(S), \end{aligned}$$

(43)

as follows. Let $x\in S,~s,t\ge 0,~ s\le t,~\Gamma \in \mathcal{B}(S)$ be arbitrarily fixed. Consider $n=0.$ Then

$$\begin{aligned} \frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) P^{(0)}_q(s,x,t,\hbox {d}y)= & {} \frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \hbox {e}^{-\int _s^t q_x(v)\hbox {d}v} \delta _x(\hbox {d}y) \nonumber \\= & {} \delta _x(\Gamma ) \hbox {e}^{-\int _s^t q_x(v)\mathrm{d}v-c(t-s)}\nonumber \\= & {} P_{q^f}^{(0)}(s,x,t,\Gamma ). \end{aligned}$$

(44)

Now assume that (43) holds for $n=k$. Then

$$\begin{aligned}&\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^{k+1} P^{(m)}_q(s,x,t,\hbox {d}y)\right) \\&\quad =P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=1}^{k+1} P^{(m)}_q(s,x,t,\hbox {d}y)\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \left( \sum _{m=0}^{k} P^{(m+1)}_q(s,x,t,\hbox {d}y)\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\frac{\hbox {e}^{-c(t-s)}}{f(x)}\int _\Gamma f(y) \\&\qquad \quad \left( \sum _{m=0}^{k}\int _s^t \hbox {e}^{-\int _s^u q_x(v)\hbox {d}v} \left( \int _S \tilde{q}(dz|x,u) P^{(m)}_q(u,z,t,\hbox {d}y)\right) \hbox {d}u\right) \\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\int _{s}^t \hbox {e}^{-\int _s^u (q_x(v)+c)\hbox {d}v} \int _S \frac{f(z)}{f(x)} \tilde{q}(dz|x,u) \\&\quad \qquad \left( \int _\Gamma \frac{f(y)}{f(z)} \hbox {e}^{-c(t-u)}\sum _{m=0}^k P^{(m)}_q(u,z,t,\hbox {d}y) \right) \hbox {d}u\\&\quad = P_{q^f}^{(0)}(s,x,t,\Gamma )+\sum _{m=0}^k P_{q^f}^{(m+1)}(s,x,t,\Gamma ), \end{aligned}$$

where the first equality is by (44), the third equality is by (2), and the last equality is by (2), the inductive supposition and (7). Now (43) holds for $n=k+1$, and thus for all $n=0,1,\ldots $ by induction. $\square $

References

1.
Anderson, W.: Continuous-Time Markov Chains. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
2.
Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, Berlin (2011)CrossRefzbMATHGoogle Scholar
3.
Bertsekas, D., Shreve, S.: Stochastic Optimal Control. Academic Press, New York (1978)zbMATHGoogle Scholar
4.
Blok, H., Spieksma, F.: Countable state Markov decision processes with unbounded jump rates and discounted optimality equation and approximations. Adv. Appl. Probab. 47, 1088–1107 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
5.
Chen, M.: Coupling for jump processes. Acta Math. Sin. (Engl. Ser.) 2, 123–136 (1986)MathSciNetzbMATHGoogle Scholar
6.
Chen, M.: A comment on the book “Continuous-time Markov Chains” by W. J. Anderson. Chin. J. Appl. Probab. Stat. 12, 55–59 (1996). Updated version also available at arXiv:1412.5856
7.
Chen, M.: From Markov Chains to Non-Equilibrium Particle Systems. World Scientific, Singapore (2004)CrossRefzbMATHGoogle Scholar
8.
Chen, M.: Eigenvalues, Inequalities, and Ergodic Theory. Springer, London (2005)zbMATHGoogle Scholar
9.
Chen, M.: Practical criterion for uniqueness of q-processes. Chin. J. Appl. Probab. Stat. 31, 213–224 (2015). arXiv:1503.02119
10.
Chow, P., Khasminskii, R.: Methods of Lyapunov functions for analysis of absorption and explosion in Markov chains. Probl. Inf. Transm. 47, 232–250 (2011). (Translation of the original Russian text at Problemy Peredachi Informatsii 47, 19–38 (2011))Google Scholar
11.
Eibeck, A., Wagner, W.: Stochastic interacting particle systems and nonlinear kinetic equations. Ann. Appl. Probab. 13, 845–889 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
12.
Feinberg, E., Mandava, M., Shiryaev, A.: On solutions of Kolmogorov’s equations for nonhomogeneous jump Markov processes. J. Math. Anal. Appl. 411, 261–270 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
13.
Feinberg, E., Mandava, M., Shiryaev, A.: Kolmogorov’s equations for jump Markov processes with unbounded jump rates (2016). arXiv:1603.02367
14.
Feller, W.: On the integro-differential equations of purely discontinuous Markoff processes. Trans. Am. Math. Soc. 48, 488–515 (1940)MathSciNetCrossRefzbMATHGoogle Scholar
15.
Guo, X.: Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Oper. Res. 32, 73–87 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
16.
Hu, D.: Analytic Theory of Markov Processes in General State Spaces. Wuhan University Press, Wuhan (2013). Republication of the edition originally published in 1985 (in Chinese) Google Scholar
17.
Jacod, J.: Multivariate point processes: predictable projection, Radon–Nykodym derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie verw. Gebite. 31, 235–253 (1975)CrossRefzbMATHGoogle Scholar
18.
Kitaev, M., Rykov, V.: Controlled Queueing Systems. CRC Press, Boca Raton (1995)zbMATHGoogle Scholar
19.
Menshikov, M., Petritis, D.: Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach. Stoch. Proc. Appl. 124, 2388–2414 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
20.
Meyn, S., Tweedie, R.: Markov Chains and Stochastic Stability. Springer, London (1993)CrossRefzbMATHGoogle Scholar
21.
Meyn, S., Tweedie, R.: Stability of Markov processes III: Forster–Lyapunov criteria for continous time processes. Adv. Appl. Probab. 25, 518–548 (1993)CrossRefzbMATHGoogle Scholar
22.
Piunovskiy, A., Zhang, Y.: Discounted continuous-time markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-Q. J. Oper. Res. 12, 49–75 (2014). The extended version is available at arXiv:1103.0134
23.
Reuter, G.: Denumerable Markov processes and the associated contraction semigroups on $l$. Acta Math. 97, 1–46 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
24.
Serfozo, R.: Convergence of Lebesgue integrals with varying measures. Sankhya 44, 380–402 (1982)MathSciNetzbMATHGoogle Scholar
25.
Spieksma, F.: Countable state Markov processes: non-explosiveness and moment function. Probab. Eng. Inf. Sci. 29, 623–637 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
26.
Spieksma, F.: Kolmogorov forward equation and explosiveness in countable state Markov processes. Ann. Oper. Res. 241, 3–22 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
27.
Wagner, W.: Explosion phenomena in stochastic coagulation-gragmentation models. Ann. Appl. Probab. 15, 2081–2112 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
28.
Ye, L., Guo, X.: Construction and regularity of transition functions on Polish spaces under measurability conditions. Acta Math. Appl. Sin. (Engl. Ser.) 29, 1–14 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
29.
Zheng, J., Zheng, X.: A martingale approadch to $q$-processes. Kexue Tongbao (Chin. Sci. Bull.) 32, 1457–1459 (1987)zbMATHGoogle Scholar
30.
Zheng, J.: Phase Transitions of Ising Model on Lattice Fractals, Martingale Approach for $q$-Processes. Ph.D. thesis. Beijing Normal University (1993) (in Chinese) Google Scholar

Copyright information

Authors and Affiliations

Yi Zhang
- 1
Email author

1.Department of Mathematical SciencesUniversity of LiverpoolLiverpoolUK

On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes

Abstract

Keywords

Mathematics Subject Classification (2010)

Notes

Acknowledgements

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article