Nonparametric Estimation of Mean Residual Life Function Using Scale Mixtures

Abstract

It is often of interest in clinical trials and reliability studies to estimate the remaining lifetime of a subject or a device given that it survived up to a given period of time, that is commonly known as the so-called mean residual life function (mrlf). There have been several attempts in literature to estimate the mrlf nonparametrically ranging from empirical estimates to more sophisticated smooth estimation. Given the well known one-to-one relation between survival function and mrlf, one can plug-in any known estimates of the survival function (e.g., Kaplan–Meier estimate) into the functional form of mrlf to obtain an estimate of mrlf. In this chapter, we present a scale mixture representation of mrlf and use it to obtain a smooth estimate of the mrlf under right censoring. Asymptotic properties of the proposed estimator are also presented. Several simulation studies and a real data set are used for investigating the empirical performance of the proposed method relative to other well-known estimates of mrlf. A comparative analysis shows computational advantages of the proposed estimator in addition to somewhat superior statistical properties in terms of bias and efficiency.

Appendices

Appendix

A: Characterization of MRLF

Lemma 8.1

Let h(t) be the hazard function of a lifetime random variable T. Then the mrlf, m(t) of T is differentiable and is obtained by solving the following differential equation:

$$\begin{aligned} m^\prime (t) + 1=m(t)h(t)\; \text{ for } \text{ all } t>0, \end{aligned}$$

(8.7)

where $m^\prime (\cdot )$ denotes the derivative of $m(\cdot )$.

Proof

From (8.1), it follows that $m(t)S(t)=\int _t^\infty S(u) du$. The result follows by differentiating both sides of the previous identity with respect to (wrt) t and using the definition of $h(t)=-S^\prime (t)/S(t)$.

It thus follows (8.7) that given h(t) we can obtain

$$\begin{aligned} m(t)=\int _t^\infty \exp \left\{ -\int _t^u h(v)dv\right\} du, \end{aligned}$$

(8.8)

and given m(t) we can obtain

$$\begin{aligned} h(t)={m^\prime (t)+1\over m(t)}. \end{aligned}$$

(8.9)

Also under mild regularity conditions, an mrlf completely determines the distribution as can be seen by the following characterization result:

Theorem 8.3

(Hall-Wellner 1981) Let $m:\mathbb {R}^+\rightarrow \mathbb {R}^+$ be a function that satisfies the following conditions:

• (i) m(t) is right continuous and $m(0)>0$;

• (ii) $e(t)\equiv m(t)+t$ is non-decreasing;

• (iii) if $m(t-)=0$ for some $t=t_0$, then $m(t)=0$ on $[t_0,\infty )$;

• (iv) if $m(t-)>0$ for all t, then $\int _0^\infty 1/m(u)du=\infty .$

Let $\tau \equiv \inf \{t: m(t-)=0\}\le \infty$, and define

$$\begin{aligned} S(t) =\frac{m(0)}{m(t)}\exp \left\{ -\int ^t_0\frac{1}{m(u)}du\right\} . \end{aligned}$$

Then $F(t)=1-S(t)$ is a cdf on $\mathbb {R}^+$ with $F(0)=0$ and $\tau =\inf \{t: F(t)=1\}$, finite mean m(0) and mrlf m(t).

Proof

See Hall and Wellner (1981), p. xxx.

B: Chaubey-Sen’s (1999, 2008) Estimator

Given a sample of n iid positive values random variables, $T_i{\mathop {\sim }\limits ^{iid}}F(\cdot )$ for $i=1,\ldots ,n$, the empirical survival function $S_n$ is defined by

$$\begin{aligned} S_n(t)={1\over n}\sum _{i=1}^n\mathbb {I}(T_i>t) \end{aligned}$$

Define a set of nonnegative valued Poisson weights as

$$\begin{aligned} w_k(t\lambda _n)=e^{-t\lambda _n}{(t\lambda _n)^k\over k!},\; k=0,1,2,\ldots , \end{aligned}$$

where $\lambda _n=n/T_{(n)}$ and $T_{(n)}=\max _{1\le i\le n}T_i$. Notice that $\lambda _n$ is chosen data-dependent, which makes the weight function stochastic. A smoothed empirical survival function can be obtained as

$$\begin{aligned} \tilde{S}_n(t)=\sum _{k\ge 0} S_n{\left( k\over \lambda _n\right) }w_k(t\lambda _n). \end{aligned}$$

Plugging the smoothed empirical survival function into (8.3), the smooth estimator of the mrlf is given by

$$\begin{aligned} \tilde{m}_n(t)={1\over \lambda _n}{\sum _{k=0}^n\sum _{r=0}^k((t\lambda _n)^{(k-r)}/ (k-r)!)S_n(k/\lambda _n)\over \sum _{(k=0)}((t\lambda _n)^k/k!)S_n(k/\lambda _n)}. \end{aligned}$$

(8.10)

Chaubey and Sen (1999) proved that $\tilde{m}_n(t)$ is a consistent estimator of m(t) and $\lambda _n^{1/2}(\tilde{m}_n(t)-m(t)){\mathop {\rightarrow }\limits ^{d}}N\left( 0, {m(t)\over S(t)}\right)$ pointwise. Later Chaubey and Sen (2008) extended their methodology for the censored case and derived the asymptotic properties.

C: The Details of the Calculation of $\hat{m}_m(t)$

$\hat{m}_m(t)$, the smooth estimator of m(t) for censored data (complete data are special case of censored data where weight function $w_j={1\over n}$), can be calculated with a closed form. The details of the calculation are given here.

$$\begin{aligned}&\hat{m}_m(t)=\int _0^\infty \hat{m}_e(u)f\left( u\bigg |k_n, {t\over k_n}\right) du =\sum _{l=0}^{n-1}\int _{X_l}^{X_{l+1}}\hat{m}_e(u)f\left( u\bigg |k_n, {t\over k_n}\right) du\\= & {} \sum _{l=0}^{n-1}\left[ {\sum _{j=l+1}^n X_j w_j\over \sum _{j=l+1}^n w_j} \int _{X_l}^{X_{l+1}} f\left( u\bigg |k_n, {t\over k_n}\right) du -{\Gamma (k_n+1)\over \Gamma (k_n)}{t\over k_n}\int _{X_l}^{X_{l+1}} {u^{k_n}\left( {t\over k_n}\right) ^{k_n+1}\over \Gamma (k_n+1)} e^{- ( {t\over k_n})u}du\right] \\= & {} \sum _{l=0}^{n-1}\left[ {\sum _{j=l+1}^n X_j w_j\over \sum _{j=l+1}^n w_j} \int _{X_l}^{X_{l+1}} f\left( u\bigg |k_n, {t\over k_n}\right) du -t\int _{X_l}^{X_{l+1}}f\left( u\bigg |k_n+1, {t\over k_n}\right) du\right] \\= & {} \sum _{l=0}^{n-1}{\sum _{j=l+1}^n X_j w_j\over \sum _{j=l+1}^n w_j} \left[ F\left( X_{l+1}\bigg |k_n, {t\over k_n}\right) - F\left( X_{l}\bigg |k_n, {t\over k_n}\right) \right] \\&-\sum _{l=0}^{n-1}t\left[ F\left( X_{l+1}\bigg |k_n+1, {t\over k_n}\right) - F\left( X_{l}\bigg |k_n+1, {t\over k_n}\right) \right] \\= & {} \sum _{l=0}^{n-1}{\sum _{j=l+1}^n X_j w_j\over \sum _{j=l+1}^n w_j} \left[ F\left( X_{l+1}\bigg |k_n, {t\over k_n}\right) -F\left( X_{l}\bigg |k_n, {t\over k_n}\right) \right] \\&-t\left[ F\left( X_{n}\bigg |k_n+1, {t\over k_n}\right) - F\left( X_{0}\bigg |k_n+1, {t\over k_n}\right) \right] \\= & {} \sum _{l=0}^{n-1}{\sum _{j=l+1}^n X_j w_j\over \sum _{j=l+1}^n w_j} \left[ F\left( X_{l+1}\bigg |k_n, {t\over k_n}\right) - F\left( X_{l}\bigg |k_n, {t\over k_n}\right) \right] -tF\left( X_{n}\bigg |k_n+1, {t\over k_n}\right) , \end{aligned}$$

where $w_j=\hat{S}(X_j)-\hat{S}(X_j-)$ and $\hat{S}(\cdot )$ is the KM estimate.

D: Proof of Theorem 8.2

(a) Proof of consistency

First we prove part (i) of Theorem 8.2 by showing that $\hat{m}_m(t)$ is pointwise consistent for estimating m(t). The main tool to prove consistency is based on the following well known approximation result originally due to Feller [7], p. xxx but extended by Petrone and Veronese [22] for a wider application:

Lemma 8.2

Feller Approximation Lemma (Petrone and Veronese [22]):

Let $g(\cdot )$ be a bounded and right continuous function on $\mathbb {R}$ for each t. Let $Z_k(t)$ be a sequence of random variables for each t such that $\mu _k(t)\equiv E[Z_k(t)]\rightarrow t$ and $\sigma ^2_k(t)\equiv Var[Z_k(t)]\rightarrow 0$ as $k\rightarrow \infty$. Then

$$\begin{aligned} E[g(Z_k(t))]\rightarrow g(t)\;\;\forall t. \end{aligned}$$

First, notice that by definition, $\hat{m}_e(\cdot )$ is a right continuous function on $[0, T_{(n)}]$ and $\hat{m}_e(t)=0$ for $t>T_{(n)}$ and hence $\hat{m}_e(\cdot )$ is a bounded function on $[0, \infty )$.

Let $Z_{n}(t)\sim Ga(k_n,{t\over k_n})$ for $t>0$, where $Ga(k_n,{t\over k_n})$ denotes a Gamma distribution with mean $\mu _n(t)=t$ and variance $\sigma ^2_{n}(t)={t^2\over k_n}$ and the density function is given by

$$f_{k_n,t}(z)=f\left( z\Big |k_n,{t\over k_n}\right) =\left( {k_n\over t}\right) ^{k_n} {1\over \Gamma ({k_n})} z^{k_n-1}e^{-{k_nz\over t}}.$$

It easily follows that we can write our scale mixture estimator as

$$\begin{aligned} \hat{m}_m(t)= & {} \int _0^\infty {\hat{m}_e(t\theta )\over \theta }\pi (\theta |k_n)d\theta \nonumber \\= & {} \int _0^\infty \hat{m}_e(u)f\left( u\Big |k_n,{t\over k_n}\right) du \nonumber \\= & {} E[\hat{m}_e(Z_n(t))], \end{aligned}$$

(8.11)

Thus, by Feller approximation result in Lemma 8.2 it follows that $E[m(Z_n(t))]$ converges (pointwise) to m(t) if we choose the sequence $k_n$ such that $k_n\rightarrow \infty$ as $n\rightarrow \infty$. Next, by the consistency of $\hat{m}_e(t)$ and Dominated Convergence Theorem it follows that $E[\hat{m}_e(Z_n(t))-m(Z_n(t))]\rightarrow 0$ and $n\rightarrow \infty$. Hence it follows that $\hat{m}_m(t)$ converges (pointwise) in probability to m(t) as $k_n\rightarrow \infty$. This completes the proof of (i) in Theorem 8.2.

(b) Proof of Asymptotic Normality

First, notice that we can write,

$$\begin{aligned} \sqrt{n}(\hat{m}_m(t)-m(t))=\sqrt{n}(\hat{m}_m(t)-\hat{m}_e(t))+\sqrt{n}(\hat{m}_e(t)-m(t)), \end{aligned}$$

(8.12)

where it is known from previous literature that $\sqrt{n}(\hat{m}_e(t)-m(t))\sim GP(0,\sigma (\cdot ,\cdot ))$ and expression of the covariance function $\sigma (\cdot ,\cdot )$ are as given in the statement of Theorem 8.2. Thus, it is sufficient to establish that first term in Eq. 8.12 converges in probability to zero as $n\rightarrow$ by choosing a suitable growth rate of the $k_n$ sequence as a function of n.

Using the scale mixture formulation it follows that $\sqrt{n}(\hat{m}_m(t)-\hat{m}_e(t))=\sqrt{n}E[{\hat{m}_e(tY_n)\over Y_n}-\hat{m}_e(t)]$ where $Y_n\sim Ga(k_n+1,1/k_n)$

We use a first order Taylor’s expansion around t to write

$$\begin{aligned} {\hat{m}_e(tY_n)\over Y_n}={\hat{m}_e(t)\over Y_n}+{(tY_n-t)\hat{m}_e^\prime (t_n^*)\over Y_n}, \end{aligned}$$

where $t_n^*{\mathop {\rightarrow }\limits ^{P}}t$ as $n\rightarrow \infty$ because $Y_n{\mathop {\rightarrow }\limits ^{p}} 1$. It can be seen from (8.5) that $\hat{m}_e(t)$ is differentiable if $t\in (X_l,X_{l+1}),l=1,\ldots ,n-1$ and $\hat{m}_e(t)$ is not differentiable if $t=X_l,l=1,\ldots ,n-1$. Thus, $\hat{m}_e^\prime (t_n^*)$ exists for $t_n^*\in [0,\infty )\backslash \{X_1,\ldots X_n\}$.

Second, we substitute the Taylor’s expansion into $\sqrt{n}E[{\hat{m}_e(tY_n)\over Y_n}-\hat{m}_e(t)]$

$$\begin{aligned} \sqrt{n}(\hat{m}_m(t)-\hat{m}_e(t))= & {} \sqrt{n}E[{\hat{m}_e(tY_n)\over Y_n}-\hat{m}_e(t)]\\= & {} \sqrt{n}E\left[ {\hat{m}_e(t)\over Y_n}-\hat{m}_e(t)+{(tY_n-t)\hat{m}_e\prime (t_n^*)\over Y_n}\right] \\= & {} E\left[ \sqrt{n}\left( {1\over Y_n}-1\right) \hat{m}_e(t)-\sqrt{n}\left( {1\over Y_n}-1\right) t\hat{m}_e\prime (t_n^*)\right] \end{aligned}$$

In order to complete the proof it is now sufficient to show that $\sqrt{n}\left( {1\over Y_n}-1\right) {\mathop {\rightarrow }\limits ^{P}}0$ as $n\rightarrow \infty$. However, as $Y_n\sim Ga(k_n+1, 1/k_n)$, it follows that

$$E\left[ \sqrt{n}\left( {1\over Y_n}-1\right) \right] =\sqrt{n}\left( E\left[ {1\over Y_n}\right] -1\right) =\sqrt{n}\left( {k_n\over k_n+1-1}-1\right) =0$$

and

$$Var\left[ \sqrt{n}\left( {1\over Y_n}-1\right) \right] =n\;Var \left[ {1\over Y_n}\right] =n{k_n^2\over k_n^2(k_n-1)}={n\over k_n-1}.$$

Thus id we assume that $k_n/n\rightarrow \infty$, i.e., $k_n=cn^{1+\epsilon }$ where $\epsilon >0$, it can be seen that $var\left[ \sqrt{n}\left( {1\over Y_n}-1\right) \right] \rightarrow 0$ as $n\rightarrow \infty$. Therefore, $\sqrt{n}\left( {1\over Y_n}-1\right) {\mathop {\rightarrow }\limits ^{P}}0$ as $n\rightarrow \infty$ and this completes the proof of Theorem 8.2.

Remark 8.1

Notice that the asymptotic properties of $\hat{m}_m(t)$ are the same as those of the estimator of Yang [26] for complete data and the same as those of the estimator of Ghorai et al. [8] for censored data.