A hybrid Newton-type method for censored survival data using double weights in linear models

Abstract

As an alternative to the Cox model, the rank-based estimating method for censored survival data has been studied extensively since it was proposed by Tsiatis [Tsiatis AA (1990) Ann Stat 18:354–372] among others. Due to the discontinuity feature of the estimating function, a significant amount of work in the literature has been focused on numerical issues. In this article, we consider the computational aspects of a family of doubly weighted rank-based estimating functions. This family is rich enough to include both estimating functions of Tsiatis (1990) for the randomly observed data and of Nan et al. [Nan B, Yu M, Kalbfleisch JD (2006) Biometrika (to appear)] for the case-cohort data as special examples. The latter belongs to the biased sampling problems. We show that the doubly weighted rank-based discontinuous estimating functions are monotone, a property established for the randomly observed data in the literature, when the generalized Gehan-type weights are used. Though the estimating problem can be formulated to a linear programming problem as that for the randomly observed data, due to its easily uncontrollable large scale even for a moderate sample size, we instead propose a Newton-type iterated method to search for an approximate solution of the (system of) discontinuous monotone estimating equation(s). Simulation results provide a good demonstration of the proposed method. We also apply our method to a real data example.

Appendices

Appendix I

Proof of (3.6)

To evaluate the difference between $\widetilde{S}_n(\beta, W_n,{\varvec{\xi}})$ and $\widetilde{S}_n(\beta^{+}, W_n,{\varvec{\xi}})$, it helps to see how $\widetilde{\mathcal{R}}_{(i)}^{\beta^{+}}$ changes when β changes to β⁺. To simplify notation, we replace $n_{\mathcal{C}_{\mathcal{D}}}$ by n and still keep its meaning as the number of completely observed subjects.

Apparently,

$$\begin{aligned} \widetilde{\mathcal{R}}_{(j)}^{\beta^+} &= \widetilde{\mathcal{R}}_{(j)}^{\beta},\quad \hbox{ for } j\not\in \{k, k+1\},\\ \widetilde{\mathcal{R}}_{(k)}^{\beta^+} &= \bigl \{R_{(k)}^{\beta^+}, R_{(k+1)}^{\beta^+},R_{(k+2)}^{\beta^+}, \ldots,R_{(n)}^{\beta^+}\bigl \}\\ &= \bigl \{R_{(k+1)}^{\beta}, R_{(k)}^{\beta}, R_{(k+2)}^{\beta},\ldots, R_{(n)}^{\beta}\bigl \}\\ &= \widetilde{\mathcal{R}}_{(k)}^{\beta} =\{{\widetilde{\mathcal{R}}_{(k+2)}^{\beta} ,R_{(k)}^{\beta}, R_{(k+1)}^{\beta}}\},\\ \end{aligned}$$

(7.1)

$$\widetilde{\mathcal{R}}_{(k+1)}^{\beta^+} = \bigl \{R_{(k+1)}^{\beta^+}, R_{(k+2)}^{\beta^+}, \ldots, R_{(n)}^{\beta^+}\bigl \} = \{\widetilde{\mathcal{R}}_{(k+2)}^{\beta}, R_{(k)}^{\beta}\} ,$$

(7.2)

where

$$\widetilde{\mathcal{R}}_{(k+2)}^{\beta} = \bigl\{R_{(k+2)}^{\beta}, \ldots, R_{(n)}^{\beta} \bigr\}.$$

To simplify the calculation, we use

$$W_n = G_n \cdot \sum_{j=1}^n \rho_j = \sum_{j\in {\mathcal{R}}_ {(k)}^{\beta} } \rho_j$$

which will not affect the conclusion since $\sum^n_{j=1} \rho_j$ is a constant.

Now write $\widetilde{S}_n(\beta,W_n, \varvec{xi})$ and $\widetilde{S}_n(\beta^+,W_n, \varvec{xi})$ as

$$\widetilde{S}_n(\beta,W_n, {\varvec{xi}}) =\sum_{i=1}^n \Delta_{(i)}^{\beta} \xi_{(i)}^\beta \Bigg\{ \sum_{j\in {\mathcal{R}}_{(i)}^{\beta} } \rho_j Z_{(i)}^{\beta} - \sum_{j\in {\mathcal{R}}_{(i)}^{\beta} } \rho_j Z_j \Bigg\}$$

(7.3)

and

$$\begin{aligned} \widetilde{S}_n(\beta^+,W_n, {\varvec{\xi}}) =\sum_{i=1}^n \Delta_{(i)}^{\beta^+} \xi_{(i)}^{\beta^+} \Bigg\{ \sum_{j\in {\mathcal{R}}_{(i)}^{\beta^+} } \rho_j Z_{(i)}^{\beta^+} - \sum_{j\in {\mathcal{R}}_{(i)}^{\beta^+} } \rho_j Z_j \Bigg\} . \end{aligned}$$

(7.4)

Denote

$$C= \sum_{i\not\in \{k, k+1\}} \Delta_{(i)}^{\beta} \xi_{(i)}^\beta \Bigg\{ \sum_{j\in {\mathcal{R}}_{(i)}^{\beta} } \rho_j Z_{(i)}^{\beta} - \sum_{j\in {\mathcal{R}}_{(i)}^{\beta} } \rho_j Z_j \Bigg\}$$

(7.5)

Then by utilizing the changes in risk sets and labels as listed in (3.2), (3.3) and (7.1), (7.2), we can write $\widetilde{S}_n(\beta,W_n, {\varvec{\xi}})$ as

$$\begin{aligned} \widetilde{S}_n(\beta,W_n, {\varvec{\xi}}) &= C + \Delta_{(k)}^{\beta} \xi_{(k)}^\beta \Bigg\{ \sum_{j\in {\mathcal{R}}_{(k+2)}^{\beta} } \rho_j (Z_{(k)}^{\beta} - Z_j) + R_{(k+1)}^{\beta}\rho_{(k+1)}^{\beta}\left(Z_{(k)}^{\beta} -Z_{(k+1)}^{\beta}\right) \Bigg\}\\ &\quad \quad +\Delta_{(k+1)}^{\beta} \xi_{(k+1)}^\beta \sum_{j\in \mathcal{R}_{(k+2)}^{\beta} } \rho_j \left(Z_{(k+1)}^{\beta} - Z_j \right). \end{aligned}$$

(7.6)

Similarly,

$$\begin{aligned} \widetilde{S}_n(\beta^+,W_n, {\varvec{\xi}}) &= C + \Delta_{(k)}^{\beta^+} \xi_{(k)}^{\beta+} \Bigg\{\sum_{j\in \mathcal{R}_{(k+2)}^{\beta^+} } \rho_j (Z_{(k)}^{\beta^+} - Z_j) + R_{(k+1)}^{\beta^+}\rho_{(k+1)}^{\beta^+}\left(Z_{(k)}^{\beta^+}-Z_{(k+1)}^{\beta^+} \right) \Bigg\}\\ &\quad\quad+ \Delta_{(k+1)}^{\beta^+} \xi_{(k+1)}^{\beta+} \sum_{j\in \mathcal{R}_{(k+2)}^{\beta^+} } \rho_j \left(Z_{(k+1)}^{\beta^+} - Z_j\right)\\ &= C + \Delta_{(k+1)}^{\beta} \xi_{(k+1)}^\beta \Bigg\{ \sum_{j\in \mathcal{R}_{(k+2)}^{\beta} } \rho_j (Z_{(k+1)}^{\beta} - Z_j) +R_{(k)}^{\beta}\rho_{(k)}^{\beta}\left(Z_{(k+1)}^{\beta}-Z_{(k)} ^{\beta}\right) \Bigg\}\\ &\quad\quad+ \Delta_{(k)}^{\beta} \xi_{(k)}^\beta \sum_{j\in \mathcal{R}_{(k+2)}^{\beta}} \rho_j \left(Z_{(k)}^{\beta} - Z_j\right). \end{aligned}$$

(7.7)

Hence subtract (7.6) from (7.7) and by R _i ρ _i  =  ρ _i for all i, we obtain

$$\begin{aligned} &\widetilde{S}_n(\beta^+,W_n, {\varvec{\xi}}) -\widetilde{S}_n(\beta,W_n , {\varvec{\xi}})\\ &\quad\quad= \Delta_{(k+1)}^{\beta} \xi_{(k+1)}^{\beta} \rho_{(k)}^{\beta}\left(Z_{(k+1)}^{\beta}-Z_{(k)}^{\beta} \right) - \Delta_{(k)}^{\beta} \xi_{(k)}^{\beta}\rho_{(k+1)}^{\beta}\left(Z_{(k)}^{\beta}-Z_{(k+1)}^ {\beta}\right) \\ &\quad\quad= \left(Z_{(k+1)}^{\beta}-Z_{(k)}^{\beta} \right) \left(\Delta_{(k+1)}^{\beta} \xi_{(k+1)}^{\beta}\rho_{(k)}^{\beta} + \Delta_{(k)}^{\beta} \xi_{(k)}^{\beta}\rho_{(k+1)}^{\beta} \right). \end{aligned}$$

Appendix II

Convergence of the hybrid Newton algorithm for the univariate case

Because $\widetilde{S}_n(\beta)$ is a right-continuous step function of β, usually there is no formally defined root. Let the interval [a,b) corresponds to the two consecutive segments of $\widetilde{S}_n(\beta)$ just below and above the horizontal axis, which contains the point where $\widetilde{S}_n(\beta)$ changes its sign. Then any point in [a,b) can be defined as a “root” of $\widetilde{S}_n(\beta)$. A very special situation that rarely occurs is that [a,b) corresponds to the segment where $\widetilde{S}_n(\beta)=0$. The following Proposition A1 shows that the proposed algorithm yields an “approximated solution” that is close to or inside the interval [a,b) due to the monotonicity of $\widetilde{S}_n(\beta)$. We do not consider the rate of numerical convergence here because it usually requires assumptions for second derivatives of the evaluated function in Newton type algorithms (see e.g., Stewart 1996). The number of iterations, however, is bounded in the worst scenario by the sample size n under the following assumption (7.8), which can be easily seen from the following proposition because the number of jumps of $\widetilde{S}_n(\beta)$ is bounded by n.

Proposition A1

Suppose

$$\min_\beta |\widetilde{S}_n(\beta)| < \epsilon$$

(7.8)

for a small value ε  >  0, then the hybrid Newton algorithm given in Sect. 5 will stop at k-th iteration for a finite k where $|\widetilde{S}_n(\beta^{(k)})| < \epsilon$.

Proof

Because $\widetilde{S}_n(\beta)$ is a step function with finite number of steps for a given sample size n, the stopping criterion $|\widetilde{S}_n(\beta^{(k)})| < \epsilon$ will be achieved for some finite k if the following inequalities hold for any m prior convergence:

$$|\widetilde{S}_n(\beta^{(m+1)})| \leq |\widetilde{S}_n(\beta^{(m)})| \hbox{ and } \min \Big\{ |\widetilde{S}_n(\beta^{(m+1)})|, |\widetilde{S}_n(\beta^{(m+2)})| \Big\} < |\widetilde{S}_n(\beta^{(m)})| .$$

(7.9)

We now show that the above inequalities in (7.9) always hold following the updating rules (5.5) and (5.6). Because $|\widetilde{S}_n(\beta)|$ is a non-decreasing function and the slope in (5.5) is always positive, if β^(m+1) is obtained from (5.5), then we must have $|\widetilde{S}_n(\beta^{(m+1)})| < |\widetilde{S}_n(\beta^{(m)})|$, which implies (7.9).

If β^(m+1) is obtained from (5.6) and both $\widetilde{S}_n(\beta^{(m+1)})$ and $\widetilde{S}_n(\beta^{(m)})$ have the same sign, then we have either $|\widetilde{S}_n(\beta^{(m+1)})| < |\widetilde{S}_n(\beta^{(m)})|$ that implies (7.9), or $|\widetilde{S}_n(\beta^{(m+1)})| = |\widetilde{S}_n(\beta^{(m)})|$ with β^(m+1) be the left-end of the interval where the above equality holds due to the right-continuity of $\widetilde{S}_n(\beta)$. For the latter, no matter β^(m+2) is obtained from (5.5) or (5.6), we must have $|\widetilde{S}_n(\beta^{(m+2)})| < |\widetilde{S}_n(\beta^{(m+1)})|$, thus (7.9) holds.

If β^(m+1) is obtained from (5.6), but $\widetilde{S}_n(\beta^{(m+1)})$ and $\widetilde{S}_n(\beta^{(m)})$ have different signs, then either $|\widetilde{S}_n(\beta^{(m+1)})| < |\widetilde{S}_n(\beta^{(m)})|$ when the updating is from the second option in (5.6), or both β^(m) and β^(m+1) are inside the interval [a,b) when the updating is from the first option in (5.6). For the former case, (7.9) holds; for the latter case, we must have $|\widetilde{S}_n(\beta^{(m+1)})| = \min_\beta|\widetilde{S}_n(\beta)| < \epsilon$ by assumption (7.8) and hence achieve convergence. The case that $\widetilde{S}_n(\beta^{(m+1)}) = 0$ is trivial. We thus have completed the proof.

Note: If assumption (7.8) does not hold, in other words, the pre-specified small value ε is too small for a given data set, then the algorithm can never achieve convergence. However, for a large enough maximum allowed iteration number, the algorithm will jump around a few fixed points after certain iteration. These fixed points should be inside the interval [a,b) for the univariate case, and any of them can be claimed as a “root” of $|\widetilde{S}_n(\beta)|$. Similar phenomenon exists for the multivariate case. So it is necessary to set a maximum iteration number for the algorithm to avoid possible dead loop, and it is also useful to print out the results of the final several steps if the maximum iteration number is exceeded.