Abstract
This paper studies semiparametric regression analysis of panel count data, which arise naturally when recurrent events are considered. Such data frequently occur in medical follow-up studies and reliability experiments, for example. To explore the nonlinear interactions between covariates, we propose a class of partially linear models with possibly varying coefficients for the mean function of the counting processes with panel count data. The functional coefficients are estimated by B-spline function approximations. The estimation procedures are based on maximum pseudo-likelihood and likelihood approaches and they are easy to implement. The asymptotic properties of the resulting estimators are established, and their finite-sample performance is assessed by Monte Carlo simulation studies. We also demonstrate the value of the proposed method by the analysis of a cancer data set, where the new modeling approach provides more comprehensive information than the usual proportional mean model.
Similar content being viewed by others
References
Andrews DF, Herzberg AM (1985) Data: a collection of problems from many fields for the student and research worker. Springer, New York
Balakrishnan N, Zhao X (2009) New multi-sample nonparametric tests for panel count data. Ann Stat 37:1112–1149
Byar DP (1980) The Veterans Administration study of chemoprophylaxis for recurrent stage I bladder tumors: comparisons of placebo, pyridoxine, and topical thiotepa. In: Pavane-Macaluso M, Smith PH, Edsmyr F (eds) Bladder tumors and other topics in urological oncology. Plenum, New York, pp 363–370
Cheng MY, Honda T, Li J, Peng H (2014) Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data. Ann Stat 42:1819–1849
Cheng SC, Wei LJ (2000) Inferences for a semiparametric model with panel data. Biometrika 87:89–97
Hu XJ, Lagakos SW, Lockhart RA (2009a) Generalized least squares estimation of the mean function of a counting process based on panel counts. Stat Sinica 19:561–580
Hu XJ, Lagakos SW, Lockhart RA (2009b) Marginal analysis of panel counts through estimating functions. Biometrika 96:445–456
Hu XJ, Sun J, Wei LJ (2003) Regression parameter estimation from panel counts. Scand J Stat 30:25–43
Hua L, Zhang Y (2012) Spline-based semiparametric projected generalized estimating equation method for panel count data. Biostatistics 13:440–454
Huang J (1999) Efficient estimation of the partly linear additive Cox model. Ann Stat 27:1536–1563
Huang CY, Wang MC, Zhang Y (2006) Analysing panel count data with informative observation times. Biometrika 93:763–775
Lindqvist BH, Kvaløy JT, Aaserud S (2015) Residual plots to reveal the functional form for covariates in parametric accelerated failure time models. Lifetime Data Anal 21:353–378
Lu M, Zhang Y, Huang J (2007) Estimation of the mean function with panel count data using monotone polynomial splines. Biometrika 94:705–718
Lu M, Zhang Y, Huang J (2009) Semiparametric estimation methods for panel count data using monotone B-splines. J Am Stat Assoc 104:1060–1070
Schumaker L (1981) Spline functions: basic theory. Wiley, New York
Sun J, Fang HB (2003) A nonparametric test for panel count data. Biometrika 90:199–208
Sun J, Kalbfleisch JD (1995) Estimation of the mean function of point processes based on panel count data. Stat Sinica 5:279–290
Sun J, Tong X, He X (2007) Regression analysis of panel count data with dependent observation times. Biometrics 63:1053–1059
Sun J, Wei LJ (2000) Regression analysis of panel count data with covariate-dependent observation and censoring times. J R Stat Soc Ser B 62:293–302
Thall PF, Lachin JM (1988) Analysis of recurrent events: nonparametric methods for random-interval count data. J Am Stat Assoc 83:339–347
Van der Vaart AW (2002) Semiparametric Statistics. In Lectures on Probability Theory and Statistics, Ecole d’Eté de Probabilités de Saint-Flour XXIX-1999. Lecture Notes in Mathematics 1781:330–457
Van der Vaart AW, Wellner JA (1996) Weak convergence and empirical processes. Springer, New York
Wellner JA, Zhang Y (2000) Two estimators of the mean of a counting process with panel count data. Ann Stat 28:779–814
Wellner JA, Zhang Y (2007) Two likelihood-based semiparametric estimation methods for panel count data with covariates. Ann Stat 35:2106–2142
Zeng D, Lin D, Yin GS (2005) Maximum likelihood estimation for the proportional odds model with random effects. J Am Stat Assoc 100:470–482
Zhang Y (2002) A semiparametric pseudolikelihood estimation method for panel count data. Biometrika 89:39–48
Zhang Y (2006) Nonparametric \(k\)-sample tests for panel count data. Biometrika 93:777–790
Zhang F, Chen X, Zhou Y (2014) Proportional hazards model with varying coefficients for length-biased data. Lifetime Data Anal 20:132–157
Zhang Y, Jamshidian M (2003) The gamma-frailty Poisson model for the nonparametric estimation of panel count data. Biometrics 59:1099–1106
Acknowledgments
The authors would like to thank the Editor, Professor Mei-Ling Ting Lee, the Associate Editor and the two reviewers for their constructive and insightful comments and suggestions that greatly improved the paper. Tong’s research was partly supported by BCMIIS, NSF China Zhongdian Project 11131002 and NSFC (No. 10971015). Zhao’s research was partly supported by the Research Grants Council of Hong Kong (PolyU 504011 and PolyU 503513), the Natural Science Foundation of China (No. 11371299), and The Hong Kong Polytechnic University.
Author information
Authors and Affiliations
Corresponding author
Appendix: Proofs of asymptotic results
Appendix: Proofs of asymptotic results
In this section we present the proofs of Theorems 3.1, 3.2 and 3.3.
1.1 Proof of Theorem 3.1
Here, we only present the proof of part (i) since part (ii) can be verified similarly. Let
where P and \(P_n\) denote the probability measure and the empirical measure, respectively. Let \(h(x)=x\log x-x+1\). Note that \(h(x)\ge (x-1)^2/4\) for \(0\le x\le 5\). For any \(\theta \) in a sufficiently small neighborhood of \(\theta _0\),
Then, using (7.1) and the arguments similar to those in Wellner and Zhang (2007), we can show that \(M^{ps}(\theta _0)=M^{ps}(\theta )\) if and only if \(\beta =\beta _0\), \(\Lambda (t)=\Lambda _0(t)\) a.e. with respect to \(\mu _1\), and \(\phi =\phi _0\) by C3 and C7.
By the similar arguments as those used in Wellner and Zhang (2007) again, we can also show that \({\hat{\Lambda }}^{ps}_n(t)\) is uniformly bounded in probability for \(t\in [0, b]\) if \(\mu _1([b, \tau ])>0\) for some \(0<b<\tau \) or \(t\in [0, \tau ]\) if \(\mu _1(\{\tau \})>0\).
By Helly-Selection Theorem and compactness of \(\Theta _n\), it follows that \({\hat{\theta }}_n^{ps}=({\hat{\beta }}^{ps}_n, {\hat{\Lambda }}^{ps}_n, {\hat{\phi }}^{ps}_n)\) has a subsequence \({\hat{\theta }}^{ps}_{n_k}=({\hat{\beta }}^{ps}_{n_k}, {\hat{\Lambda }}^{ps}_{n_k}, {\hat{\phi }}^{ps}_{n_k})\) converging to \(\theta ^+=(\beta ^+, \Lambda ^+, \phi ^+)\), where \(\Lambda ^+\) is a nondecreasing bound function on [0, b] for \(0<b<\tau \) and it can be defined on \([0, \tau ]\) if \(\mu _1(\{\tau \})>0\).
Note that \(\Theta _n\) is compact, and the function \(m_\theta ^{ps}(x)\) is upper semicontinuous in \(\theta \) for almost all x. Furthermore, \(m^{ps}_\theta (X)\le M^{ps}_0(X)<\infty \) with \(PM^{ps}_0(X)<\infty \) by C4. Thus, by Theorem A.1 of Wellner and Zhang (2000), we have
By the Dominated Convergence Theorem and C4, \(M^{ps}(\theta )\) is continuous in \(\theta \). Therefore, for any \(\varepsilon >0\), there exists \(\phi ^*_0\in \Psi _n\) such that
Clearly,
and
Then, using (7.2) and the arguments similar to those used in Lu et al. (2009), we can show that \(M^{ps}(\theta ^+)=M^{ps}(\theta _0)\), which yields \(\beta ^+=\beta _0\), \(\Lambda ^+=\Lambda _0\), a.e., and \(\phi ^+=\phi _0\). Therefore, we obtain the weak consistency of \(({\hat{\beta }}_n^{ps}, {\hat{\Lambda }}^{ps}_n, {\hat{\phi }}^{ps}_n)\) in the metric \(d_1\).
1.2 Proof of Theorem 3.2
To obtain the rate of convergence, we will apply Theorem 3.2.5 of Van der Vaart and Wellner (1996). Let \( m^{ps}_\theta (X)\), \( M^{ps}_n(\theta )\), and \(M^{ps}(\theta )\) be as defined in Appendix A.1. Let \(\mu (u,v,w)=\Lambda (u)\exp \{v\phi (w)\}\), \(\mu _0(u,v,w)=\Lambda _0(u)\exp \{v\phi _0(w)\}\) and \(g(t)=\mu _t(U, Z, V,W)\exp (Z'\beta _t)\), where \((U,Z,V,W)\sim \nu _1\), \(\mu _t=t\mu +(1-t)\mu _0\), \(\beta _t=t\beta +(1-t)\beta _0\) for \(0\le t\le 1\). Then,
By the mean value theorem, there exits a \(0\le \xi \le 1\) such that \(g(1)-g(0)=g'(\xi )\). Since
then from (7.1) we have
for a constant \(c_1\), where \(g_1(U,Z,V,W)=Z'(\beta -\beta _0)\mu _0(U,V,W)\), \(g_2(U,V,W)=(\mu -\mu _0)(U,V,W)\), and \(h_1(U,Z,V,W)=1+\xi \frac{(\mu -\mu _0)(U,V,W)}{\mu _0(U,V,W)}\) in the notation of Lemma 8.8 of van der Vaart (2002, page 432). To apply van der Vaart’s Lemma, we need to show that
for a constant \(c<1\). By the Cauchy-Schwarz inequality and condition (C13), we can show that (7.3) holds for \(c=1-\eta _1\). Let
Then
and
where \(g_3(U,V,W)=V(\phi (W)-\phi _0(W))\Lambda _0(U)\), \(g_4(U)=(\Lambda -\Lambda _0)(U)\), and \(h_2(U,V,W)=1+\zeta \frac{(\Lambda -\Lambda _0)(U)}{\Lambda _0(U)}\). Similarly, we can show that
So, by van der Vaart’s Lemma, we have
To derive the rate of convergence, next we need to find a \(\varphi _n(\delta )\) such that
Let
From the result of Theorem 2.7.5 of Van der Vaart and Wellner (1996) and Lemma A.2 of Huang (1999), for any \(\epsilon \le \delta \), we have
where \(||\cdot ||_{P, B}\) is the Bernstein Norm defined as \(||f||_{P, B}=\{2P(e^{|f|}-1-|f|)\}^{1/2}\) by van der Vaart and Wellner (1996, page 324). Moreover, we can show that
for any \(m^{ps}_\theta (X)-m^{ps}_{\theta _0}(X)\in \mathcal{F}^{ps}_\delta \). Therefore, by Lemma 3.4.3 of Van der Vaart and Wellner (1996), we obtain
where
Thus,
It is easy to see that \(\varphi _n(\delta )/\delta \) is decreasing in \(\delta \), and
for \(r_n=n^{\frac{1-v}{3}}\) and \(0<v<1/2\). Hence, it follows from Theorem 3.2.5 of Van der Vaart and Wellner (1996) that \( n^{\frac{1-v}{3}} d_1({\hat{\theta }}^{ps}_n, \theta _0)=O_p(1). \) Similarly, we can obtain the rate of convergence for \({\hat{\theta }}_n\).
1.3 Proof of Theorem 3.3
First, we prove part (i). Recall that
We define a sequence of maps \(S^{ps}_n\) mapping a neighborhood of \((\beta _0,\Lambda _0, \phi _0)\), denoted by \(\mathcal {U}\), in the parameter space for \((\beta ,\Lambda , \phi )\) into \(l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) as :
Correspondingly, we define the limit map \(S^{ps} : \mathcal {U} \longrightarrow l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) as
where
and
To derive the asymptotic normality of the estimators \((\hat{\beta }^{ps}_n,\hat{\Lambda }^{ps}_n,\hat{\phi }^{ps}_n)\), motivated by the proofs of Theorem 3.3.1 of Van der Vaart and Wellner (1996, page 310) and Theorem 2 of Zeng et al. (2005) , we need to verify the following five conditions.
-
(a1)
\(\sqrt{n}(S^{ps}_n - S^{ps}) (\hat{\beta }^{ps}_n,\hat{\Lambda }^{ps}_n,\hat{\phi }^{ps}_n) - \sqrt{n}(S^{ps}_n - S^{ps}) (\beta _0,\Lambda _0, \phi _0) = o_p(1)\).
-
(a2)
\(S^{ps}(\beta _0, \Lambda _0, \phi _0) = 0\) and \(S^{ps}_n(\hat{\beta }^{ps}_n, \hat{\Lambda }^{ps}_n,\hat{\phi }^{ps}_n) = o_p(n^{-1/2}).\)
-
(a3)
\(\sqrt{n}(S^{ps}_n - S^{ps}) (\beta _0,\Lambda _0, \phi _0)\) converges in distribution to a tight Gaussian process on \(l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\).
-
(a4)
\( S^{ps}(\beta ,\Lambda , \phi )\) is Fr\(\acute{e}\)chet-differentiable at \((\beta _0, \Lambda _0, \phi _0)\) with a continuously invertible derivative \(\dot{S}^{ps}(\beta _0,\Lambda _0, \phi _0).\)
-
(a5)
\(S^{ps}(\hat{\beta }^{ps}_n,\hat{\Lambda }^{ps}_n,\hat{\phi }^{ps}_n) -S^{ps}(\beta _0,\Lambda _0, \phi _0)-\dot{S}^{ps}(\beta _0,\Lambda _0, \phi _0)({\hat{\beta }}^{ps}_n - \beta _0,{\hat{\Lambda }}^{ps}_n - \Lambda _0, {\hat{\phi }}^{ps}_n - \phi _0)=o_p(n^{-1/2}). \)
Condition (a1) holds since
is a Donsker class for some \(\delta > 0\), and that
as \((\beta ,\Lambda , \phi ) \longrightarrow (\beta _0,\Lambda _0, \phi _0)\) in \(d_1\).
Clearly, \(S^{ps}(\beta _0, \Lambda _0,\phi _0) = 0\). For \(h_3 \in \mathcal {F}_r\), let \(h_{3n}\) be the B-spline function approximation of \(h_3\) with \(||h_3-h_{3n}||_\infty =O(n^{-vr})\) by Corollary 6.21 of Schumaker (1981, page 227). Then we have \(S^{ps}_n(\hat{\beta }^{ps}_n, \hat{\Lambda }^{ps}_n,\hat{\phi }^{ps}_n)[\mathbf h _1, {h}_2, h_{3n}] = 0\). Thus, for \((\mathbf h _1, {h}_2, h_3) \in {H_1\times H_2\times \mathcal{F}_r}\),
where
and
From (a1), we have \(I_{n1}=o_p(1)\) and \(I_{n2}=o_p(1)\). Next we need to show \(I_{n3}=o_p(1)\) and \(I_{n4}=o_p(1)\) . Note that
and
by Theorem 3.2. Thus (a2) holds for \(\frac{1}{6r-2}<v<\frac{1}{2}\).
Condition (a3) holds because \({H_1\times H_2\times \mathcal{F}_r}\) is a Donsker class and the functionals \(A^{ps}_{n1}, A^{ps}_{n2}, A^{ps}_{n3}\) are bounded Lipschitz functions with respect to \({H_1\times H_2\times \mathcal{F}_r}\).
For (a4), by the smoothness of \(S^{ps}(\beta ,\Lambda , \phi )\), the Fr\(\acute{e}\)chet differentiability holds and the derivative of \(S^{ps}(\beta ,\Lambda , \phi )\) at \((\beta _0,\Lambda _0, \phi _0)\), denoted by \(\dot{S}^{ps}(\beta _0,\Lambda _0, \phi _0)\), is a map from the space \(\{( \beta - \beta _0, \Lambda - \Lambda _0,\phi - \phi _0): (\beta ,\Lambda , \phi ) \in \mathcal {U}\}\) to \(l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) and
Thus, we have
where
and
where \(F_W\) denotes the cumulative distribution of W.
Next, we show that \(Q^{ps}=(Q^{ps}_1, Q^{ps}_2, Q^{ps}_3)\) is one-to-one, that is, for \((\mathbf h _1,h_2, h_3) \in {H_1\times H_2\times \mathcal{F}_r}\), if \(Q^{ps}(\mathbf h _1,h_2, h_3) = 0\), then \(\mathbf h _1=\mathbf{0},h_2=0, h_3=0\).
Suppose that \(Q^{ps}(\mathbf h _1,h_2, h_3) = 0\). Then \(\dot{S}^{ps}(\beta _0,\Lambda _0, \phi _0)(\beta - \beta _0,\Lambda - \Lambda _0, \phi - \phi _0)[\mathbf h _1, {h}_2, h_3]=0\) for any \((\beta , \Lambda , \phi )\) in the neighborhood \(\mathcal U\). In particular, we take \(\beta =\beta _0+\epsilon \mathbf{h}_1, \Lambda =\Lambda _0+\epsilon h_2, \phi =\phi _0+\epsilon h_3\) for a small constant \(\epsilon \). Thus we have
which yields
and so \(\mathbf h _1=\mathbf{0},h_2=0, h_3=0\) by C7.
Next we show that (a5) holds. Write
where
and
It is easy to show that \(B_{n1}=O_p(d_1^{2}({\hat{\theta }}^{ps}_n, \theta _0))\), \(B_{n2}=O_p(d_1^{2}({\hat{\theta }}^{ps}_n, \theta _0))\), and \(B_{n3}=O_p(d_1^{2}({\hat{\theta }}^{ps}_n, \theta _0))\). Thus, by Theorem 3.2, (a5) holds for \(0<v<1/4\).
It follows from (7.4), (a1), (a2) and (a5) that
uniformly in \(\mathbf h _1\), \( {h}_2\) and \(h_3\).
For each \((\mathbf h _1, {h}_2, h_3)\in {H_1\times H_2\times \mathcal{F}_r}\), since \(Q^{ps}\) is invertible, there exists \((\mathbf h ^{ps}_1, {h}^{ps}_2, h^{ps}_3)\in {H_1\times H_2\times \mathcal{F}_r}\) such that
Therefore, we have
where
To prove part (ii), we define a sequence of maps \(S_n\) mapping a neighborhood of \((\beta _0,\Lambda _0, \phi _0)\), \(\mathcal {U}\), in the parameter space for \((\beta ,\Lambda , \phi )\) into \(l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) as:
Write \(\Delta N_i(T_{K_i,j})=N_i(T_{K_i,j})-N_i(T_{K_i,j-1})\), \(\Delta \Lambda (T_{K_i,j})=\Lambda (T_{K_i,j})-\Lambda (T_{K_i,j-1})\), and \(\Delta h(T_{K_i,j})=h(T_{K_i,j})-h(T_{K_i,j-1})\).
Then, we have
Correspondingly, we define the limit map \(S : \mathcal {U} \longrightarrow l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) as
where
and
Furthermore, the derivative of \(S(\beta ,\Lambda , \phi )\) at \((\beta _0,\Lambda _0, \phi _0)\), denoted by \(\dot{S}(\beta _0,\Lambda _0, \phi _0)\), is a map from the space \(\{( \beta - \beta _0, \Lambda - \Lambda _0,\phi - \phi _0): (\beta ,\Lambda , \phi ) \in \mathcal {U}\}\) to \(l^{\infty }({H_1\times H_2\times \mathcal{F}_r})\) and
where
and
Next, we show that \(Q=(Q_1, Q_2, Q_3)\) is one-to-one, that is, for \((\mathbf h _1,h_2, h_3) \in {H_1\times H_2\times \mathcal{F}_r}\), if \(Q(\mathbf h _1,h_2, h_3) = 0\), then \(\mathbf h _1=\mathbf{0},h_2=0, h_3=0\)
Suppose that \(Q(\mathbf h _1,h_2, h_3) = 0\). Then \(\dot{S}(\beta _0,\Lambda _0, \phi _0)(\beta - \beta _0,\Lambda - \Lambda _0, \phi - \phi _0)[\mathbf h _1, {h}_2, h_3]=0\) for any \((\beta , \Lambda , \phi )\) in the neighborhood \(\mathcal U\). In particular, we take \(\beta =\beta _0+\epsilon \mathbf{h}_1, \Lambda =\Lambda _0+\epsilon h_2, \phi =\phi _0+\epsilon h_3\) for a small constant \(\epsilon \). Thus we have
which yields
and so \(\mathbf h _1=\mathbf{0},h_2=0, h_3=0\) by C7.
Similarly, we can show that \(S(\beta _0, \Lambda _0, \phi _0) = 0\), \(S_n(\hat{\beta }_n, \hat{\Lambda }_n,\hat{\phi }_n) = o_p(n^{-1/2})\), and
for \(0<v<1/4\). Thus it follows that
uniformly in \(\mathbf h _1\), \( {h}_2\) and \(h_3\).
For each \((\mathbf h _1, h_2, h_3)\in (\mathbf h _1,h_2, h_3)\), since Q is invertible, there exists \((\mathbf h ^{*}_1, \mathbf h ^*_2, h^{*}_3)\in (\mathbf h _1,h_2, h_3)\) such that
Thus, we have
where
Rights and permissions
About this article
Cite this article
He, X., Feng, X., Tong, X. et al. Semiparametric partially linear varying coefficient models with panel count data. Lifetime Data Anal 23, 439–466 (2017). https://doi.org/10.1007/s10985-016-9368-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10985-016-9368-x