Instrumental variable based SEE variable selection for Poisson regression models with endogenous covariates

Abstract

In the case of containing endogenous covariates, we study the variable selection problem for Poisson regression models based on the instrumental variable adjustment technology. By using a modified smooth-threshold estimating equation technology, we propose an instrumental variable based variable selection procedure. The proposed method can attenuate the effect of endogenous covariates, and is easy for application in practice. We also prove that this variable selection procedure is consistent in theory. Some simulations and a real data analysis are given to evaluate the performance of the proposed method, and simulation results show that the proposed variable selection procedure is workable.

Appendix

Proof of Theorems

In this section, we present the technical proofs of Theorems 1–3. For convenience and simplicity, let C denote a positive constant which may be different values at each appearance throughout this paper.

Proof of Theorem 1

Let $\beta =\beta _{0}+\tau \gamma$, where $\tau =n^{-1/2}$. We first prove that, for any given $\epsilon >0$, and a large n, there exists a constant C such that

$$\begin{aligned} P\left\{ \inf _{\Vert \gamma \Vert =C}(\beta _{0}-\beta )^{T}{\hat{S}}_{n}(\beta )>0\right\} \ge 1-\epsilon . \end{aligned}$$

(9)

This will imply that there exists a local solution to the equation ${\hat{S}}_{n}(\beta )=0$ such that ${\hat{\beta }}=\beta _{0}+O_{p}(n^{-1/2})$ with probability at least $1-\epsilon$.

Denote $\varLambda (\beta )=(\beta _{0}-\beta )^{T}{\hat{S}}_{n}(\beta )$, then a simple calculation yields

$$\begin{aligned} \displaystyle \varLambda (\beta )= & {} \displaystyle (\beta _{0}-\beta )^{T}{\hat{S}}_{n}(\beta _{0}) -(\beta -\beta _{0})^{T}\frac{\partial }{\partial \beta }{\hat{S}}_{n}({\tilde{\beta }})(\beta -\beta _{0})\nonumber \\\equiv & {} J_{n1}+J_{n2}, \end{aligned}$$

(10)

where ${\tilde{\beta }}$ lies between $\beta$ and $\beta _{0}$. Next we will consider $J_{n1}$ and $J_{n2}$, respectively. For $J_{n1}$, by some elementary calculations, we have

$$\begin{aligned} \begin{array}{lll} J_{n1}&{}=&{}\displaystyle -\tau \gamma ^{T}(I_{p}-{\hat{\varDelta }}){\hat{U}}(\beta _{0})-\tau \gamma ^{T}{\hat{\varDelta }}\beta _{0}\\ &{}=&{}\displaystyle -\tau \gamma ^{T}(I_{p}-{\hat{\varDelta }}) \sum _{i=1}^{n}{\hat{X}}_{i}\varepsilon _{i} -\tau \gamma ^{T}(I_{p}-{\hat{\varDelta }}) \sum _{i=1}^{n}{\hat{X}}_{i}[\exp (X_{i}^{T}\beta _{0})-\exp ({\hat{X}}_{i}^{T}\beta _{0})]\\ &{}&{}\quad -\,\tau \gamma ^{T}{\hat{\varDelta }}\beta _{0} \\ &{}\equiv &{}J_{n1,1}+J_{n1,2}+J_{n1,3}. \end{array} \end{aligned}$$

Note that $E\{\varepsilon _{i}|Z_{i}\}=0$, we can prove that

$$\begin{aligned} \frac{1}{\sqrt{n}}\sum _{i=1}^{n}Z_{i}\varepsilon _{i}=O_{p}(1). \end{aligned}$$

Then, invoking $\Vert I_{p}-{\hat{\varDelta }}\Vert \le 1-\min \{{\hat{\delta }}_{k}\}$, a simple calculation yields

$$\begin{aligned} J_{n1,1}= & {} \displaystyle -\tau \gamma ^{T}(I_{p}-{\hat{\varDelta }}) {\hat{\varGamma }}_{XZ}{\hat{\varGamma }}_{ZZ}^{-1}\sum _{i=1}^{n}Z_{i}\varepsilon _{i}\nonumber \\= & {} \displaystyle O_{p}\left( \tau \sqrt{n}\right) \Vert \gamma \Vert =O_{p}(1)\Vert \gamma \Vert . \end{aligned}$$

(11)

By using Taylor expansion, we can derive that

$$\begin{aligned} \begin{array}{lll} \exp (X_{i}^{T}\beta _{0})-\exp ({\hat{X}}_{i}^{T}\beta _{0})&{}=&{}\exp ({\tilde{X}}_{i}^{T}\beta _{0})(X_{i}^{T}\beta _{0}-{\hat{X}}_{i}^{T}\beta _{0})\\ &{}=&{}\exp ({\tilde{X}}_{i}^{T}\beta _{0})[(\varGamma -{\hat{\varGamma }})Z_{i}+e_{i}]^{T}\beta _{0}, \end{array} \end{aligned}$$

where ${\tilde{X}}_{i}^{T}\beta _{0}$ lies between $X_{i}^{T}\beta _{0}$ and ${\hat{X}}_{i}^{T}\beta _{0}$. Then, invoking ${\hat{\varGamma }}=\varGamma +O_{p}(n^{-1/2})$ and $E(e_{i}|Z_{i})=0$, and using the similar argument to the proof of $J_{n1,1}$, we have that

$$\begin{aligned} J_{n1,2}= & {} \displaystyle -\tau \gamma ^{T}(I_{p}-{\hat{\varDelta }}) {\hat{\varGamma }}_{XZ}{\hat{\varGamma }}_{ZZ}^{-1}\sum _{i=1}^{n}Z_{i}\exp ({\tilde{X}}_{i}^{T}\beta _{0}) [(\varGamma -{\hat{\varGamma }})Z_{i}+e_{i}]^{T}\beta _{0}\nonumber \\= & {} O_{p}(1)\Vert \gamma \Vert . \end{aligned}$$

(12)

In addition, since ${\hat{\delta }}_{k}=\min \{1,\lambda /|{\hat{\beta }}_{k}^{0}|^{2}\}$, we have

$$\begin{aligned} J_{n1,3}=O_{p}(\tau )\Vert \gamma \Vert =o_{p}(1)\Vert \gamma \Vert . \end{aligned}$$

(13)

Then, invoking (11)–(13), we can obtain

$$\begin{aligned} J_{n1}=O_{p}(1)\Vert \gamma \Vert . \end{aligned}$$

(14)

Now consider $J_{n2}$. A simple calculation yields

$$\begin{aligned} \begin{array}{lll} \displaystyle \frac{\partial }{\partial \beta }{\hat{S}}_{n}(\beta ) &{}=&{}\displaystyle \frac{\partial }{\partial \beta }\left[ (I_{p}-{\hat{\varDelta }})\sum _{i=1}^{n}{\hat{X}}_{i}(Y_{i}-\exp ({\hat{X}}_{i}^{T}\beta ))+ {\hat{\varDelta }}\beta \right] \\ &{}=&{} \displaystyle -(I_{p}-{\hat{\varDelta }})\sum _{i=1}^{n}{\hat{X}}_{i}{\hat{X}}_{i}^{T}\exp ({\hat{X}}_{i}^{T}\beta )+{\hat{\varDelta }}. \end{array} \end{aligned}$$

Hence, we have

$$\begin{aligned} \begin{array}{lll} J_{n2} &{}=&{}\displaystyle -(\beta -\beta _{0})^{T}\frac{\partial }{\partial \beta }{\hat{S}}_{n}({\tilde{\beta }})(\beta -\beta _{0})\\ &{}=&{}\displaystyle \tau ^{2}\gamma ^{T}(I_{p}-{\hat{\varDelta }})\sum _{i=1}^{n} {\hat{X}}_{i}{\hat{X}}_{i}^{T}\exp ({\hat{X}}_{i}^{T}{\tilde{\beta }})\gamma + \tau ^{2}\gamma ^{T}{\hat{\varDelta }}\gamma \\ &{}\equiv &{} J_{n2,1}+J_{n2,2}. \end{array} \end{aligned}$$

By some calculations, we can prove that

$$\begin{aligned} J_{n2,1}=O_{p}(\tau ^{2}n)\Vert \gamma \Vert ^{2}=O_{p}(1)\Vert \gamma \Vert ^{2}. \end{aligned}$$

(15)

In addition, it is easy to show that

$$\begin{aligned} J_{n2,2}=O_{p}(\tau ^{2})\Vert \gamma \Vert ^{2}=o_{p}(1)\Vert \gamma \Vert ^{2}. \end{aligned}$$

(16)

Hence invoking (14)–(16), and by choosing a sufficiently large C, $J_{n2,1}$ dominates $J_{n2,2}$ and $J_{n1}$ uniformly in $\Vert \gamma \Vert =C$. Furthermore, $J_{n2,1}$ is positive for the sufficiently large C. Then by (10), we have that for any given $\epsilon >0$, if we choose C large enough, (9) holds. This implies, with probability at least $1-\epsilon$, that there exists a local solution ${\hat{\beta }}$ such that

$$\begin{aligned} {\hat{\beta }}=\beta _{0}+O_{p}(\tau ). \end{aligned}$$

Together this with $\tau =n^{-1/2}$, we complete the proof of Theorem 1. $\square$

Proof of Theorem 2

Let ${\mathcal {A}}_{0}^{c}=\{k|\beta _{k0}=0\}$, then for any given $k\in {\mathcal {A}}_{0}^{c}$, by using the similar arguments as in the proof of Theorem 1, we have that the initial estimator ${\hat{\beta }}_{k}^{(0)}$ satisfies that ${\hat{\beta }}_{k}^{(0)}=O_{p}(n^{-1/2})$. Then note that $n\lambda \rightarrow \infty$, we can derive that

This implies that

$$\begin{aligned} P({\hat{\delta }}_{k}=1, \hbox {for ~all~}k\in {\mathcal {A}}_{0}^{c})\rightarrow 1. \end{aligned}$$

(17)

On the other hand, for any given $\epsilon >0$ and $k\in {\mathcal {A}}_{0}$, by the condition $n^{1/2}\lambda \rightarrow 0$, we have that

This implies that ${\hat{\delta }}_{k}=o_{p}(n^{-1/2})$ for all $k\in {\mathcal {A}}_{0}$. Hence we prove that

$$\begin{aligned} P({\hat{\delta }}_{k}<1, \hbox {for ~all~} k\in {\mathcal {A}}_{0})\rightarrow 1. \end{aligned}$$

(18)

Then, invoking (17) and (18), we complete the proof of Theorem 2. $\square$

Proof of Theorem 3

From Theorem 2, we know that, with probability tending to 1, ${\hat{\beta }}_{k}=0$ for $k\in {\mathcal {A}}_{0}^{c}$, and ${\hat{\beta }}_{\mathcal {A}_{0}}$ satisfies the smooth-threshold generalized estimating equations

$$\begin{aligned} (I_{|\mathcal {A}_{0}|}-{\hat{\varDelta }}_{\mathcal {A}_{0}}){\hat{U}} ({\hat{\beta }}_{\mathcal {A}_{0}})+{\hat{\varDelta }}_{\mathcal {A}_{0}}{\hat{\beta }}_{\mathcal {A}_{0}}=0. \end{aligned}$$

(19)

Following Theorem 1, we have ${\hat{\beta }}_{\mathcal {A}_{0}}=\beta _{\mathcal {A}_{0}}+O_{p}(n^{-1/2})$. Then, applying the Taylor expansion to (19) at $\beta _{\mathcal {A}_{0}}$, we can get

$$\begin{aligned} o_{p}(1)=\frac{1}{\sqrt{n}}{\hat{U}}(\beta _{\mathcal {A}_{0}}) +\frac{1}{\sqrt{n}}\frac{\partial }{\partial \beta _{\mathcal {A}_{0}}}{\hat{U}}(\beta _{\mathcal {A}_{0}}) ({\hat{\beta }}_{\mathcal {A}_{0}}-\beta _{\mathcal {A}_{0}}) +\frac{1}{\sqrt{n}}{\hat{G}}_{\mathcal {A}_{0}}{\hat{\beta }}_{\mathcal {A}_{0}}, \end{aligned}$$

(20)

where ${\hat{G}}_{\mathcal {A}_{0}}=(I_{|\mathcal {A}_{0}|}-{\hat{\varDelta }}_{\mathcal {A}_{0}})^{-1} {\hat{\varDelta }}_{\mathcal {A}_{0}}.$ In addition, some calculations yield

$$\begin{aligned} \displaystyle \left\| \frac{1}{\sqrt{n}}{\hat{G}}_{\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}\right\| ^{2}\le & {} \displaystyle \frac{1}{n(1-\max _{k\in \mathcal {A}_{0}}{\hat{\delta }}_{k})^{2}} \sum _{k\in \mathcal {A}_{0}}\frac{(\lambda \beta _{k})^{2}}{({\hat{\beta }}_{k}^{(0)})^{4}}\\= & {} \displaystyle \frac{\lambda ^{2}}{n(1-\max _{k\in \mathcal {A}_{0}}{\hat{\delta }}_{k})^{2}} \sum _{k\in \mathcal {A}_{0}}\left| {\hat{\beta }}_{k}^{(0)-1} +(\beta _{k}-{\hat{\beta }}_{k}^{(0)}){\hat{\beta }}_{k}^{(0)-2}\right| ^{2}\\\le & {} \displaystyle O_{p}(n^{-1}\lambda ^{2})\sum _{k\in \mathcal {A}_{0}}\left( 2\left| {\hat{\beta }}_{k}^{(0)}\right| ^{-2} +2\left| (\beta _{k}-{\hat{\beta }}_{k}^{(0)}){\hat{\beta }}_{k}^{(0)-2}\right| ^{2}\right) \\\le & {} \displaystyle O_{p}(n^{-1}\lambda ^{2})\left( 2|\mathcal {A}_{0}|\min _{k\in \mathcal {A}_{0}}|{\hat{\beta }}_{k}^{(0)}|^{-2} +2\min _{k\in \mathcal {A}_{0}}|{\hat{\beta }}_{k}^{(0)}|^{-4} \Vert \beta _{\mathcal {A}_{0}}-{\hat{\beta }}_{\mathcal {A}_{0}}^{(0)}\Vert ^{2}\right) \\= & {} \displaystyle O_{p}(n^{-1}\lambda ^{2})+O_{p}(n^{-2}\lambda ^{2})\\= & {} \displaystyle O_{p}((\sqrt{n}\lambda )^{2}n^{-2})(1+O_{p}(n^{-1})=o_{p}(n^{-2}). \end{aligned}$$

Hence, we have

$$\begin{aligned} \frac{1}{\sqrt{n}}{\hat{G}}_{\mathcal {A}_{0}}{\hat{\beta }}_{\mathcal {A}_{0}} =\frac{1}{\sqrt{n}}{\hat{G}}_{\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}} +\frac{1}{\sqrt{n}}{\hat{G}}_{\mathcal {A}_{0}}({\hat{\beta }}_{\mathcal {A}_{0}}-\beta _{\mathcal {A}_{0}}) =o_{p}(n^{-1}). \end{aligned}$$

(21)

Furthermore, invoking ${\hat{\varGamma }}=\varGamma +O_{p}(n^{-1/2})$ and ${\hat{X}}_{i,{\mathcal {A}}_{0}}={\hat{\varGamma }}_{{\mathcal {A}}_{0}}Z_{i,{\mathcal {A}}_{0}}$, we can derive that

$$\begin{aligned} \displaystyle \frac{1}{n}\frac{\partial }{\partial \beta _{\mathcal {A}_{0}}}{\hat{U}}(\beta _{\mathcal {A}_{0}})= & {} \displaystyle \frac{1}{n}\frac{\partial }{\partial \beta _{\mathcal {A}_{0}}} \sum _{i=1}^{n}{\hat{X}}_{i,\mathcal {A}_{0}}(Y_{i}-\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}))\nonumber \\= & {} \displaystyle -\frac{1}{n}\sum _{i=1}^{n}\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) {\hat{X}}_{i,\mathcal {A}_{0}}{\hat{X}}^{T}_{i,\mathcal {A}_{0}}\nonumber \\= & {} \displaystyle -\frac{1}{n}\sum _{i=1}^{n}\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) {\hat{X}}_{i,\mathcal {A}_{0}}{\hat{X}}^{T}_{i,\mathcal {A}_{0}}\nonumber \\&\,\displaystyle +\frac{1}{n}\sum _{i=1}^{n}[\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) -\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})] {\hat{X}}_{i,\mathcal {A}_{0}}{\hat{X}}^{T}_{i,\mathcal {A}_{0}}\nonumber \\= & {} \displaystyle -H_{n}(\beta _{\mathcal {A}_{0}})+o_{p}(1). \end{aligned}$$

(22)

Invoking (20)–(22), we have

$$\begin{aligned} H_{n}(\beta _{\mathcal {A}_{0}})\sqrt{n}({\hat{\beta }}_{\mathcal {A}_{0}}-\beta _{\mathcal {A}_{0}})= \frac{1}{\sqrt{n}}{\hat{U}}(\beta _{\mathcal {A}_{0}})+o_{p}(1). \end{aligned}$$

(23)

Next we show $\frac{1}{\sqrt{n}}{\hat{U}}(\beta _{\mathcal {A}_{0}})$ is asymptotically normally distributed with mean zero and covariance matrix $M(\beta _{\mathcal {A}_{0}})$. Applying a Taylor expansion to $\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})$ at $X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}$, it is easy to show that

$$\begin{aligned} \exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) =\displaystyle \exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})+ \exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) ({\hat{X}}_{i,\mathcal {A}_{0}}-X_{i,\mathcal {A}_{0}})^{T}\beta _{\mathcal {A}_{0}} +O_{p}(n^{-1}). \end{aligned}$$

Hence we have

$$\begin{aligned} \displaystyle \frac{1}{\sqrt{n}}{\hat{U}}(\beta _{\mathcal {A}_{0}})= & {} \displaystyle \frac{1}{\sqrt{n}} \sum _{i=1}^{n}{\hat{X}}_{i,\mathcal {A}_{0}}(Y_{i}-\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}))\nonumber \\= & {} \displaystyle \frac{1}{\sqrt{n}} \sum _{i=1}^{n}{\hat{X}}_{i,\mathcal {A}_{0}}(Y_{i}-\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}))\nonumber \\&\,\displaystyle +\frac{1}{\sqrt{n}} \sum _{i=1}^{n} {\hat{X}}_{i,\mathcal {A}_{0}} [\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}) -\exp ({\hat{X}}^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})]\nonumber \\= & {} \displaystyle \frac{1}{\sqrt{n}} \sum _{i=1}^{n}\varGamma _{\mathcal {A}_{0}}Z_{i,\mathcal {A}_{0}} (Y_{i}-\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}))\nonumber \\&\,\displaystyle +\sum _{i=1}^{n}\varGamma _{\mathcal {A}_{0}}Z_{i,\mathcal {A}_{0}} \exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})e^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}} +o_{p}(1)\nonumber \\= & {} \displaystyle \frac{1}{\sqrt{n}} \sum _{i=1}^{n}\varGamma _{\mathcal {A}_{0}}Z_{i,\mathcal {A}_{0}} [(Y_{i}-\exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}}))- \exp (X^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}})e^{T}_{i,\mathcal {A}_{0}}\beta _{\mathcal {A}_{0}} ]\nonumber \\&+\,o_{p}(1)\nonumber \\\equiv & {} \displaystyle \frac{1}{\sqrt{n}}\sum _{i=1}^{n}\eta _{i,\mathcal {A}_{0}}+o_{p}(1). \end{aligned}$$

(24)

Note that $E(\frac{1}{\sqrt{n}}\sum _{i=1}^{n}\eta _{i,\mathcal {A}_{0}})=0$, and $Cov(\frac{1}{\sqrt{n}}\sum _{i=1}^{n}\eta _{i,\mathcal {A}_{0}})=M_{n}(\beta _{\mathcal {A}_{0}})\rightarrow M(\beta _{\mathcal {A}_{0}})$, then using the Central Limits Theorem, we have

$$\begin{aligned} \frac{1}{\sqrt{n}}\sum _{i=1}^{n}\eta _{i,\mathcal {A}_{0}}{\mathop {\longrightarrow }\limits ^{\mathcal {L}}} N(0, M(\beta _{\mathcal {A}_{0}})). \end{aligned}$$

(25)

Hence, invoking (23)–(25) and the Slutsky theorem, we complete the proof of Theorem 3. $\square$