Abstract
We study the jackknife variance estimator for a general class of two-sample statistics. As a concrete application, we consider samples with a common mean but possibly different, ordered variances as arising in various fields such as interlaboratory experiments, field studies, or the analysis of sensor data. Estimators for the common mean under ordered variances typically employ random weights, which depend on the sample means and the unbiased variance estimators. They take different forms when the sample estimators are in agreement with the order constraints or not, which complicates even basic analyses such as estimating their variance. We propose to use the jackknife, whose consistency is established for general smooth two-sample statistics induced by continuously Gâteux or Fréchet-differentiable functionals, and, more generally, asymptotically linear two-sample statistics, allowing us to study a large class of common mean estimators. Furthermore, it is shown that the common mean estimators under consideration satisfy a central limit theorem (CLT). We investigate the accuracy of the resulting confidence intervals by simulations and illustrate the approach by analyzing several data sets.
Similar content being viewed by others
References
Bhattarcharya, C. (1980). Estimation of a common mean and recovery of interblock information. The Annals of Statistics, 8, 205–211.
Brown, L. D., & Cohen, A. (1974). Point and confidence interval estimation of a common mean and recovery of interblock information. The Annals of Statistics, 2, 963–976.
Cemer, I. (2011). Noise measurement. Sensors online. https://www.sensorsmag.com/embedded/noise-measurement.
Chang, Y.-T., Oono, Y., & Shinozaki, N. (2012). Improved estimators for the common mean and ordered means of two normal distributions with ordered variances. Journal of Statistical Planning and Inference, 142(9), 2619–2628.
Chang, Y.-T., & Shinozaki, N. (2008). Estimation of linear functions of ordered scale parameters of two gamma distributions under entropy loss. Journal of the Japan Statistical Society, 38(2), 335–347.
Chang, Y-T., & Shinozaki, N. (2015). Estimation of two ordered normal means under modified Pitman nearness criterion. Annals of the Institute of Statistical Mathematics, 67, 863–883. https://doi.org/10.1007/s10463-014-0479-4.
Cochran, W. (1937). Problems arising in the analysis of a series of similar experiments. JASA, 4, 172–175.
Cressie, N. (1997). Jackknifing in the presence of inhomogeneity. Technometrics, 39(1), 45–51.
Degerli, Y. (2000). Analysis and reduction of signal readout circuitry temporal noise in CMOS image sensors for low light levels. IEEE Transactions on Electron Devices, 47(5), 949–962.
Efron, B. (1982). The jackknife, the bootstrap and other resampling plans (Vol. 38)., CBMS-NSF regional conference series in applied mathematics Philadelphia, PA: Society for Industrial and Applied Mathematics (SIAM).
Efron, B., & Hastie, T. (2016). Computer age statistical inference: Algorithms, evidence, and data science (Vol. 5)., Institute of mathematical statistics (IMS) monographs New York: Cambridge University Press.
Efron, B., & Stein, C. (1981). The jackknife estimate of variance. The Annals of Statistics, 9(3), 586–596.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap (Vol. 57)., Monographs on statistics and applied probability New York: Chapman and Hall.
Elfessi, A., & Pal, N. (1992). A note on the common mean of two normal populations with order restrictions in location-scale families. Communications in Statistics—Theory and Methods, 21(11), 3177–3184.
Fisher, R. (1932). Statistical methods for research workers (4th ed.). London: Oliver and Boyd.
Heyl, P., & Cook, G. (1936). The value of gravity in Washington. Journal of Research of the U.S. Bureau of Standards, 17, 805–839.
Keller, T., & Olkin, I. (2004). Combining correlated unbiased estimators of the mean of a normal distribution. A Festschrift for Herman Rubin, 45, 218–227.
Kubokawa, T. (1989). Closer estimation of a common mean in the sense of Pitman. Annals of the Institute of Statistical Mathematics, 41(3), 477–484.
Lee, Y. (1991). Jackknife variance estimators in the one-way random effects model. Annals of the Institute of Statistical Mathematics, 43(4), 707–714.
Lin, D. (2010). Quantified temperature effect in a CMOS image sensor. IEEE Transactions on Electron Devices, 57(2), 422–428.
Mehta, J., & Gurland, J. (1969). Combinations of unbiased estimators of the mean which consider inequality of unknown variances. JASA, 64(327), 1042–1055.
Miller, R. G. (1974). The jackknife—A review. Biometrika, 61, 1–15.
Nair, K. (1980). Variance and distribution of the Graybill–Deal estimator of the common mean of two normal populations. The Annals of Statistics, 8(1), 212–216.
Nair, K. (1982). An estimator of the common mean of two normal populations. Journal of Statistical Planning and Inference, 6, 119–122.
Pitman, E. (1937). The closest estimates of statistical parameters. Proceedings of the Cambridge Philosophical Society, 33, 212–222.
Quenouille, M. (1949). Approximate tests of correlation in time series. Mathematical Proceedings of the Cambridge Philosophical Society, 11, 68–84.
Shao, J. (1993). Differentiability of statistical functionals and consistency of the jackknife. The Annals of Statistics, 21(1), 61–75.
Shao, J., & Wu, C. F. (1989). A general theory for jackknife variance estimation. The Annals of Statistics, 17, 1176–1197.
Sinha, B. (1985). Unbiased estimation of the variance of the GD estimator of the common mean of several normal populations. Canadian Journal of Statistics, 13(3), 243–247.
Steland, A. (2015). Vertically weighted averages in Hilbert spaces and applications to imaging: Fixed sample asymptotics and efficient sequential two-stage estimation. Sequential Analysis, 34(3), 295–323.
Steland, A. (2017). Fusing photovoltaic data for improved confidence intervals. AIMS Energy, 5, 113–136.
Tippett, L. (1931). The method of statistics. London: Williams and Norgate.
Tukey, J. W. (1958). Bias and confidence in not quite large samples (abstract). Annals of Mathematical Statistics, 29, 614.
van Eeden, C. (2006). Restricted parameter space estimation problems., Lecture notes in statistics Berlin: Springer.
Voinov, V. (1984). Variance and its unbiased estimator for the common mean of several normal populations. Sankhya: The Indian Journal of Statistics, Series B, 46, 291–300.
Acknowledgements
This work was supported by JSPS Kakenhi grants #JP26330047 and #JP8K11196. Parts of this paper have been written during research visits of the first author at Mejiro University, Tokyo. He thanks for the warm hospitality. Both authors thank Hideo Suzuki, Keio University at Yokohama, for invitations to his research seminar, Shinozaki Nobuo, Takahisa Iida, Shun Matsuura and the participants for comments and discussion. The authors gratefully acknowledge the support of Prof. Takenori Takahashi, Mejiro University and Keio University Graduate School, and Akira Ogawa, Mejiro University, for providing and discussing the chip manufacturing data. They would like to thank anonymous referees for the helpful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Miscellaneous results
The following result provides the asymptotic linearity of the sample variance and provides several higher order convergence rates of the remainder term under moment conditions on the random variables.
Lemma A.1
Let \(\xi _1, \dots , \xi _n\) be i.i.d. random variables \({\textit{with}}\, E |\xi _1|^{12} < \infty\). Then, the sample variance \(S_n^2 = \frac{1}{n}\sum _{i=1}^n (\xi _i - \overline{\xi })^2\) is asymptotically linear:
where \({\widetilde{ \xi }}_i = \xi _i - E( \xi _i )\), \(1 \le i \le n\), and the remainder term \(R_n = ( \frac{1}{n} \sum _{i=1}^n (X_i-\mu ))^2\) satisfies the following:
and
as \(n \rightarrow \infty\). The assertions also hold for the unbiased variance estimator \({\widetilde{ S }}_n^2 = \frac{n}{n-1} S_n^2\).
Proof of Lemma A.1
A direct calculation shows that
with \(R_n = ( \overline{{\widetilde{ \xi }}} )^2\). Using the fact that \(E( \sum _{i=1}^n {\widetilde{ \xi }}_i )^r = O( n^{r/2} )\), if \(E | \xi _i |^r < \infty\), the estimates in (33) follow easily, that is:
and
To show (34), we observe that
By virtue of the \(C_r\)-inequality:
for \(r = 2, 4\). Observe that
and
such that
Similarly:
and
leading to
The additional arguments to treat \({\widetilde{ S }}_n^2\) are straightforward and omitted. \(\square\)
Lemma A.2
Suppose that \(X_{ij}, j = 1, \dots , n_i\), are i.i.d. with finite eight moments and strictly ordered variances \(\sigma _1^2 < \sigma _2^2\). Then
for some constant C.
Proof
By Lemma A.1, we have the following:
with \(L_{ni} = \frac{1}{n_i} \sum _{j=1}^{n_i} [ ( X_{ij}- \mu )^2 - \sigma _i^2 ]\) and \(R_{ni} = ( \frac{1}{n_i} \sum _{j=1}^{n_i} (X_{ij}-\mu ) )^2\), for \(i = 1, 2\). For \(r = 1, 2\), we have the following:
Hence
Now, we may conclude that:
\(\square\)
Lemma A.3
If \(\xi _1, \dots , \xi _{n_1}\) are i.i.d. with finite second moment and \(\max _{1 \le i \le n_1} E| {\widehat{ \xi }}_i - \xi _i | = o(1)\), then
and
as \(n_1 \rightarrow \infty\).
Proof
Since
as \(n_1 \rightarrow \infty\). Similarly:
such that the squared sample moments converge in \(L_1\), as well, since
as \(n_1 \rightarrow \infty\), follows. \(\square\)
Lemma A.4
Suppose that \(Z_1, Z_2, \dots\) are i.i.d. with \(E( Z_1^2 )< \infty\) and \(\xi _1, \xi _2, \dots\) are random variables with \(\max _{1 \le i \le n} | Z_i - \xi _i | = o(1)\), as \(n \rightarrow \infty\), a.s.. Then, \(\frac{1}{n} \sum _{i=1}^n \xi _i \rightarrow E(Z_1)\), a.s., and \(\frac{1}{n} \sum _{i=1}^n \xi _i^2 \rightarrow E( Z_1^2 )\), a.s.
Proof
First notice that, by the strong law of large numbers:
as \(n \rightarrow \infty\), a.s.. For the sample moments of order 2, we have the estimate:
as \(n \rightarrow \infty\). \(\square\)
Appendix B: Proof of Theorem 4.1
In the sequel, \(\Vert \cdot \Vert _2\) denotes the vector-2 norm, whereas \(\Vert \cdot \Vert _{L_p}\) stands for the \(L_p\)-norm of a random variable, i.e., \(\Vert Z \Vert _{L_p} = \left( E |Z|^p \right) ^{1/p}\).
As discussed in Remark 3.3, we let:
Also notice that \(\mu _1 = \mu _2 = \mu\). We have the following:
where
Observe that
such that, especially, the function \(\varphi\) satisfies the following:
Observe that
We show the result in detail for \(\theta\) with \(\sigma _1^2 < \sigma _2^2\); the other case (\(\sigma _1^2 > \sigma _2^2\)) is treated analogously and we only indicate some essential steps. Furthermore, we only discuss \(\phi ( {\widehat{ \theta }}_N)\), since \(\psi ( {\widehat{ \theta }}_N)\) can be handled similarly. We have to consider the following:
which can be rewritten as follows:
The first term is the leading one, whereas the last two terms are of the order \(1/N^2\) in the \(L_2\)-sense, since
and for \(\sigma _1^2 < \sigma _2^2\), we have the following:
by Lemma A.2. By symmetry, we also have the following:
By our assumptions on \(\gamma ^\le\), we have for the leading term of \(\phi ( {\widehat{ \theta }}_N ) - \phi ( \theta )\):
which can be written as follows:
for some \({\widehat{ \theta }}_N^*\) between \({\widehat{ \theta }}_N\) and \(\theta\), where \({\widehat{ \theta }}_N = ( {\widehat{ \theta }}_{Ni} )_{i=1}^5\) and \(\theta = (\theta _i)_{i=1}^5\). \(\nabla [\gamma ^\le ( \theta ) \mu _1]\) denotes the gradient of the function \(\bar{\theta } \mapsto \gamma ^\le (\bar{\theta }) \bar{\theta }_4\), \(\bar{\theta } =(\bar{\theta }_1, \dots , \bar{\theta }_5)' \in \Theta\), evaluated at (the true value) \(\theta\), and \(\partial _{ij} [\gamma ^\le \mu _1]( {\widehat{ \theta }}_N^* )\) stands for the second partial derivative with respect to \(\theta _i\) and \(\theta _j\) evaluated at a point \({\widehat{ \theta }}_N^*\). Observe that the linear term in the above expansion is asymptotically linear, i.e., it can be written as follows:
for appropriate random variables \(\xi _{ij}\), since the coordinates are asymptotically linear, that is:
This follows from the facts that \(\overline{X}_i\), \(i = 1, 2\), are linear statistics and \({\widetilde{ S }}_i^2 - \sigma _i^2\), \(i = 1, 2\), are asymptotically linear by virtue of Lemma A.1. Thus, it suffices to estimate the second-order terms of the Taylor expansion. First, observe that
for \(i = 1, \dots , 5\), such that
where \(\Vert \cdot \Vert _2\) denotes the vector-2 norm. For the arithmetic means (40) follows, because the remainder term vanishes and \(E(\sum _{j=1}^N \xi _{ij} )^r = O(N^{r/2})\) if \(E|\xi _{ij}|^r < \infty\), and for the estimators \({\widetilde{ S }}_i^2\) because of Lemma A.1. The quadratic terms can be estimated as follows:
Here, \(\Vert x \Vert _\infty = \max _{1\le i \le \ell } | x_i |\) denotes the \(l_\infty\)-norm of a vector \(x = (x_1, \dots , x_\ell )' \in {\mathbb {R}}^{\ell }\). Therefore, we obtain the estimate:
for the quadratic term of the above Taylor expansion. Switching back to the definition (35) of \({\widehat{ \theta }}_N\) and \(\theta\), the above arguments show that, for ordered variances, \(\sigma _1^2 < \sigma _2^2\):
for some remainder \(R_N^\phi\) with \(N E( R_N^\phi )^2 = O(1/N)\). The treatment of \(\psi ( {\widehat{ \theta }}_N ) - \psi (\theta )\) is similar and leads to the following:
for some remainder term \(R_N^\psi\) with \(N E( R_N^\psi )^2 = O(1/N)\). Putting things together and collecting terms lead to the following:
for a remainder \(R_N^\gamma\) with \(N E( R_N^\gamma )^2 = O(1/N)\). Replacing the indicators \({\mathbf {1}}_{\{ {\widetilde{ S }}_1^2 \le {\widetilde{ S }}_2^2 \}}\) and \({\mathbf {1}}_{\{ {\widetilde{ S }}_1^2 > {\widetilde{ S }}_2^2 \}}\) by \({\mathbf {1}}_{\{ \sigma _1^2 \le \sigma _2^2 \}} (=1)\) and \({\mathbf {1}}_{\{ \sigma _1^2 > \sigma _2^2 \}} (=0)\), respectively, adds additional correction terms \(R_N^{(1)}\) and \(R_N^{(2)}\) with \(N E( R_N^{(i)} )^2 = o(1)\), \(i = 1, 2\). This can be seen as follows: we have, for instance:
where
Observe that, for \(\sigma _1^2 < \sigma _2^2\), we have \(J = | {\mathbf {1}}_{ \{ {\widetilde{ S }}_1^2 \le {\widetilde{ S }}_2^2 \}} - {\mathbf {1}}_{ \{ \sigma _1^2 \le \sigma _2^2 \} } | = {\mathbf {1}}_{ \{ {\widetilde{ S }}_1^2 > {\widetilde{ S }}_2^2 \}}\). If we put \(C = \max _j \sup _\theta \left| \frac{ \partial [\gamma ^\le (\theta ) \mu _1 ] }{ \partial \theta _j } \right| < \infty\), we may estimate the following:
Using
and \(P( {\widetilde{ S }}_1^2 > {\widetilde{ S }}_2^2 ) = O( 1/N^2 )\); see (37) and (41), we, therefore, arrive at
If, contrary, \(\sigma _1^2 > \sigma _2^2\), we observe that \(J = {\mathbf {1}}_{ \{ {\widetilde{ S }}_1^2 \le {\widetilde{ S }}_2^2 \}}\), and similar arguments combined with (38) imply the following:
Putting things together, we, therefore, arrive at assertion (i):
since \(\mu _1 = \mu _2 = \mu\) by assumption.
It remains to discuss the form of the leading term when \(\mu = \mu _1 = \mu _2\) by calculating the partial derivatives. Observe that
and, thus, vanishes if \(\mu _1 = \mu _2\). Hence, the asymptotic linear statistic governing \({\widehat{ \mu }}_N(\gamma ) - \mu\) does not depend on the sample variances. Furthermore:
and
Hence, under the common mean constraint \(\mu _1 = \mu _2\), those two partial derivatives are given by \(\gamma ^\le (\theta )\) and \(1-\gamma ^\le (\theta )\), respectively. Thus, we may conclude that:
with \(N E( R_N^2 ) = O(1/N)\), which verifies (ii).
(iii) Assertion (ii) shows that \({\widehat{ \mu }}_N(\gamma )\) is an asymptotically linear two-sample statistic with kernels \(h_1(x) = \gamma (\theta ) (x-\mu _1)\) and \(h_2(x) = (1-\gamma (\theta )) (x-\mu _2)\), which satisfy that \(\int | h_i(x) |^4 \, {\text {d}} F_i(x) < \infty\), \(i = 1, 2\), and a remainder term \(R_N\) with \(N E(R_N^2) = o(1)\), as \(N \rightarrow \infty\), such that (8) holds. It remains to verify (9). Our starting point is the Taylor expansion (39), where we have to study the second sum. First observe that replacing \(\partial _{ij}[\gamma ^{\le }(\theta )\mu _1]({\widehat{ \theta }}_N^*)\) by \(\partial _{ij}[\gamma ^{\le }(\theta )\mu _1](\theta )\) gives error terms \(F_{N,ij}\) satisfying the following:
The first factor is o(1) , by Fubini’s theorem, since the integrand is continuous, bounded, and converges to 0, as \(N \rightarrow \infty\), a.s. Furthermore:
such that
as \(N \rightarrow \infty\), follows. In what follows, denote by \({\widetilde{ R }}_N\) the second term in (39) with \({\widehat{ \theta }}_N^*\) replaced by \(\theta\), that is:
such that \(F_N = \frac{1}{2} \sum _{i,j=1}^5 F_{N,ij}\) satisfies
Clearly, (42) implies that
To show
as \(N \rightarrow \infty\), by virtue of (43), it suffices to show that
as \(N \rightarrow \infty\). Indeed, since we already know that \(N^2 E(R_N^2) = o(1)\), we have the following:
with \(N^2 E( F_N^2 ) = o(1)\) and
as \(N \rightarrow \infty\). Hence, \(N^2 E( {\widetilde{ R }}_N^2 ) = o(1)\), as \(N \rightarrow \infty\), as well. Due to the high rate of convergence of \(F_N\) in (43), it follows that \(N^2 E( F_N - F_{N-1} )^2 = o(1)\), as \(N \rightarrow \infty\), since
as \(N \rightarrow \infty\). Now, it follows that (44) implies \(N^2E(R_N - R_{N-1})^2 = o(1)\), as \(N \rightarrow \infty\), since, by Minkowski’s inequality:
where
as \(N \rightarrow \infty\). This verifies that it is sufficient to show (44). As the sum is finite and the second-order derivatives \(\partial _{ij}[ \gamma {\le }( \theta ) \mu _1 ]( \theta )\) do not depend on N and are bounded, (44) follows, if we establish, for each pair \(i, j \in \{ 1, \dots , 5 \}\):
as \(N \rightarrow \infty\). We shall make use of the decomposition \({\widehat{ \theta }}_{Ni} - \theta _i = L_{Ni} + R_{Ni}\) in a linear statistic and an additional remainder term (if \({\widehat{ \theta }}_{Ni}\) is not a sample mean), for \(i = 1, \dots , 5\). (45) follows, if we prove that
as \(N \rightarrow \infty\), for \(i, j = 1, \dots , 5\). Those estimates will follow from the following estimates for \(L_{Ni}\) and \(R_{Ni}\):
as \(N \rightarrow \infty\), for \(i, j = 1, \dots , 5\). (50) holds, because for a linear statistic, say, \(L_N = N^{-1} \sum _{i=1}^N \xi _i\), with i.i.d. mean zero summands \(\xi _1, \dots , \xi _N\) with finite fourth moment, the formula
leads to the estimate
since \(E( \sum _{i<N} \xi _i )^4 = O(N^2)\). For the sample means, the remainder term vanishes, such that (51)–(53) hold trivially, and for the sample variances (51)–(53) are shown in Lemma A.1. Now, (46)–(48) can be shown as follows. For each pair \(i, j \in \{ 1, \dots , 5 \}\), the identity
yields
such that
which verifies (47). Similarly, we have the following:
where
which leads to (46). Combining those results shows that the remainder term, \(R_N\), also satisfies the condition (9). Therefore, consistency and asymptotic unbiasedness of the jackknife variance estimator follow from Theorem 2.1. The proof is complete. \(\square\)
Rights and permissions
About this article
Cite this article
Steland, A., Chang, YT. Jackknife variance estimation for general two-sample statistics and applications to common mean estimators under ordered variances. Jpn J Stat Data Sci 2, 173–217 (2019). https://doi.org/10.1007/s42081-019-00034-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42081-019-00034-2