Power for the Test of Homogeneity in Fixed and Random Effects Models

Abstract

This chapter will illustrate methods for the power of the test of homogeneity in fixed and random effects models. In fixed effects models, the test of homogeneity provides evidence about whether the effect sizes in a meta-analysis are measuring a common effect size. The test of homogeneity in random effects models is a test of the statistical significance of the variance component, the between-studies variance. The chapter gives examples of how to compute the power for the test of homogeneity in both fixed and random effects models.

Appendix

5.1.1 Computing Power for the Tests of Homogeneity and the Variance Component

The examples in this chapter require the computation of the area under the non-central chi-square that is either less than or greater than a given critical value, c_α. Below I give the functions and or commands necessary to obtain the power for the test of homogeneity in fixed effects, and the test of the variance component in random effects. The program Exel only provides the values of the cumulative central chi-square distribution, and thus cannot be used for the examples in this chapter.

5.1.1.1 SPSS

Using the Compute menu in SPSS, the function IDF.CHISQ(p, df) provides the critical value c _α for a central chi-square distribution where p equals the area in the left tail, and df equals the degrees of freedom. The following command returns the critical value for the example in Sect. 5.2.3.

$$ critval = IDF.CHISQ(0.95,9) $$

SPSS gives the critical value for the command above as 16.919. The function given by NCDF.CHISQ(quant, df, nc) gives the area less than or equal to quant = c _α for a non-central chi-square distribution with df degrees of freedom and non-centrality parameter equal to nc. To compute the power for Example 5.2.3,

$$ power = 1 - NCDF.CHISQ(16.92,9,\,4) $$

returns the value 0.225.

The example in Sect. 5.3.3 uses the central chi-square distribution with non-integer degrees of freedom. The command in SPSS

$$ power = 1 - CDF.CHISQ(21.974,15.58) $$

returns the value 0.128.

5.1.1.2 SAS

In SAS, the function CINV(p, df) provides the critical value, c _α , of the cumulative central chi-square distribution function with df degrees of freedom that is less than c _α . The function CDF(‘CHISQUARE’, x, df, ncp) provides the area in the left tail that is less than or equal to x in a non-central chi-square distribution with df degrees of freedom and non-centrality parameter equal to ncp. To compute the power for the example in Sect. 5.2.3, we first obtain the critical value for the central chi-square distribution with 9° of freedom at the α = 0.05 level as below

$$ critval = cinv(0.95,9). $$

The command above returns a value of 16.9190. The power of the first test in Sect. 5.2.3 is computed in SAS as

$$ power = 1 - cdf(\hbox{`}chisquare\hbox{'},\,16.92,\,9,\,4) $$

The above function results in a value of 0.22536. The example in Sect. 5.3.3 requires the central chi-square distribution with non-integer degrees of freedom. Power for the first example in Sect. 5.2.3 is computed as

$$ power = 1 - cdf(\hbox{`}chisquare\hbox{'},\,21.974,\,15.58,\,0) $$

This command results in a value of 0.12834.

5.1.1.3 R

In R, the function qchisq(p, df, ncp, lower.tail = TRUE, log.p = FALSE) gives the critical value for the cumulative chi-square distribution with df degrees of freedom for the area equal to p. The option lower.tail = TRUE indicates that we want p to equal the area in the lower tail. The option log.p = FALSE indicates that we do not want the value of p in log units. The command below computes the critical value for the 95% point of the chi-square distribution with 9 degrees of freedom and non-centrality parameter = 0. When the non-centrality parameter is equal to 0, the distribution is a central chi-square distribution.

$$ \begin{array}{ccccc} > {\hbox{ qchisq}}\left( {0.{95},{ 9},{\hbox{ ncp}} = 0,{\hbox{ lower}}.{\hbox{tail }} = {\hbox{ TRUE}},{ \log }.{\hbox{p }} = {\hbox{ FALSE}}} \right) \hfill \\ \left[ {1} \right]{ 16}.{91898} \end{array}$$

To obtain the power for the second computation in Example 5.2.3, we use the command

$$ \begin{array}{ccccc} > {\hbox{ pchisq}}\left( {{16}.{919},{ 9},{\hbox{ ncp}} = {9},{\hbox{ lower}}.{\hbox{tail }} = {\hbox{ FALSE}},{ \log }.{\hbox{p }} = {\hbox{ FALSE}}} \right) \hfill \\ \left[ {1} \right]{ }0.{51}0{4429} \end{array}$$

Note that in this example, we are asking for the area in the upper tail as indicated by the option lower.tail = FALSE. The first example in Sect. 5.3.3 is computed as

$$ \begin{array}{ccccc} {{> \,pchisq(21.97,\,15.58,\,ncp = 0,\,lower.tail = FALSE,\,\log.p = FALSE) \hfill}} \\ \left[ 1 \right]\,0.1284637 \end{array}$$

As in the prior example, we want the upper tail, so lower.tail = FALSE.