Applications of Sequential Methods in Multiple Hypothesis Testing
- 34 Downloads
One of the main computational burdens in genome-wide statistical applications is the evaluation of large scale multiple hypothesis tests. Such tests are often implemented using replication-based methods, such as the permutation test or bootstrap procedure. While such methods are widely applicable, they place a practical limit on the computational complexity of the underlying test procedure. In such cases it would seem natural to apply sequential procedures. For example, suppose we observe the first ten replications of an upper-tailed statistic under a null distribution generated by random permutations, and of those ten, five exceed the observed value. It would seem reasonable to conclude that the P-value will not be small enough to be of interest, and further replications should not be needed.
While such methods have been proposed in the literature, for example by Hall in 1983, by Besag and Clifford in 1991 and by Lock in 1991, they have not been widely applied in multiple testing applications generated by high dimensional data sets, where they would likely be of some benefit. In this article related methods will first be reviewed. It will then be shown how commonly used multiple testing procedures may be modified so as to introduce sequential procedures while preserving the validity of reported error rates. A number of examples will show how such procedures can reduce computation time by an order of magnitude with little loss in power.
KeywordsMultiple hypothesis testing Sequential hypothesis testing Gene expression analysis
This work was supported by NIH grant R21HG004648.
- 9.Hall, W. J. (1983). Some sequential tests for matched pairs: A sequential permutation test. In P. K. Sen (ed.), Contributions to statistics: essays in honour of Norman L. Johnson, (pp. 211–228). Amsterdam: North-Holland.Google Scholar
- 13.Mootha, V. K., Lindgren, C. M., Eriksson, K. F., Subramanian, A., Sihag, S., Lehar, J., et al. (2003). PGC-1 α-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nature Genetics, 100, 605–610.Google Scholar
- 16.Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., et al. (2005). Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proceedings of the National Academy of Sciences of the United States of America, 102, 15545–15550.Google Scholar
- 19.Yang, H. Y., & Speed, T. (2003). Design and analysis of comparative microarray experiments. In T. Speed (ed.) Statistical analysis of gene expression microarray data (pp. 35–92). Boca Raton, FL: Chapman and Hall.Google Scholar