Skip to main content
SpringerLink
Log in

Multiple-Hypothesis Testing Strategy

  • Chapter
  • First Online:
Modern Issues and Methods in Biostatistics

Part of the book series: Statistics for Biology and Health ((SBH))

  • 3395 Accesses

Abstract

In this chapter, we will discuss multiple hypothesis-testing issues from a frequentist perspective. The Bayesian approaches for multiple-testing problems will be discussed briefly in Chap. 10. As we all know, a typical hypothesis test in the frequentist paradigm can be written as

$${H}_{o} : \delta \in {\Omega }_{0}\ \mathrm{or}\ {H}_{a} : \delta \in {\Omega }_{1},$$
(1.1)

where δ is a parameter such as treatment effect, the domain Ω 0 can be, for example, a set of nonpositive values, and the domain Ω 1 can be the negation of Ω 0. In this case, (1.1) becomes

$${H}_{o} : \delta \leq 0\ \mathrm{or}\ {H}_{a} : \delta > 0.$$
(1.2)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Further Readings and References

  • Bauer, P., Rohmel, J., Maurer, W., Hothorn, L.: Testing strategies in multi-dose experiments including active control. Stat. Med. 17, 2133–2146 (1998)

    Article  Google Scholar 

  • Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57(2), 289–300 (1995)

    MATH  MathSciNet  Google Scholar 

  • Benjamini, Y., Liu, W.: A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. J. Stat. Plan. Inference 82, 163–170 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  • Benjamini, Y., Yekutieli, D.: The control of the false discovery rate in multiple testing under dependency. Ann. Stat. 29(4), 1165–1188 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  • Berger, R.L.: Multiparameter hypothesis testing and acceptance sampling. Technometrics 24, 295–300 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  • Birnbaum, A.: On the foundations of statistical inference (with discussion). J. Am. Stat. Assoc. 57, 269–326 (1962)

    Article  MATH  MathSciNet  Google Scholar 

  • Blanchard, G., Roquain, E.: Adaptive false discovery rate control under independence and dependence. J. Mach. Learn. Res. 10, 2837–2871 (2009)

    MathSciNet  Google Scholar 

  • Bretz, F., Schmidli1, H., König, F., Racine1, A., Maurer, W.: Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: General concepts. Biom. J. 48:4.48, 623–634 (2006)

    Google Scholar 

  • Bretz, F., Hothorn, T., Westfall, P.: Multiple Comparison Using R. CRC Press, Boca Raton (2010)

    Book  Google Scholar 

  • Chang, M.: Adaptive Design Theory and Implementation Using SAS and R. Chapman and Hall/CRC, Boca Raton (2007)

    Google Scholar 

  • Chang, M.: Limitations of hypothesis testing in clinical trials: Discussion of “Some Controversial Multiple Testing Problems in Regulatory Applications.” J. Biopharm. Stat. 19, 35–41 (2009)

    Article  Google Scholar 

  • Chang, M., Chow, S.C.: Analysis strategy of multiple-endpoint adaptive design. J. Biopharm. Stat. 17, 1189–1200 (2007)

    Article  MathSciNet  Google Scholar 

  • Chen, X., Luo, X., Capizzi, T.: The application of enhanced parallel gatekeeping strategies. Stat. Med. 24, 1385–1397 (2005)

    Article  MathSciNet  Google Scholar 

  • Dmitrienko, A., Offen, W.W., Westfall, P.H.: Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat. Med. 22, 2387–2400 (2003)

    Article  Google Scholar 

  • Dmitrienko, A., Molenberghs, G., Chuang-Stein, C., Offen, W.: Analysis of Clinical Trials Using SAS: A Practical Guide. SAS, Cary (2005)

    Google Scholar 

  • Dmitrienko, A., Offen, W., Wang, O., Xiao, D.: Gatekeeping procedures in dose–response clinical trials based on the Dunnett test. Pharm. Stat. 5, 19–28 (2006a)

    Article  Google Scholar 

  • Dmitrienko, A., Tamhane, A.C., Wang, X., Chen, X.: Stepwise gatekeeping procedures in clinical trial applications. Biom. J. 48, 984–991 (2006b)

    Article  MathSciNet  Google Scholar 

  • Dmitrienko, A., Wiens, B.L., Westfall, P.H.: Fallback tests in dose–response clinical trials. J. Biopharm. Stat. 16, 745–755 (2006c)

    Article  MathSciNet  Google Scholar 

  • Dmitrienko, A., Tamhane, A.C.: Gatekeeping procedures with clinical trial applications. Pharma. Stat. 6, 171–180 (2007)

    Article  Google Scholar 

  • Dmitrienko, A., Wiens, B.L., Tamhane, A.C., Wang, X.: Tree-structured gatekeeping tests in clinical trials with hierarchically ordered multiple objectives. Stat. Med. 26, 2465–2478 (2007)

    Article  MathSciNet  Google Scholar 

  • Dmitrienko, A., Tamhane, A.C., Bretz, F.: Multiple Testing Problems in Pharmaceutical Statistics. Chapman and Hall/CRC, Boca Raton (2010)

    Google Scholar 

  • Dunnett, C.W.: A multiple comparison procedure for comparing several treatments with a control. J. Am. Stat. Assoc. 50, 1096–1121 (1955)

    Article  MATH  Google Scholar 

  • European Medicines Agency: ICH Topic E 9 Statistical Principles for Clinical Trials. Step 5. http://www.ema.europa.eu (1998). Accessed 8 Jan 2011

  • Finner, H., Roter, M.: On the false discovery rate and expected type I errors. Biom. J. 43, 985–1005 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  • Ghosh, J.K., Delampady, M., Samanta, T.: An Introduction to Bayesian Analysis. Springer, New York (2006)

    MATH  Google Scholar 

  • Grechanovsky, E., Hochberg, Y.: Closed procedures are better and often admit a shortcut. J. Stat. Plan. Inference 76, 79–91 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  • Hochberg, Y.: A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 800–802 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  • Hochberg, Y., Benjamini, Y.: More powerful procedures for multiple significance testing. Stat. Med. 9, 811–818 (1990)

    Article  Google Scholar 

  • Hochberg, Y., Tamhane, A.C.: Multiple Comparison Procedures. Wiley, New York (1987)

    Book  MATH  Google Scholar 

  • Holm, S.: A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979)

    MATH  MathSciNet  Google Scholar 

  • Hommel, G.: A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75, 383–386 (1988)

    Article  MATH  Google Scholar 

  • Hommel, G., Bretz, F.: Aesthetics and power considerations in multiple testing – a contradiction? Biom. J. 50, 657–666 (2008)

    Article  MathSciNet  Google Scholar 

  • Hommel, G., Bretz, F., Maurer, W.: Powerful shortcuts for multiple testing procedures with special reference to gatekeeping strategies. Stat. Med. 99, 25–41 (2007)

    MathSciNet  Google Scholar 

  • Hsu, J.C.: Multiple Comparisons: Theory and Methods. Chapman and Hall, London (1996)

    MATH  Google Scholar 

  • Hung, H.M.J.: Some controversival multiple testing problems in regulatory applications. J. Biopharm. Stat. 4, 1–25 (2008)

    MathSciNet  Google Scholar 

  • Huque, M., Röhmel, J.: Multiplicity problems in clinical trials: A regulatory perspective. In: Dmitrienko, A., Tamhane, A.C., Bretz, F. (eds.) Multiple Testing Problems in Pharmaceutical Statistics. Chapman and Hall/CRC, Boca Raton (2010)

    Google Scholar 

  • Kanji, G.K.: 100 Statistical Tests, 3rd edn. Sage Publications, London (2006)

    Google Scholar 

  • Lehmacher, W., Wassmer, G., Reimeir, P.: Procedure for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics 47, 511–521 (1991)

    Article  Google Scholar 

  • Lehmann, E.L., Romano, J.P.: Generalizations of the familywise error rate. Ann. Stat. 33, 1138–1154 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  • Marcus, R., Peritz, E., Gabriel, K.R.: On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63, 655–660 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  • Lemuel, A.M.: Multiple analysis in clinical trials. Springer-Verlag, New York (2003)

    Google Scholar 

  • Pocock, S., Geller, N., Tsiatis, A.: Analysis of multiple endpoints in clinical trials. Biometrics 43, 487–498 (1987)

    Article  MathSciNet  Google Scholar 

  • Posch, M., Zehetmayer, S., Bauer, P.: Hunting for significance with the false discovery rate. J. Am. Stat. Assoc. 104(486), 832–840 (2009)

    Article  MathSciNet  Google Scholar 

  • Quan, H., Luo, X., Capizzi, T.: Multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active control. Stat. Med. 24, 2151–2170 (2005)

    Article  MathSciNet  Google Scholar 

  • Rom, D.M.: A sequentially rejective test procedure based on a modified Bonferroni inequality. Biometrika 77, 663–665 (1990)

    Article  MathSciNet  Google Scholar 

  • Roy, S.N.: On a heuristic method for test construction and its use in multivariate analysis. Ann. Stat. 24, 220–238 (1953)

    Article  MATH  Google Scholar 

  • Sarkar, S.K., Chang, C.K.: Simes’ method for multiple hypothesis testing with positively dependent test statistics. J. Am. Stat. Assoc. 92, 1601–1608 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Shaffer, J.P.: Modified sequentially rejective multiple test procedures. J. Am. Stat. Assoc. 81, 826–831 (1986)

    Article  MATH  Google Scholar 

  • Sidak, Z.: Rectangular confidence regions for the means of multivariate normal distributions. J. Am. Stat. Assoc. 62, 626–633 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  • Simes, R.J.: An improved Bonferroni procedure for multiple tests of significance. Biometrika 63, 655–660 (1986)

    MathSciNet  Google Scholar 

  • Storey, J., Taylor, J., Siegmund, D.: Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B, 66(1), 187–205 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  • Tsong, Y., Wang, S.J., Hung, H.M.J., Cui, L.: Statistical issues on objective, design, and analysis of noninferiority active-controlled clinical trial. J. Biopharm. Stat. 13, 29–41 (2003)

    Article  MATH  Google Scholar 

  • Wang, X.: Gatekeeping procedures for multiple endpoints. Ph.D. Dissertation, Department of Statistics, Northwestern University, Evanston (2006)

    Google Scholar 

  • Westfall, P.H., Krishen, A.: Optimally weighted, fixed-sequence and gatekeeper multiple testing procedures. J. Stat. Plan. Inference 99, 25–41 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  • Westfall, P.H., Tobias, R.D., Rom, D., Wolfinger, R.D., Hochberg, Y.: Multiple Comparisons and Multiple Tests Using the SAS System. SAS Institute, Cary (1999)

    Google Scholar 

  • Westfall, P.H., Young, S.S.: Resampling-Based Multiple Testing: Examples and Methods for p-value Adjustment. Wiley, New York (1993)

    Google Scholar 

  • Wiens, B.L.: A fixed-sequence Bonferroni procedure for testing multiple endpoints. Pharm. Stat. 2, 211–215 (2003)

    Article  Google Scholar 

  • Wiens, B., Dmitrienko, A.: The fallback procedure for evaluating a single family of hypotheses. J. Biopharm. Stat. 15, 929–942 (2005)

    Article  MathSciNet  Google Scholar 

  • Xu, H.: Using the partitioning principle to control generalized familywise error rate. Ph.D. Dissertation, The Ohio State University, Columbus (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Biometrics, AMAG Pharmaceuticals, Inc., Hayden Ave. 100, 02421, Lexington, Massachusetts, USA

    Mark Chang

Authors
  1. Mark Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Chang, M. (2011). Multiple-Hypothesis Testing Strategy. In: Modern Issues and Methods in Biostatistics. Statistics for Biology and Health. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9842-2_1

Download citation

Publish with us

Policies and ethics

Search

Navigation