Skip to main content
SpringerLink
Log in

Statistical Methods for the Assessment of Clinical Relevance

  • Chapter
  • First Online:
Developments in Statistical Evaluation of Clinical Trials

Abstract

It is commonly accepted that the results of clinical trials to be important for medical practice do not only have to be statistically significant but also clinically relevant. While an elaborated and canonical methodology exists for statistical significance tests, there is no common sense so far on how to judge the clinical relevance of a medical finding. The assessment of the clinical relevance of a study result should provide quantified information about its practical importance. For this, both statistical procedures and appropriate effect measures on which the relevance judgment is based are required. The test for relevant superiority and the relevance assessment based on the observed effect are presented as two statistical approaches for the assessment of clinical relevance. The properties of these procedures are investigated and contrasted. Furthermore, an overview of effect measures used for relevance assessment is given and their characteristics are illustrated. Application of the methods is illustrated with a clinical trial example.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Browne, R.H.: The t-test p value and its relationship to the effect size and P(X > Y). The American Statistician 64, 30–33 (2010)

    Google Scholar 

  2. Brunner, E., Munzel, U.: Nichtparametrische Datenanalyse. Springer, Berlin (2002)

    Google Scholar 

  3. Buyse, M.: Reformulating the hazard ratio to enhance communication with clinical investigators. Clinical Trials 5, 641–642 (2008)

    Google Scholar 

  4. Cohen, J.: The cost of dichotomization. Applied Psychological Measurement 7, 249–253 (1983)

    Google Scholar 

  5. Committee for Medicinal Products for Human Use (CHMP): Guideline on the choice of the non-inferiority margin (2005). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003636.pdf. Cited 14 December 2012

  6. Committee for Medicinal Products for Human Use (CHMP): Guideline on clinical investigation of medicinal products for the treatment of multiple sclerosis (2006). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003485.pdf. Cited 14 December 2012

  7. Committee for Medicinal Products for Human Use (CHMP): Guideline on clinical investigation of medicinal products indicated for the treatment of social anxiety disorder (SAD) (2006). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003490.pdf. Cited 14 December 2012

  8. Committee for Medicinal Products for Human Use (CHMP): Guideline on clinical investigation of medicinal products in the treatment of Parkinson’s disease (2008). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003540.pdf. Cited 14 December 2012

  9. Committee for Medicinal Products for Human Use (CHMP): Guideline on medicinal products for the treatment of Alzheimer’s disease and other dementias (2008). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003562.pdf. Cited 14 December 2012

  10. Committee for Medicinal Products for Human Use (CHMP): Guideline on clinical investigation of medicinal products in the treatment of epileptic disorders (2010). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2010/01/WC500070043.pdf. Cited 14 December 2012

  11. Committee for Medicinal Products for Human Use (CHMP): Guideline on the clinical investigation of medicinal products for the treatment of attention deficit hyperactivity disorder (ADHD) (2010). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2010/08/WC500095686.pdf. Cited 14 December 2012

  12. Committee for Medicinal Products for Human Use (CHMP): Draft Guideline on clinical investigation of medicinal products in the treatment of depression (2011). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2011/10/WC500116160.pdf. Cited 14 December 2012

  13. Committee for Medicinal Products for Human Use (CHMP): Guideline on the evaluation of medicinal products indicated for treatment of bacterial infections (2011). URL http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003417.pdf. Cited 14 December 2012

  14. Conover, W.J., Salsburg, D.: Locally most powerful tests for detecting treatment effects when only a subset of patients can be expected to “respond” to treatment. Biometrics 44, 189–196 (1988)

    Google Scholar 

  15. Cook, R.J., Sackett, D.L.: The number needed to treat: a clinically useful measure of treatment effect. BMJ 310, 452 (1995)

    Google Scholar 

  16. Deyi, B.A., Kosinski, A.S., Snapinn, S.M.: Power considerations when a continuous outcome variable is dichotomized. Journal of Biopharmaceutical Statistics 8, 337–352 (1998)

    Google Scholar 

  17. Fedorov, V., Mannino, F., Zhang, R.: Consequences of dichotomization. Pharmaceutical Statistics 8, 50–61 (2009)

    Google Scholar 

  18. Friedman, L.: Clinical significance vs. statistical significance. In: P. Armitage, T. Coltan (eds.) Encyclopedia of Biostatistics, pp. 676–678. John Wiley and Sons, New York (1998)

    Google Scholar 

  19. Gondan, M.: How to demonstrate both statistical significance and clinical relevance in survival data. Internal Report, Institute of Medical Biometry and Informatics, University of Heidelberg (2012)

    Google Scholar 

  20. Heidrich, H., Cachovan, M., Kreutzig, A., Rieger, H., Trampisch, H.J.: Guidelines for therapeutic studies in Fontaine’s stages II - IV peripheral arterial occlusive disease. Vasa 24, 114–119 (1995)

    Google Scholar 

  21. Heidrich, H., Trampisch, H.J., Röhmel, J.: Comments on guidelines for therapeutic studies in Fontaine’s stage II-IV peripheral arterial occlusive disease. Vasa 25, 73–75 (1996)

    Google Scholar 

  22. Johnson, R.A., Verrill, S., Moore II, D.H.: Two-sample rank tests for detecting changes that occur in a small proportion of the treated population. Biometrics 43, 641–655 (1986)

    Google Scholar 

  23. Jones, P.W.: Interpreting thresholds for a clinically significant change in health status in asthma and COPD. European Respiratory Journal 19, 396–404 (2002)

    Google Scholar 

  24. Kieser, M., Friede, T., Gondan, M.: Assessment of statistical significance and clinical relevance. Statistics in Medicine 32(10), 1707–1719 (2012). DOI 10.1002/sim.5634

    Google Scholar 

  25. Kieser, M., Hauschke, D.: Assessment of clinical relevance by considering point estimates and associated confidence intervals. Pharmaceutical Statistics 4, 101–107 (2005)

    Google Scholar 

  26. Kieser, M., Röhmel, J., Friede, T.: Power and sample size determination when assessing the clinical relevance of trial results by ‘responder analyses’. Statistics in Medicine 23, 3287–3305 (2004)

    Google Scholar 

  27. Koziol, J., Jia, Z.: The concordance index C and the Mann-Whitney parameter Pr(X > Y) with randomly censored data. BMJ 51, 467–474 (2009)

    Google Scholar 

  28. MacCallum, R.C., Zhang, S., Preacher, K.J., Rucker, D.D.: On the practice of dichotomization of quantitative variables. Psychological Methods 7, 19–40 (2002)

    Google Scholar 

  29. Newcombe, R.G.: Confidence intervals for an effect size measure based on the Mann-Whitney statistic. Part 1: general issues and tail-area-based methods. Statistics in Medicine 25, 543–557 (2006)

    Google Scholar 

  30. Newcombe, R.G.: Confidence intervals for an effect size measure based on the Mann-Whitney statistic. Part 2: asymptotic methods and evaluation. Statistics in Medicine 25, 559–573 (2006)

    Google Scholar 

  31. Okun, M.S., Gallo, B.V., Mandybur, G., Jagid, J., Foote, K.D., Revilla, F.J., Alterman, R., Jankovic, J., Simpson, R., Junn, F., Verhagen, L., Arle, J.E., Ford, B., Goodman, R.R., Stewart, R.M., Horn, S., Baltuch, G.H., Kopell, B.H., Marshall, F., Peichel, D., Pahwa, R., Lyons, K.E., Tröster, A.I., Vitek, J.L., Tagliati, M., for the SJM DBS Study Group: Subthalamic deep brain stimulation with a constant-current device in Parkinson’s disease: an open-label randomised controlled trial. The Lancet Neurology 11, 140–149 (2012)

    Google Scholar 

  32. Peacock, J.L., Sauzet, O., Ewings, S.M., Kerry, S.M.: Dichotomising continuous data while retaining statistical power using a distributional approach. Statistics in Medicine 31, 3089–3103 (2012)

    Google Scholar 

  33. Schacht, A., Bogaerts, K., Bluhmki, E., Lesaffre, E.: A new nonparametric approach for baseline covariate adjustment for two-group comparative studies. Biometrics 64, 1110–1116 (2008)

    Google Scholar 

  34. Seiler, C.M., Fröhlich, B.E., Veit, J.A., Gazyakan, E., Wente, M.N., Wollermann, C., Deckert, A., Witte, S., Victor, N., Büchler, M.W., Knaebel, H.P.: Protocol design and current status of CLIVIT: a randomized controlled multicentre relevance trial comparing clips versus ligatures in thyroid surgery. Trials 7, 27 (2006)

    Google Scholar 

  35. Senn, S.: Disappointing dichotomies. Pharmaceutical Statistics 2, 239–240 (2003)

    Google Scholar 

  36. Snapinn, S.M., Jiang, Q.: Responder analyses and the assessment of a clinically relevant treatment effect. Trials 8, 31 (2007)

    Google Scholar 

  37. Uryniak, T., Chan, I.S.F., Fedorov, V.V., Jiang, Q., Oppenheimer, L., Snapinn, S.M., Teng, C.H., Zhang, J.: Responder analyses - a PhRMA position paper. Statistics in Biopharmaceutical Research 3, 476–487 (2011)

    Google Scholar 

  38. Victor, N.: Clinically relevant differences and shifted nullhypotheses. Methods of Information in Medicine 26, 109–116 (1987)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Medical Biometry and Informatics, University of Heidelberg, Heidelberg, Germany

    Meinhard Kieser

Authors
  1. Meinhard Kieser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meinhard Kieser .

Editor information

Editors and Affiliations

  1. Department of Biostatistics, Erasmus Medical Center, Rotterdam, The Netherlands

    Kees van Montfort

  2. Behavioural Science Institute, University of Nijmegen, Nijmegen, The Netherlands

    Johan Oud

  3. Department of Hematology, Erasmus Medical Center, Rotterdam, The Netherlands

    Wendimagegn Ghidey

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kieser, M. (2014). Statistical Methods for the Assessment of Clinical Relevance. In: van Montfort, K., Oud, J., Ghidey, W. (eds) Developments in Statistical Evaluation of Clinical Trials. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-55345-5_11

Download citation

Publish with us

Policies and ethics

Search

Navigation