How to guarantee finding a statistically significant difference: the use and abuse of subgroup analyses
In 1988 the Lancet published a very large randomised clinical trial of intravenous streptokinase, oral aspirin, both, or neither for treatment of suspected acute myocardial infarction . The ISIS-2 trial recruited 17,187 patients from 417 hospitals. The authors concluded that there were benefits both from streptokinase and from aspirin. This paper contained a complex table reporting subgroup analyses, and rather intriguingly, the first analysis was by astrological birth sign. The results suggested that for people born under the star signs Gemini and Libra, aspirin was no better than placebo; for others, aspirin had a strongly beneficial effect. Why, one might wonder, did a highly respected journal publish such arrant nonsense?
The use of patient reported outcomes (PROs) in clinical trials can lead to problems with ‘multiple testing’, that is, the testing of multiple hypotheses and the associated problem of how to interpret resultant P values . Most commonly, this problem arises...
KeywordsMigraine Subgroup Analysis Streptokinase Rizatriptan Interaction Test
- 1.ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. (1988). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet, ii, 349–360.Google Scholar
- 4.Sankoh, A. J., Huque, M. F., & Dubey, S. D. (1997). Some comments on frequently used multiple endpoint adjustment methods in clinical trials. Statistics in Medicine, 16, 2529–2542. doi: 10.1002/(SICI)1097-0258(19971130)16:22<2529::AID-SIM692>3.0.CO;2-J.PubMedCrossRefGoogle Scholar
- 6.Brookes, S. T., Whitely, E., Egger, M., Smith, G. D., Mulheran, P. A., & Peters, T. J. (2004). Subgroup analyses in randomized trials: Risks of subgroup-specific analyses; power and sample size for the interaction test. Journal of Clinical Epidemiology, 57, 229–236. doi: 10.1016/j.jclinepi.2003.08.009.PubMedCrossRefGoogle Scholar
- 10.Brookes, S. T., Whitley, E., Peters, T. J., Mulheran, P. A., Egger, M., & Davey Smith, G. (2001). Subgroup analyses in randomised controlled trials: Quantifying the risks of false-positives and false-negatives. Health Technology Assessment, 5(33), 1–56. From http://www.hta.nhs.uk/fullmono/mon533.pdf. Accessed 12 March 2009.
- 13.Martin, V., Cady, R., Mauskop, A., Seidman, L. S., Rodgers, A., Hustard, C. M., et al. (2008). Efficacy of rizatriptan for menstrual migraine in an early intervention model: A prospective subgroup analysis of the rizatriptan TAME (Treat a Migraine Early) studies. Headache, 48, 226–235.PubMedCrossRefGoogle Scholar