Blind Guides for Explorers: P Values, Subgroups and Data Dredging
Research investigators are trained to be thorough, to find everything worth findings in their data. Having invested great time and effort in their studies, these scientists want and need to examine the data systematically and completely. In clinical experiments where there are multiple endpoints, investigators will examine the effect of therapy on each one, with the anticipated reward of identifying a therapy effect for the experiment’s primary endpoint. However, investigators are aware that there may be unanticipated findings in other analyses, that, unlike the primary endpoint analysis, are nonprospectively stated evaluations. Investigators believe that, like unsuspected pots of gold, these tantalizing surprises might lie just under the surface, hidden from view, waiting to be found. If the experiment demonstrated that an intervention reduces the incidence of heart attacks, then maybe there is a hidden relationship between marital status and heart attacks. Perhaps there is an unanticipated relationship between the patient’s astrologic sign and the occurrence of a heart attack? Sometimes it is not the investigator who is raising these questions. In the process of publication, reviewers of the manuscript will sometimes ask that additional analyses be carried out. These analyses can include considering the effect of the intervention in subsets of the data. Does the therapy work equally well in women and men? Does it work equally in different racial groups? What about in patients with a previous heart attack? These analyses are demanded by others, but are also not prospectively stated.
KeywordsHeart Attack Confirmatory Analysis Strip Mining Subgroup Finding Prevent Heart Attack Trial
Unable to display preview. Download preview PDF.
- 3.Davis, B.R., Cutler, J.A., Gordon, D.J., Furberg, C.D., Wright, J.T., Cushman,C., Grimm, R.H., LaRosa, J., Whelton, P.K., Perry, H.M., Alderman, M.H., Ford, C.E., Oparil, S., Francis, C., Proscham, M., Pressel, S., Black, H.R., and Hawkins, C.M., for the ALLHAT Research Group. (1996) “Rationale and design for the antihypertensive and lipid lowering treatment to prevent heart attack trial,” American Journal of Hypertension 9: 342–360.PubMedCrossRefGoogle Scholar
- 4.Yusef, S., Wittes, J., Probstfield, J., and Tyroler, H.A., (1991) “Analysis and Interpretation of Treament Effects in Subgroups of Patients in Randomized Clinical Trials,” Journal of the American Medical Association 2266: 93–98.Google Scholar
- 5.Sacks, F.M., Pfeffer, M.A., Moyé, L.A., Rouleau, J.L., Rutherford, J.D., Cole, T.G., Brown, L., Warnica, J.W., Arnold, J.M., Wun, C.C., Davis, B.R., and Braunwald, E., for the Cholesterol and recurrent Events Trial Investigators (1996) “The effect of pravastatin on coronary events after myocardial infarction in patients with average cholesterol levels,” N Engl J Med 335: 1001–1009.PubMedCrossRefGoogle Scholar
- 6.Lewis, S.J., Sacks, F.M., Mitchell, J.S., East, C., Glasser, S., Kell, S., Letterer, R., Limacher, M., Moyé, L.A., Rouleau, J.L., Pfeffer, M.A., and Braunwald, E., (1998) “Effect of pravastatin on cardiovascular events in women after myocardial infarction: the cholesterol and recurrent events (CARE) trial,” Journal of the American College of Cardiology 32: 140–146.PubMedCrossRefGoogle Scholar
- 7.Sacks, F.M., Moyé, L.A., Davis, B.R., Cole, T.G., Rouleau, J.L., Nash, D.T., Pfeffer, M.A., and Braunwald, E. (1998) “Relationship between plasma LDL concentrations during treatment with pravastatin and recurrent coronary events in the Cholesterol and Recurrent Events trial,” Circulation 97: 1446–1452.PubMedCrossRefGoogle Scholar
- 8.Cohen, A.J. “Replication” Epidemiology 8: 341–343.Google Scholar
- 9.Beaglehole, R., Bonita, R. and Kjellström, T., (1993) Causation in epidemiology, Geneva: World Health Organization, pp. 71–81.Google Scholar
- 12.Pfeffer, M.A., Brauwald, E., and Moyé L.A., (1992) “Effect of Captopril on mortality and morbidity in patients with left ventricular dysfunction after myocardial infarction—results of the Survival and Ventricular Enlargement Trial,” New England Journal of Medicine 327: 669–677.PubMedCrossRefGoogle Scholar