Analysis Issues – A Lot of Choices

In the statistical analysis of data from a clinical trial, one would expect the same set of data to lead to the same result. However analysts have many choices – what statistical test to use, what type of data to analyze, whether the underlying mathematical assumptions of the tests they use were met, etc. Depending on those decisions, different conclusions about a treatment's efficacy and safety could be supported. Furthermore, today the recommended research practice for analyzing a study is the intention-to-treat approach whereby all subjects the researcher intended to treat are included in the analysis, even if they didn't meet the protocol requirements or left the trial prematurely. This approach can cause a bias, but the alternative that allows researcher to decide who should be dropped and who should remain in an analysis can also introduce bias. The importance of statistical decisions is reflected in an astonishing report claiming that over half of all medical research findings are false. Too few subjects, small differences between treatments, the number of tests preformed and the incorrect interpretation of probabilities were used to support that premise.