Safety Analysis and the Alternative Hypothesis

Abstract

The type I error is the chance of finding a difference where there is none.

The type II error is the chance of finding no difference where there is one.

With safety analyses we try and reject the alternative hypothesis of an adverse effect, with efficacy analyses we try and reject the null hypothesis of no treatment effect.

With efficacy analyses the null hypothesis is usually rejected with a type I error of 5% and a type II error may very well be as large as 50%. With safety analysis we are more interested in smaller type II errors, because we will have increasing chance of rightly rejecting the alternative hypothesis, and that is good. After all, the main purpose of safety analyses in clinical trials is to find no adverse effects rather than the opposite. With a type I error of 20% instead of 5% we can reject the type II error at 25%. This means, that the chance of rightly rejecting the alternative hypothesis is 75%, rather than 50%. This result is in better agreement with the incentive to rightly reject the alternative hypothesis, which is the main incentive of safety assessments in clinical trials.