Bayesian Monitoring

Classical procedures that focus on the type 1 error rate necessarily punish us for taking interim looks at the data; the more we look, the higher the boundaries become. Meier (1975) [M75] puzzled over this: “…it seems hard indeed to accept the notion that I should be influenced in my judgment by how frequently he peeked at the data while he was collecting it.” It also seems strange that conclusions should depend on how the data were collected; for example, different p-values will obtain for the same trial results depending on whether the trial was designed for a specified number of patients or a specified number of events. This led Berger and Berry (1988) [BB88] to quip, “Indeed, if the investigator died after reporting the data but before reporting the design of the experiment, it would be impossible to calculate the p-value or other standard measures of evidence.” Also, frequentists do not have a universally accepted method of incorporating unplanned data, such as would occur if investigators decided to extend a trial on the basis of promising but not statistically significant results.

Bayesian methodology offers a way of overcoming these obstacles by focusing not on controlling the error rate under a single point null hypothesis Ө = 0, but on the trade-off between type 1 and type 2 errors, using prior opinion to help decide how likely each is. A major advantage is that Bayesian monitoring boundaries do not depend at all on how many looks have been taken or how the data were sampled (e.g., whether the trial continued until a fixed number of patients was reached or until a fixed number of events occurred). Bayesian methods also offer a formal way of incorporating information from outside the trial. Their major disadvantage is that they require us to formalize our prior opinion of the treatment effect through a prior distribution, and different priors lead to different conclusions for the same data.