Updating for a New Setting

Abstract

A prediction model ideally provides valid predictions of outcome for individual patients at another setting than where the model was developed, e.g., differing in time and place. The validity of predictions can be assessed by comparing observed outcomes and predictions when empirical data from this external setting are available. Various patterns of invalidity may, however, be observed as we have seen in Chap. 19. Detection of calibration-in-the-large problems should have top priority since miscalibration can cause systematically wrong decision-making with the model (negative Net Benefit). Obviously, we may subsequently aim to update the model to improve predictions for future patients from the new setting. We discuss several approaches for updating a previously developed model. The risk is that simply re-estimating all regression coefficients in a model might replace reliable but slightly biased estimates by unbiased but very unreliable ones, particularly if the validation data set is relatively small.