Data Modeling IV-Fine-Tuning Your Model

Abstract

We have called this chapter “fine-tuning your model” because it covers a variety of concepts and methods that all share a common goal: to get more out of your model. Section 6.1 revisits the topic of model quality. Recall that we have already discussed the quality of a model in Chapter 3 (in Section 3.2.3, to be more precise). Why do we revisit the same topic again in this chapter? Because a model’s quality depends on one’s point of view. In Section 3.2.3, we have argued that a model is “good” if it explains a large portion of the uncertainty in the response variable or, in other words, if it explains many of the patterns observed in the data. However, since observed data is necessarily data from the past, our assessment of a model’s quality has been retrospective until this point. But what about a model’s ability to predict the future? A model that can explain the past very well does not necessarily predict the future equally well. In Section 6.1, we will thus revisit the topic of a model’s quality and discuss the shortcomings of model fit statistics such as R-squared. We will also change our point of view and discuss ideas that can help us assess a model’s ability to “look into the future.”