Regression and Model Fitting

Abstract

In previous chapters, we have discussed methods for simulating a system defined by a particular model, but we haven’t talked about how to build such a model or verify that the model we are using is correct or suitable. Building/finding the optimal model for any system is one of the most important tasks in any engineering design. If the system is simple, we can study the physics behind it and come up with an analytical and mathematical model. But most real world systems are very complex and finding an exact model for them is very difficult. Another way to build a model is by empirical fitting. In this method, we first perform a series of experiments on the system to collect samples of input and output signals. Then we try to find a relation between output and inputs with some basic assumptions about the system. Given any two data sets, the derivation of a relation is known as regression. In this chapter, we will first learn the basic concepts and methods of regression and then use this knowledge to perform model fitting.