Machine Learning Principles

Abstract

Machine learning (ML) algorithms seek to extract the most beneficial information out of the raw data. As mentioned in previous chapters, preprocess data preparation actions might be needed in order to make the data set ready to be fed into the ML algorithms. ML algorithms have gotten the attention of researchers all over the world during the last few decades. In traditional algorithms, the analyst had to define the rules, which was not always accessible or easily possible, in order to obtain the output. ML algorithms develop the models or rules based on the training data set which include input and output data points. ML algorithms try to understand more beneficial information regarding the system based on the training data set. The built model can be tested using the verification data set which does not have any overlap with the training sets. If the model acquires the acceptable performance measures, it can be applied for other cases to perform the prediction process. It should be considered that ML algorithms are not supposed to do the magic. Indeed, they are supposed to explore and analyze the training data, which the human brain is not able to handle that, to develop a predictive model in which its performance is acceptable for the verification data set. Predictive models should be generalized which means performing satisfactorily for both training and verifying data sets.

The main purpose of this chapter is to present the most commonly used statistical and ML algorithms more toward the application side rather than the theories behind the development of the algorithms. Therefore, this chapter of the book is summarizing the most important applications and features of the ML algorithms. It should be noted that the term “machine learning (ML)” is not interchangeable with “artificial intelligence (AI).” Indeed, ML is a subfield of the AI which sometimes referred to as “predictive modeling” or “predictive algorithms.” One of the most famous theorems in ML area is called “no free lunch.” This theorem indicates that there is not an algorithm which can globally perform better than others for all the applications. Based on this theorem, an analyst should have advanced knowledge regarding the ability of the algorithms in order to select the most applicable one.

In this chapter, several commonly used supervised ML algorithms are presented in detail. In general, supervised algorithms are categorized into either regression or classification tasks. For each of the prediction task, the theories behind a few ML algorithms are explained. A few examples of the algorithm selection criteria are also discussed in this chapter. A numerical example is also presented for regression and classification tasks in order to explain the performance of the various algorithms given the same data set.