Linear Algebra and Optimization for Machine Learning

A Textbook

  • Charu C. Aggarwal

Table of contents

  1. Front Matter
    Pages I-XXI
  2. Charu C. Aggarwal
    Pages 41-95
  3. Charu C. Aggarwal
    Pages 97-139
  4. Charu C. Aggarwal
    Pages 141-203
  5. Charu C. Aggarwal
    Pages 205-253
  6. Charu C. Aggarwal
    Pages 255-297
  7. Charu C. Aggarwal
    Pages 299-337
  8. Charu C. Aggarwal
    Pages 339-378
  9. Charu C. Aggarwal
    Pages 379-410
  10. Charu C. Aggarwal
    Pages 411-446
  11. Charu C. Aggarwal
    Pages 447-482
  12. Back Matter
    Pages 483-495

About this book


This textbook introduces linear algebra and optimization in the context of machine learning. Examples and exercises are provided throughout this text book together with access to a solution’s manual. This textbook targets graduate level students and professors in computer science, mathematics and data science. Advanced undergraduate students can also use this textbook. The chapters for this textbook are organized as follows:

1. Linear algebra and its applications: The chapters focus on the basics of linear algebra together with their common applications to singular value decomposition, matrix factorization, similarity matrices (kernel methods), and graph analysis. Numerous machine learning applications have been used as examples, such as spectral clustering, kernel-based classification, and outlier detection. The tight integration of linear algebra methods with examples from machine learning differentiates this book from generic volumes on linear algebra. The focus is clearly on the most relevant aspects of linear algebra for machine learning and to teach readers how to apply these concepts.

2. Optimization and its applications: Much of machine learning is posed as an optimization problem in which we try to maximize the accuracy of regression and classification models. The “parent problem” of optimization-centric machine learning is least-squares regression. Interestingly, this problem arises in both linear algebra and optimization, and is one of the key connecting problems of the two fields.  Least-squares regression is also the starting point for support vector machines, logistic regression, and recommender systems. Furthermore, the methods for dimensionality reduction and matrix factorization also require the development of optimization methods. A general view of optimization in computational graphs is discussed together with its applications to back propagation in neural networks. 

A frequent challenge faced by beginners in machine learning is the extensive background required in linear algebra and optimization. One problem is that the existing linear algebra and optimization courses are not specific to machine learning; therefore, one would typically have to complete more course material than is necessary to pick up machine learning. Furthermore, certain types of ideas and tricks from optimization and linear algebra recur more frequently in machine learning than other application-centric settings. Therefore, there is significant value in developing a view of linear algebra and optimization that is better suited to the specific perspective of machine learning.


Linear Algebra Optimization Machine Learning Deep Learning Neural Networks Dynamic Programming Support Vector Machines Linear Regression Matrix Algebra Numerical Algebra Gradient Descent

Authors and affiliations

  • Charu C. Aggarwal
    • 1
  1. 1.IBM T.J. Watson Research CenterYorktown HeightsUSA

Bibliographic information

  • DOI
  • Copyright Information Springer Nature Switzerland AG 2020
  • Publisher Name Springer, Cham
  • eBook Packages Computer Science
  • Print ISBN 978-3-030-40343-0
  • Online ISBN 978-3-030-40344-7
  • Buy this book on publisher's site