On the PLS Algorithm for Multiple Regression (PLS1)

  • Yoshio TakaneEmail author
  • Sébastien Loisel
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 173)


Partial least squares (PLS) was first introduced by Wold in the mid 1960s as a heuristic algorithm to solve linear least squares (LS) problems. No optimality property of the algorithm was known then. Since then, however, a number of interesting properties have been established about the PLS algorithm for regression analysis (called PLS1). This paper shows that the PLS estimator for a specific dimensionality S is a kind of constrained LS estimator confined to a Krylov subspace of dimensionality S. Links to the Lanczos bidiagonalization and conjugate gradient methods are also discussed from a somewhat different perspective from previous authors.


Krylov subspace NIPALS PLS1 algorithm Lanczos bidiagonalization Conjugate gradients Constrained principal component analysis (CPCA) 


  1. Abdi, H.: Partial least squares regression. In: Salkind, N.J. (ed.) Encyclopedia of Measurement and Statistics, pp. 740–54. Sage, Thousand Oaks (2007)Google Scholar
  2. Arnoldi, W.E.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Q. Appl. Math. 9, 17–29 (1951)MathSciNetzbMATHGoogle Scholar
  3. Bro, R., Eldén, L.: PLS works. J. Chemom. 23, 69–71 (2009)CrossRefGoogle Scholar
  4. de Jong, S.: SIMPLS: an alternative approach to partial least squares regression. J. Chemom. 18, 251–263 (1993)Google Scholar
  5. Eldén, L.: Partial least-squares vs Lanczos bidiagonalization–I: analysis of a projection method for multiple regression. Comput. Stat. Data Anal. 46, 11–31 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  6. Golub, G.H., van Loan, C.F.: Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore (1989)zbMATHGoogle Scholar
  7. Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur Stand. 49, 409–436 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Lohmöller, J.B.: Latent Variables Path-Modeling with Partial Least Squares. Physica-Verlag, Heidelberg (1989)CrossRefzbMATHGoogle Scholar
  9. Phatak, A., de Hoog, F.: Exploiting the connection between PLS, Lanczos methods and conjugate gradients: alternative proofs of some properties of PLS. J. Chemom. 16, 361–367 (2002)Google Scholar
  10. Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Saunders, C., et al. (eds.) SLSFS 2005. LNCS 3940, pp. 34–51. Springer, Berlin (2006)Google Scholar
  11. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society of Industrial and Applied Mathematics, Philadelphia (2003)CrossRefzbMATHGoogle Scholar
  12. Saad, Y., Schultz, M.H.: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Comput. 7, 856–869 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  13. Takane, Y.: Constrained Principal Component Analysis and Related Techniques. CRC Press, Boca Raton (2014)zbMATHGoogle Scholar
  14. Wold, H.: Estimation of principal components and related models by iterative least squares. In: Krishnaiah, P.R. (ed.) Multivariate Analysis, pp. 391–420. Academic, New York (1966)Google Scholar
  15. Wold, H. (1982) Soft modeling: the basic design and some extensions. In: Jöreskog, K.G., Wold, H. (eds.) Systems Under Indirect Observations, Part 2, pp. 1–54. North-Holland, Amsterdam (1982)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.University of VictoriaVictoriaCanada
  2. 2.Heriot-Watt UniversityEdinburghUK

Personalised recommendations