Abstract
Partial least squares (PLS) was first introduced by Wold in the mid 1960s as a heuristic algorithm to solve linear least squares (LS) problems. No optimality property of the algorithm was known then. Since then, however, a number of interesting properties have been established about the PLS algorithm for regression analysis (called PLS1). This paper shows that the PLS estimator for a specific dimensionality S is a kind of constrained LS estimator confined to a Krylov subspace of dimensionality S. Links to the Lanczos bidiagonalization and conjugate gradient methods are also discussed from a somewhat different perspective from previous authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdi, H.: Partial least squares regression. In: Salkind, N.J. (ed.) Encyclopedia of Measurement and Statistics, pp. 740–54. Sage, Thousand Oaks (2007)
Arnoldi, W.E.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Q. Appl. Math. 9, 17–29 (1951)
Bro, R., Eldén, L.: PLS works. J. Chemom. 23, 69–71 (2009)
de Jong, S.: SIMPLS: an alternative approach to partial least squares regression. J. Chemom. 18, 251–263 (1993)
Eldén, L.: Partial least-squares vs Lanczos bidiagonalization–I: analysis of a projection method for multiple regression. Comput. Stat. Data Anal. 46, 11–31 (2004)
Golub, G.H., van Loan, C.F.: Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore (1989)
Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur Stand. 49, 409–436 (1951)
Lohmöller, J.B.: Latent Variables Path-Modeling with Partial Least Squares. Physica-Verlag, Heidelberg (1989)
Phatak, A., de Hoog, F.: Exploiting the connection between PLS, Lanczos methods and conjugate gradients: alternative proofs of some properties of PLS. J. Chemom. 16, 361–367 (2002)
Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Saunders, C., et al. (eds.) SLSFS 2005. LNCS 3940, pp. 34–51. Springer, Berlin (2006)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society of Industrial and Applied Mathematics, Philadelphia (2003)
Saad, Y., Schultz, M.H.: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Comput. 7, 856–869 (1986)
Takane, Y.: Constrained Principal Component Analysis and Related Techniques. CRC Press, Boca Raton (2014)
Wold, H.: Estimation of principal components and related models by iterative least squares. In: Krishnaiah, P.R. (ed.) Multivariate Analysis, pp. 391–420. Academic, New York (1966)
Wold, H. (1982) Soft modeling: the basic design and some extensions. In: Jöreskog, K.G., Wold, H. (eds.) Systems Under Indirect Observations, Part 2, pp. 1–54. North-Holland, Amsterdam (1982)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Takane, Y., Loisel, S. (2016). On the PLS Algorithm for Multiple Regression (PLS1). In: Abdi, H., Esposito Vinzi, V., Russolillo, G., Saporta, G., Trinchera, L. (eds) The Multiple Facets of Partial Least Squares and Related Methods. PLS 2014. Springer Proceedings in Mathematics & Statistics, vol 173. Springer, Cham. https://doi.org/10.1007/978-3-319-40643-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-40643-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-40641-1
Online ISBN: 978-3-319-40643-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)