Skip to main content
SpringerLink
Log in

Advances in Incremental PCA Algorithms

  • Conference paper
  • First Online:
Parallel Processing and Applied Mathematics (PPAM 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10777))

Abstract

We present a range of new incremental (single-pass streaming) algorithms for incremental principal components analysis (IPCA) and show that they are more effective than exiting ones. IPCA algorithms process the columns of a matrix A one at a time and attempt to build a basis for a low-dimensional subspace that spans the dominant subspace of A. We present a unified framework for IPCA algorithms, show that many existing ones are parameterizations of it, propose new sophisticated algorithms, and show that both the new algorithms and many existing ones can be implemented more efficiently than was previously known. We also show that many existing algorithms can fail even in easy cases and we show experimentally that our new algorithms outperform existing ones.

This research is supported by grants 965/15 and 863/15 from the Israel Science Foundation (founded by the Israel Academy of Sciences and Humanities) and by a grant from the United States-Israel Bi-national Science Foundation (BSF).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brand, M.: Incremental singular value decomposition of uncertain data with missing values. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 707–720. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47969-4_47

    Chapter  Google Scholar 

  2. Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415(1), 20–30 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  3. Chahlaoui, Y., Gallivan, K.A., Dooren, P.V.: An incremental method for computing dominant singular spaces. In: Computational Information Retrieval, pp. 53–62 (2001)

    Google Scholar 

  4. Chahlaoui, Y., Gallivan, K.A., Van Dooren, P.: Recursive calculation of dominant singular subspaces. SIAM J. Matrix Anal. Appl. 25(2), 445–463 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chandrasekaran, S., Manjunath, B., Wang, Y.-F., Winkeler, J., Zhang, H.: An eigenspace update algorithm for image analysis. Graph. Models Image Process. 59(5), 321–332 (1997)

    Article  Google Scholar 

  6. Desai, A., Ghashami, M., Phillips, J.M.: Improved practical matrix sketching with guarantees. IEEE Trans. Knowl. Data Eng. 28(7), 1678–1690 (2016)

    Article  MATH  Google Scholar 

  7. Ghashami, M., Liberty, E., Phillips, J.M., Woodruff, D.P.: Frequent directions: simple and deterministic matrix sketching. SIAM J. Comput. 45(5), 1762–1792 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Halpern, T.: Fast and robust algorithms for large-scale streaming PCA. Master’s thesis, Tel Aviv University, July 2017. http://www.tau.ac.il/~stoledo/Pubs/MSc_Tal_Halpern.pdf

  9. Levey, A., Lindenbaum, M.: Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Trans. Image Process. 9(8), 1371–1374 (2000)

    Article  MATH  Google Scholar 

  10. Liberty, E.: Simple and deterministic matrix sketching. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 581–588. ACM (2013)

    Google Scholar 

  11. Manjunath, B., Chandrasekaran, S., Wang, Y.-F.: An eigenspace update algorithm for image analysis. In: Proceedings International Symposium on Computer Vision, pp. 551–556. IEEE (1995)

    Google Scholar 

  12. O’Brien, G.W.: Information management tools for updating an SVD-encoded indexing scheme. Master’s thesis, University of Tennessee, Knoxville (1994)

    Google Scholar 

  13. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Sci. 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  14. Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Sci. 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  15. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report CNS-TR-2011-001, California Institute of Technology (2011). http://www.vision.caltech.edu/visipedia/CUB-200-2011.html

  16. Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21(2), 782–791 (1999)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tel-Aviv University, Tel Aviv, Israel

    Tal Halpern & Sivan Toledo

Authors
  1. Tal Halpern
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sivan Toledo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sivan Toledo .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technology, Czestochowa, Poland

    Roman Wyrzykowski

  2. University of Tennessee, Knoxville, Tennessee, USA

    Jack Dongarra

  3. University of Southern California, Marina Del Rey, California, USA

    Ewa Deelman

  4. Czestochowa University of Technology, Czestochowa, Poland

    Konrad Karczewski

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Halpern, T., Toledo, S. (2018). Advances in Incremental PCA Algorithms. In: Wyrzykowski, R., Dongarra, J., Deelman, E., Karczewski, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2017. Lecture Notes in Computer Science(), vol 10777. Springer, Cham. https://doi.org/10.1007/978-3-319-78024-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78024-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78023-8

  • Online ISBN: 978-3-319-78024-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation