Abstract
In this paper we present a novel method to perform Incremental Singular Value Decomposition (ISVD) by using an adaptation of the power method for diagonalization of matrices. We find that the efficiency of the procedure depends on the initial values and present two alternative ways for initialization. Gram-Schmidt orthonormalization is employed to ensure the orthogonality of the generated singular vectors. The suggested procedure does not depend on any assumption regarding the nature of input matrix or the change (increment) in the input matrix. Moreover, the results do not deviate from exact values even after repeated increments to the original input matrix. In order to test the suggested technique we apply it to the task of Latent Semantic Indexing for query processing using the MEDLINE and TIME corpus. Seven hundred documents are used to build the initial matrix for MEDLINE and two hundred for TIME corpus. We then add one document at a time till the complete corpus is included. Our ISVD technique is applied each time a new document is added. The results obtained using both the methods for initialization are very similar to the exact results obtained using direct diagonalization routines. This implies that the proposed method remains stable even after hundreds of increments in the original matrix.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
\(UU^T = VV^T = I\), where I is an Identity Matrix.
- 2.
Right singular vectors of \(\hat{A}\) are same as the eigenvectors of \(\hat{M}\).
- 3.
- 4.
References
Baker, C.G., Gallivan, K.A., Dooren, P.V.: Low-rank incremental methods for computing dominant singular subspaces. Linear Algebra Appl. 436, 2866–2888 (2012)
Balsubramani, A., Dasgupta, S., Freund, Y.: The fast convergence of incremental PCA. In: Advances in Neural Information Processing Systems (2013)
Brand, M.: Incremental singular value decomposition of uncertain data with missing values. In: Proceedings of the 7th European Conference on Computer Vision-Part I, London, UK, pp. 707–720 (2002)
Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415, 20–30 (2006)
Bhushan, A., Sharker, M.H., Karimi, H.A.: Incremental principal component analysis based outlier detection methods for spatiotemporal data streams. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2(4), 67 (2015)
Chandrasekaran, S., Manjunath, B.S., Wang, Y.F., Winkeler, J., Zhang, H.: An Eigenspace update algorithm for image analysis. In: CVGIP: Graphical Model and Image Processing, vol. 59, pp. 321–332 (1997)
Davies, P.I., Smith, M.I.: Updating the singular value decomposition. J. Comput. Appl. Math. 170(1), 145–167 (2004)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (MD) (1996)
Hall, P.M., David Marshall, A., Martin, R.R.: Incremental eigenanalysis for classification. In: BMVC, vol. 98 (1998)
Hao, S., Xu, Y., Ke, D., Su, K., Peng, H.: SCESS: a WFSA-based automated simplified chinese essay scoring system with incremental latent semantic analysis. Nat. Lang. Eng. 22, 291–319 (2016)
Jiang, H.Y., Nguyen, T.N., Chen, I.X., Jaygarl, H., Chang, C.K.: Incremental latent semantic indexing for automatic traceability link evolution management. In: 23rd IEEE/ACM International Conference on Automated Software Engineering, 2008. ASE 2008, pp. 59–68 (2008)
Kaufman, L.: Methods for updating the singular value decomposition. DAIMI Report Series 3 (1974)
King, J.D., Li, Y.: Web based collection selection using singular value decomposition. In: Web Intelligence, pp. 104–110 (2003)
Kwok, J.T., Zhao, H.: Incremental eigen decomposition. Matrix 100(C1), C2 (2003)
Levy, A., Lindenbaum, M.: Sequential Karhunen-Loeve Basis Extraction and Its Application to Images
Oyama, T., et al.: Incremental learning method of simple-PCA. In: International Conference on Knowledge-Based and Intelligent Information and Engineering Systems. Springer, Berlin, Heidelberg (2008)
Ozawa, S., Pang, S., Kasabov, N.: A modified incremental Principal Component Analysis for on-line learning of feature space and classifier. In: Pacific Rim International Conference on Artificial Intelligence. Springer, Berlin, Heidelberg (2004)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C: The Art of Scientific Computing, 2nd edn. Cambridge University Press, New York, NY, USA (1992)
Rao, S., Medeiros, H., Kak, A.: Comparing incremental latent semantic analysis algorithms for efficient retrieval from software libraries for bug localization. SIGSOFT Softw. Eng. Notes. 40, 1–8 (2015)
Rehurek, R.: Subspace tracking for latent semantic analysis. In: Clough, P.D., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Murdock, V. (eds) ECIR, pp. 289–300 (2011)
Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)
Saad, Y.: Numerical methods for large eigenvalue problems. In: SIAM (2011)
Sabes, P.N.: Linear Algebraic Equations, SVD, and the Pseudo-Inverse. http://keck.ucsf.edu/
Sarwar, B., et al.: Incremental singular value decomposition algorithms for highly scalable recommender systems. In: Fifth International Conference on Computer and Information Science (2002)
Stewart, G.W.: An updating algorithm for subspace tracking. IEEE Trans. Signal Process. 40(6), 1535–1541 (1992)
Stewart, M., Van Dooren, P.: Updating a generalized URV decomposition. SIAM J. Matrix Anal. Appl. 22(2), 479–500 (2000)
Yang, J., et al.: A fast incremental algorithm for low rank approximations of matrices and its applications in facial images. J. Univ. Sci. Technol. China 39(9), 970–979 (2009)
Zhang, M., Hao, S., Xu, Y., Ke, D., Peng, H.: Automated essay scoring using incremental latent semantic analysis. J. Softw. 9 (2014)
Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21, 782–791 (1999)
Zhao, H., Yuen, P.C., Kwok, J.T.: A novel incremental principal component analysis and its application for face recognition. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 36(4), 873–886 (2006)
Acknowledgements
The authors gratefully acknowledge the infrastructural support provided by Indian Institute of Information Technology, Allahabad (IIIT-A). One of the authors (SG) also acknowledges the financial support from IIIT-A.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Gupta, S., Sanyal, S. (2019). Incremental Singular Value Decomposition Using Extended Power Method. In: Lee, R. (eds) Computer and Information Science. ICIS 2018. Studies in Computational Intelligence, vol 791. Springer, Cham. https://doi.org/10.1007/978-3-319-98693-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-98693-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98692-0
Online ISBN: 978-3-319-98693-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)