Abstract
In this chapter, we have proposed a NRCG-ONMF method which alternatively updates the orthogonal factor U by doing nonlinear search on Stiefel manifold, and updates the nonnegative factor V in a coordinate manner with closed form solutions. The convergence of NRCG-ONMF has been analyzed. Our approach sheds lights on an promising way to efficiently perform ONMF and shows great potential to handle large scale problems. We evaluate the proposed method on clustering tasks. Extensive experiments on both synthetic and real-world data sets demonstrate that the proposed NRCG-ONMF method outperforms other ONMF methods in terms of the effectiveness on preservation of orthogonality, optimization efficiency and clustering performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788–791, 1999.
Wei Emma Zhang, Mingkui Tan, Quan Z. Sheng, and Qingfeng Shi. Efficient Orthogonal Non-negative Matrix Factorization over Stiefel Manifold. In Proc. of the 25th ACM International Conference on Information and Knowledge Management (CIKM 2016), pages 1743–1752, Indianapolis, IN, USA, October 2016.
Pentti Paatero and Unto Tapper. Positive Matrix Factorization: A Non-negative Factor Model with Optimal Utilization of Error Estimates of Data Values. Environmetrics, 5(2):111–126, 1994.
Filippo Pompili, Nicolas Gillis, Pierre-Antoine Absil, and François Glineur. Two algorithms for orthogonal nonnegative matrix factorization with application to clustering. Neurocomputing, 141:15–25, 2014.
Hongchang Gao, Feiping Nie, Tom Weidong Cai, and Heng Huang. Robust Capped Norm Nonnegative Matrix Factorization: Capped Norm NMF. In Proc. of the 24th ACM International on Conference on Information and Knowledge Management (CIKM 2015), pages 871–880, Melbourne, Australia, October 2015.
Yehuda Koren, Robert M. Bell, and Chris Volinsky. Matrix Factorization Techniques for Recommender Systems. IEEE Computer, 42(8):30–37, 2009.
Chris H. Q. Ding, Tao Li, Wei Peng, and Haesun Park. Orthogonal Nonnegative Matrix Tri-factorizations for Clustering. In Proc. of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2006), pages 126–135, Philadelphia, USA, August 2006.
Chris H. Q. Ding and Xiaofeng He. On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering. In Proc. of the 2005 SIAM International Conference on Data Mining (SDM 2005), pages 606–610, Newport Beach, USA, April 2005.
Yu-Xiong Wang and Yu-Jin Zhang. Nonnegative Matrix Factorization: A Comprehensive Review. IEEE Transactions on Knowledge and Data Engineering, 25(6):1336–1353, 2013.
Zhao Li, Xindong Wu, and Hong Peng. Nonnegative Matrix Factorization on Orthogonal Subspace. Pattern Recognition Letters, 31(9):905–911, 2010.
Zhirong Yang and Erkki Oja. Linear and Nonlinear Projective Nonnegative Matrix Factorization. IEEE Transactions on Neural Networks, 21(5):734–749, 2010.
Seungjin Choi. Algorithms for orthogonal nonnegative matrix factorization. In Proc. of the International Joint Conference on Neural Networks (IJCNN 2008), pages 1828–1832, Hong Kong, China, June 2008.
Megasthenis Asteris, Dimitris Papailiopoulos, and Alexandros G. Dimakis. Orthogonal NMF through Subspace Exploration. In Proc. of the 29th Annual Conference on Neural Information Processing Systems (NIPS 2015), pages 343–351, Montreal, Canada, December 2014.
Chris H. Q. Ding, Tao Li, and Michael I. Jordan. Convex and Semi-Nonnegative Matrix Factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1):45–55, 2010.
Pierre-Antoine Absil, Robert E. Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008.
Magnus Rudolph Hestenes and Eduard Stiefel. Methods of Conjugate Gradients for Solving Linear Systems. Journal of the Research of the National Bureau of Standards, 49(6):409–436, 1952.
William W Hager and Hongchao Zhang. A Survey of Nonlinear Conjugate Gradient Methods. Pacific Journal of Optimization, 2(1):35–58, 2006.
Bart Vandereycken. Low-rank matrix completion by riemannian optimization. SIAM Journal on Optimization, 23(2):1214–1236, 2013.
Jorge Nocedal and J Wright Stephen. Numerical Optimization. Springer Series in Operations Research and Financial Engineering, Springer, 2006.
Jonathan Barzilai and Jonathan M Borwein. Two-Point Step Size Gradient Methods. IMA Journal of Numerical Analysis, 8(1):141–148, 1988.
Donald Goldfarb, Zaiwen Wen, and Wotao Yin. A Curvilinear Search Method for p-Harmonic Flows on Spheres. SIAM Journal on Imaging Sciences, 2(1):84–109, 2009.
Hongchao Zhang and William W. Hager. A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization. SIAM Journal on Optimization, 14(4):1043–1056, 2004.
Cho-Jui Hsieh and Inderjit S. Dhillon. Fast coordinate descent methods with variable selection for non-negative matrix factorization. In Proc. of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2011), pages 1064–1072, San Diego, USA, August 2011.
Sameer A Nene, Shree K Nayar, Hiroshi Murase, et al. Columbia object image library (COIL-20). Technical report, Columbia University, 1996.
Terence Sim, Simon Baker, and Maan Bsat. The CMU Pose, Illumination, and Expression Database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12):1615–1618, 2003.
Shi Zhong and Joydeep Ghosh. Generative Model-based Document Clustering: A Comparative Study. Knowledge and Information Systems, 8(3):374–384, 2005.
Martijn van Breukelen, Robert P. W. Duin, David M. J. Tax, and J. E. den Hartog. Handwritten digit recognition by combined classifiers. Kybernetika, 34(4):381–386, 1998.
Wei Xu, Xin Liu, and Yihong Gong. Document Clustering Based on Non-Negative Matrix Factorization. In Proc. of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), pages 267–273, Toronto, Canada, July 2003.
Deguang Kong, Chris H. Q. Ding, and Heng Huang. Robust Nonnegative Matrix Factorization using L21-Norm. In Proc. of the 20th ACM Conference on Information and Knowledge Management (CIKM 2011), pages 673–682, Glasgow, United Kingdom, October 2011.
L. Lovász and M.D. Plummer. Matching Theory. North Holland, Budapest, 1986.
Deng Cai, Xiaofei He, and Jiawei Han. Document Clustering Using Locality Preserving Indexing. IEEE Transactions on Knowledge and Data Engineering, 17(12):1624–1637, 2005.
Reeves Fletcher and Colin M Reeves. Function Minimization by Conjugate Gradients. The Computer Journal, 7(2):149–154, 1964.
Bin Shen and Luo Si. Non-Negative Matrix Factorization Clustering on Multiple Manifolds. In Proc. of the 24th AAAI Conference on Artificial Intelligence (AAAI 2010), Atlanta, USA, July 2010.
Fuming Sun, Meixiang Xu, Xuekao Hu, and Xiaojun Jiang. Graph Regularized and Sparse Nonnegative Matrix Factorization with Hard Constraints for Data Representation. Neurocomputing, 173:233–244, 2016.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Zhang, W.E., Sheng, Q.Z. (2018). An Efficient Knowledge Clustering Algorithm. In: Managing Data From Knowledge Bases: Querying and Extraction. Springer, Cham. https://doi.org/10.1007/978-3-319-94935-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-94935-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94934-5
Online ISBN: 978-3-319-94935-2
eBook Packages: Computer ScienceComputer Science (R0)