A Global Optimization Algorithm for Sparse Mixed Membership Matrix Factorization

  • Fan Zhang
  • Chuangqi Wang
  • Andrew C. Trapp
  • Patrick FlahertyEmail author
Part of the ICSA Book Series in Statistics book series (ICSABSS)


Mixed membership factorization is a popular approach for analyzing data sets that have within-sample heterogeneity. In recent years, several algorithms have been developed for mixed membership matrix factorization, but they only guarantee estimates from a local optimum. Here, we derive a global optimization algorithm that provides a guaranteed 𝜖-global optimum for a sparse mixed membership matrix factorization problem. We test the algorithm on simulated data and a small real gene expression dataset and find the algorithm always bounds the global optimum across random initializations and explores multiple modes efficiently.



We acknowledge Hachem Saddiki for valuable discussions and comments on the manuscript.


  1. Airoldi, E.M., Blei, D.M., Fienberg, S.E., Xing, E.P.: Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9, 1981–2014 (2008)zbMATHGoogle Scholar
  2. Benders, J.F.: Partitioning procedures for solving mixed-variables programming problems. Numer. Math. 4(1), 238–252 (1962)MathSciNetCrossRefGoogle Scholar
  3. Blei, D.M., Lafferty, J.D.: Correlated topic models. In: Proceedings of the International Conference on Machine Learning, pp 113–120 (2006)Google Scholar
  4. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  5. Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112(518), 859–877 (2017)MathSciNetCrossRefGoogle Scholar
  6. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  7. Dheeru, D., Karra T.E.: UCI machine learning repository. URL UCI machine learning repository (2017).
  8. Floudas, C.A.: Deterministic Global Optimization, Nonconvex Optimization and Its Applications, vol 37. Springer, Boston (2000)Google Scholar
  9. Floudas, C.A.: Deterministic Global Optimization: Theory, Methods and Applications, vol. 37. Springer, Berlin (2013)Google Scholar
  10. Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45, 3–38 (2008)MathSciNetCrossRefGoogle Scholar
  11. Floudas, C.A., Visweswaran, V.: A global optimization algorithm (GOP) for certain classes of nonconvex NLPS. Comput. Chem. Eng. 14(12), 1–34 (1990)CrossRefGoogle Scholar
  12. Geoffrion, A.M.: Generalized benders decomposition. J. Optim. Theory Appl. 10, 237–260 (1972)MathSciNetCrossRefGoogle Scholar
  13. Gorski, J., Pfeuffer, F., Klamroth, K.: Biconvex sets and optimization with biconvex functions: a survey and extensions. Math. Methods Oper. Res. 66, 373–407 (2007)MathSciNetCrossRefGoogle Scholar
  14. Gurobi Optimization, Inc (2018) Gurobi optimizer version 8.0Google Scholar
  15. Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches. Springer, Berlin (2013)zbMATHGoogle Scholar
  16. Kabán, A.: On Bayesian classification with laplace priors. Pattern Recognit. Lett. 28(10), 1271–1282 (2007)CrossRefGoogle Scholar
  17. Lancaster, P., Tismenetsky, M., et al.: The theory of matrices: with applications. Elsevier, San Diego (1985)zbMATHGoogle Scholar
  18. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  19. MacKay, D.J.C.: Bayesian interpolation. Neural Comput. 4(3), 415–447 (1992)CrossRefGoogle Scholar
  20. Mackey, L., Weiss, D., Jordan, M.I.: Mixed membership matrix factorization. In: International Conference on Machine Learning, pp 1–8 (2010)Google Scholar
  21. Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000)Google Scholar
  22. Saddiki, H., McAuliffe, J., Flaherty, P.: GLAD: a mixed-membership model for heterogeneous tumor subtype classification. Bioinformatics 31(2), 225–232 (2015)CrossRefGoogle Scholar
  23. Singh, A.P., Gordon, G.J.: A unified view of matrix factorization models. In: Lecture Notes in Computer Science, vol. 5212, pp. 358–373, Springer, Berlin (2008)Google Scholar
  24. Taddy, M.: Multinomial inverse regression for text analysis. J. Am. Stat. Assoc. 108(503), 755–770, (2013). MathSciNetCrossRefGoogle Scholar
  25. Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Sharing clusters among related groups: hierarchical Dirichlet processes. In: Advances in Neural Information Processing Systems, vol. 1, MIT Press, Cambridge (2005)Google Scholar
  26. Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., Network CGAR, et al.: The cancer genome atlas pan-cancer analysis project. Nat. Genet. 45(10), 1113 (2013)CrossRefGoogle Scholar
  27. Xiao, H., Stibor, T.: Efficient collapsed Gibbs sampling for latent Dirichlet allocation. In: Sugiyama, M., Yang, Q. (eds.) Proceedings of 2nd Asian Conference on Machine Learning, vol. 13, pp. 63–78 (2010)Google Scholar
  28. Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval–SIGIR ’03, p. 267 (2003)Google Scholar
  29. Zaslavsky, T.: Facing Up to Arrangements: Face-Count Formulas for Partitions of Space by Hyperplanes: Face-Count Formulas for Partitions of Space by Hyperplanes, vol. 154. American Mathematical Society (1975)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Fan Zhang
    • 1
    • 2
    • 3
  • Chuangqi Wang
    • 4
  • Andrew C. Trapp
    • 5
  • Patrick Flaherty
    • 6
    Email author
  1. 1.Center for Data Sciences at Brigham and Women’s HospitalBostonUSA
  2. 2.Broad Institute of Massachusetts Institute of Technology and Harvard UniversityBostonUSA
  3. 3.Department of Biomedical InformaticsHarvard Medical SchoolBostonUSA
  4. 4.Department of Biomedical EngineeringWorcester Polytechnic InstituteWorcesterUSA
  5. 5.Robert A. Foisie Business School, Worcester Polytechnic InstituteWorcesterUSA
  6. 6.Department of Mathematics & StatisticsUniversity of Massachusetts AmherstAmherstUSA

Personalised recommendations