# A Global Optimization Algorithm for Sparse Mixed Membership Matrix Factorization

Chapter

First Online:

## Abstract

Mixed membership factorization is a popular approach for analyzing data sets that have within-sample heterogeneity. In recent years, several algorithms have been developed for mixed membership matrix factorization, but they only guarantee estimates from a local optimum. Here, we derive a global optimization algorithm that provides a guaranteed *𝜖*-global optimum for a sparse mixed membership matrix factorization problem. We test the algorithm on simulated data and a small real gene expression dataset and find the algorithm always bounds the global optimum across random initializations and explores multiple modes efficiently.

## Notes

### Acknowledgements

We acknowledge Hachem Saddiki for valuable discussions and comments on the manuscript.

## References

- Airoldi, E.M., Blei, D.M., Fienberg, S.E., Xing, E.P.: Mixed membership stochastic blockmodels. J. Mach. Learn. Res.
**9**, 1981–2014 (2008)zbMATHGoogle Scholar - Benders, J.F.: Partitioning procedures for solving mixed-variables programming problems. Numer. Math.
**4**(1), 238–252 (1962)MathSciNetCrossRefGoogle Scholar - Blei, D.M., Lafferty, J.D.: Correlated topic models. In: Proceedings of the International Conference on Machine Learning, pp 113–120 (2006)Google Scholar
- Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res.
**3**, 993–1022 (2003)zbMATHGoogle Scholar - Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc.
**112**(518), 859–877 (2017)MathSciNetCrossRefGoogle Scholar - Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
- Dheeru, D., Karra T.E.: UCI machine learning repository. URL UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
- Floudas, C.A.: Deterministic Global Optimization, Nonconvex Optimization and Its Applications, vol 37. Springer, Boston (2000)Google Scholar
- Floudas, C.A.: Deterministic Global Optimization: Theory, Methods and Applications, vol. 37. Springer, Berlin (2013)Google Scholar
- Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim.
**45**, 3–38 (2008)MathSciNetCrossRefGoogle Scholar - Floudas, C.A., Visweswaran, V.: A global optimization algorithm (GOP) for certain classes of nonconvex NLPS. Comput. Chem. Eng.
**14**(12), 1–34 (1990)CrossRefGoogle Scholar - Geoffrion, A.M.: Generalized benders decomposition. J. Optim. Theory Appl.
**10**, 237–260 (1972)MathSciNetCrossRefGoogle Scholar - Gorski, J., Pfeuffer, F., Klamroth, K.: Biconvex sets and optimization with biconvex functions: a survey and extensions. Math. Methods Oper. Res.
**66**, 373–407 (2007)MathSciNetCrossRefGoogle Scholar - Gurobi Optimization, Inc (2018) Gurobi optimizer version 8.0Google Scholar
- Horst, R., Tuy, H.: Global Optimization: Deterministic Approaches. Springer, Berlin (2013)zbMATHGoogle Scholar
- Kabán, A.: On Bayesian classification with laplace priors. Pattern Recognit. Lett.
**28**(10), 1271–1282 (2007)CrossRefGoogle Scholar - Lancaster, P., Tismenetsky, M., et al.: The theory of matrices: with applications. Elsevier, San Diego (1985)zbMATHGoogle Scholar
- Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature
**401**, 788–791 (1999)CrossRefGoogle Scholar - MacKay, D.J.C.: Bayesian interpolation. Neural Comput.
**4**(3), 415–447 (1992)CrossRefGoogle Scholar - Mackey, L., Weiss, D., Jordan, M.I.: Mixed membership matrix factorization. In: International Conference on Machine Learning, pp 1–8 (2010)Google Scholar
- Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics
**155**, 945–959 (2000)Google Scholar - Saddiki, H., McAuliffe, J., Flaherty, P.: GLAD: a mixed-membership model for heterogeneous tumor subtype classification. Bioinformatics
**31**(2), 225–232 (2015)CrossRefGoogle Scholar - Singh, A.P., Gordon, G.J.: A unified view of matrix factorization models. In: Lecture Notes in Computer Science, vol. 5212, pp. 358–373, Springer, Berlin (2008)Google Scholar
- Taddy, M.: Multinomial inverse regression for text analysis. J. Am. Stat. Assoc.
**108**(503), 755–770, (2013). https://doi.org/10.1080/01621459.2012.734168 MathSciNetCrossRefGoogle Scholar - Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Sharing clusters among related groups: hierarchical Dirichlet processes. In: Advances in Neural Information Processing Systems, vol. 1, MIT Press, Cambridge (2005)Google Scholar
- Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., Network CGAR, et al.: The cancer genome atlas pan-cancer analysis project. Nat. Genet.
**45**(10), 1113 (2013)CrossRefGoogle Scholar - Xiao, H., Stibor, T.: Efficient collapsed Gibbs sampling for latent Dirichlet allocation. In: Sugiyama, M., Yang, Q. (eds.) Proceedings of 2nd Asian Conference on Machine Learning, vol. 13, pp. 63–78 (2010)Google Scholar
- Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval–SIGIR ’03, p. 267 (2003)Google Scholar
- Zaslavsky, T.: Facing Up to Arrangements: Face-Count Formulas for Partitions of Space by Hyperplanes: Face-Count Formulas for Partitions of Space by Hyperplanes, vol. 154. American Mathematical Society (1975)Google Scholar

## Copyright information

© Springer Nature Switzerland AG 2019