Abstract
Deep Archetypal Analysis (DeepAA) generates latent representations of high-dimensional datasets in terms of intuitively understandable basic entities called archetypes. The proposed method extends linear Archetypal Analysis (AA), an unsupervised method to represent multivariate data points as convex combinations of extremal data points. Unlike the original formulation, Deep AA is generative and capable of handling side information. In addition, our model provides the ability for data-driven representation learning which reduces the dependence on expert knowledge. We empirically demonstrate the applicability of our approach by exploring the chemical space of small organic molecules. In doing so, we employ the archetype constraint to learn two different latent archetype representations for the same dataset, with respect to two chemical properties. This type of supervised exploration marks a distinct starting point and let us steer de novo molecular design.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note that \(i=1..m\) (and not up to n), which reflects that deep neural networks usually require batch-wise training with batch size m.
References
Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. CoRR abs/1612.00410 (2016). http://arxiv.org/abs/1612.00410
Atkins, P., de Paula, J.: Atkins’ Physical Chemistry. OUP, Oxford (2010)
Bauckhage, C., Manshaei, K.: Kernel archetypal analysis for clustering web search frequency time series. In: 22nd International Conference on Pattern Recognition, pp. 1544–1549, August 2014. https://doi.org/10.1109/ICPR.2014.274
Bauckhage, C., Kersting, K., Hoppe, F., Thurau, C.: Archetypal analysis as an autoencoder. In: Workshop New Challenges in Neural Computation 2015, pp. 8–16, October 2015. https://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr_03_2015.pdf
Bauckhage, C., Thurau, C.: Making archetypal analysis practical. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 272–281. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03798-6_28
Cabeza, L.F., et al.: Lithium in thermal energy storage: a state-of-the-art review. Renew. Sustain. Energy Rev. 42, 1106–1112 (2015)
Canhasi, E., Kononenko, I.: Weighted hierarchical archetypal analysis for multi-document summarization. Comput. Speech Lang. 37 (2015). https://doi.org/10.1016/j.csl.2015.11.004
Cutler, A., Breiman, L.: Archetypal analysis. Technometrics 36(4), 338–347 (1994). https://doi.org/10.1080/00401706.1994.10485840. http://digitalassets.lib.berkeley.edu/sdtr/ucb/text/379.pdf
van Dijk, D., Burkhardt, D., Amodio, M., Tong, A., Wolf, G., Krishnaswamy, S.: Finding archetypal spaces for data using neural networks. arXiv preprint arXiv:1901.09078 (2019)
Chan, B.H.P., Mitchell, D., Cram, L.: Archetypal analysis of galaxy spectra. Mon. Not. Roy. Astron. Soc. 338 (2003). https://doi.org/10.1046/j.1365-8711.2003.06099.x
Hart, Y., et al.: Inferring biological tasks using pareto analysis of high-dimensional data. Nat. Methods 12(3), 233 (2015)
Hou, X., Shen, L., Sun, K., Qiu, G.: Deep feature consistent variational autoencoder. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1133–1141. IEEE (2017)
Huggins, P., Pachter, L., Sturmfels, B.: Toward the human genotope. Bull. Math. Biol. 69(8), 2723–2735 (2007). https://doi.org/10.1007/s11538-007-9244-7
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations (ICLR) (2017)
Kaufmann, D., Keller, S., Roth, V.: Copula archetypal analysis. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 117–128. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24947-6_10
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. abs/1412.6980 (2014)
Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 December 2014, Montreal, pp. 3581–3589 (2014)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. CoRR abs/1312.6114 (2013)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV), December 2015
Mørup, M., Hansen, L.K.: Archetypal analysis for machine learning and data mining. Neurocomputing 80, 54–63 (2012)
Parbhoo, S., Wieser, M., Roth, V.: Causal deep information bottleneck. arXiv e-prints arXiv:1807.02326, July 2018
Prabhakaran, S., Raman, S., Vogt, J.E., Roth, V.: Automatic model selection in archetype analysis. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 458–467. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32717-9_46
Ramakrishnan, R., Dral, P.O., Rupp, M., von Lilienfeld, O.A.: Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1 (2014)
Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 1530–1538. PMLR, Lille, 07–09 July 2015
Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models 32(2), 1278–1286 (2014)
Ruddigkeit, L., van Deursen, R., Blum, L.C., Reymond, J.L.: Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. 52(11), 2864–2875 (2012). https://doi.org/10.1021/ci300415d. pMID: 23088335
Seth, S., Eugster, M.J.A.: Probabilistic archetypal analysis. Mach. Learn. 102(1), 85–113 (2016). https://doi.org/10.1007/s10994-015-5498-8
Shoval, O., et al.: Evolutionary trade-offs, pareto optimality, and thegeometry of phenotype space. Science 336(6085), 1157–1160 (2012). https://doi.org/10.1126/science.1217405. http://science.sciencemag.org/content/336/6085/1157
Steinbeck, C., Han, Y.Q., Kuhn, S., Horlacher, O., Luttmann, E., Willighagen, E.: The Chemistry Development Kit (CDK): an open-source Java library for chemo- and bioinformatics. J. Chem. Inf. Comput. Sci. 43(2), 493–500 (2003)
Tinoco, I.: Physical Chemistry: Principles and Applications in Biological Sciences. No. S. 229-313 in Physical Chemistry: Principles and Applications in Biological Sciences. Prentice Hall, Englewood Cliffs (2002)
Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. arXiv preprint arXiv:physics/0004057 (2000)
Wieczorek, A., Wieser, M., Murezzan, D., Roth, V.: Learning sparse latent representations with the deep copula information bottleneck. In: International Conference on Learning Representations (ICLR) (2018)
Wynen, D., Schmid, C., Mairal, J.: Unsupervised learning of artistic styles with archetypal style analysis. In: Advances in Neural Information Processing Systems, pp. 6584–6593 (2018)
Acknowledgements
S. Keller is partially supported by the Swiss National Science Foundation project CR32I2 159682. M. Samarin is supported by the Swiss National Science Foundation grant 407540 167333 as part of the Swiss National Research Programme NRP 75 “Big Data”. M. Wieser is partially supported by the NCCR MARVEL, funded by the Swiss National Science Foundation and SNSF grant 51MRP0158328 (SystemsX.ch).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Keller, S.M., Samarin, M., Wieser, M., Roth, V. (2019). Deep Archetypal Analysis. In: Fink, G., Frintrop, S., Jiang, X. (eds) Pattern Recognition. DAGM GCPR 2019. Lecture Notes in Computer Science(), vol 11824. Springer, Cham. https://doi.org/10.1007/978-3-030-33676-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-33676-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33675-2
Online ISBN: 978-3-030-33676-9
eBook Packages: Computer ScienceComputer Science (R0)