Abstract
The need for feasible inference in Probabilistic Graphical Models (PGMs) has lead to tractable models like Sum-Product Networks (SPNs). Their highly expressive power and their ability to provide exact and tractable inference make them very attractive for several real world applications, from computer vision to NLP. Recently, great attention around SPNs has focused on structure learning, leading to different algorithms being able to learn both the network and its parameters from data. Here, we enhance one of the best structure learner, LearnSPN, aiming to improve both the structural quality of the learned networks and their achieved likelihoods. Our algorithmic variations are able to learn simpler, deeper and more robust networks. These results have been obtained by exploiting some insights in the building process done by LearnSPN, by hybridizing the network adopting tree-structured models as leaves, and by blending bagging estimations into mixture creation. We prove our claims by empirically evaluating the learned SPNs on several benchmark datasets against other competitive SPN and PGM structure learners.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Amer, M.R., Todorovic, S.: Sum-product networks for modeling activities with stochastic structure. In: 2012 IEEE Conference on (CVPR), pp. 1314–1321. IEEE (2012)
Ammar, S., Leray, P., Defourny, B., Wehenkel, L.: Probability density estimation by perturbing and combining tree structured markov networks. In: Sossai, C., Chemello, G. (eds.) ECSQARU 2009. LNCS, vol. 5590, pp. 156–167. Springer, Heidelberg (2009)
Ammar, S., Leray, P., Schnitzler, F., Wehenkel, L.: Sub-quadratic markov tree mixture learning based on randomizations of the Chow-Liu algorithm. In: Proceedings of the 5th European Workshop on Probabilistic Graphical Models, pp. 17–24 (2010)
Bach, F.R., Jordan, M.I.: Thin junction trees. In: Advances in Neural Information Processing Systems 14, pp. 569–576. MIT Press (2001)
Cheng, W., Kok, S., Pham, H.V., Chieu, H.L., Chai, K.M.A.: Language modeling with sum-product networks. In: 15th Annual Conference of the International Speech Communication Association, pp. 2098–2102 (2014)
Choi, M.J., Tan, V.Y.F., Anandkumar, A., Willsky, A.S.: Learning latent tree graphical models. Journal of Machine Learning Research 12, 1771–1812 (2011)
Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory 14(3), 462–467 (1968)
Dennis, A., Ventura, D.: Learning the architecture of sum-product networks using clustering on varibles. In: Advances in Neural Information Processing Systems 25, pp. 2033–2041. Curran Associates, Inc. (2012)
Gens, R., Domingos, P.: Discriminative learning of sum-product networks. In: Advances in Neural Information Processing Systems 25, pp. 3239–3247. Curran Associates, Inc. (2012)
Gens, R., Domingos, P.: Learning the structure of sum-product networks. In: Proceedings of the 30th International Conference on Machine Learning, pp. 873–880. JMLR Workshop and Conference Proceedings (2013)
Haaren, J.V., Davis, J.: Markov network structure learning: A randomized feature generation approach. In: Proceedings of the 26th Conference on Artificial Intelligence. AAAI Press (2012)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer (2009)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)
Lowd, D., Rooshenas, A.: The Libra Toolkit for Probabilistic Models. CoRR abs/1504.00110 (2015)
Lowd, D., Davis, J.: Learning markov network structure with decision trees. In: Proceedings of the 10th IEEE International Conference on Data Mining, pp. 334–343. IEEE Computer Society Press (2010)
Lowd, D., Rooshenas, A.: Learning markov networks with arithmetic circuits. In: Proceedings of the 16th International Conference on Artificial Intelligence and Statistics. JMLR Workshop Proceedings, vol. 31, pp. 406–414 (2013)
Martens, J., Medabalimi, V.: On the expressive efficiency of sum product networks. CoRR abs/1411.7717 (2014)
Meilǎ, M., Jordan, M.I.: Learning with mixtures of trees. Journal of Machine Learning Research 1, 1–48 (2000)
Peharz, R., Geiger, B.C., Pernkopf, F.: Greedy part-wise learning of sum-product networks. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part II. LNCS, vol. 8189, pp. 612–627. Springer, Heidelberg (2013)
Peharz, R., Gens, R., Domingos, P.: Learning selective sum-product networks. In: Workshop on Learning Tractable Probabilistic Models. LTPM (2014)
Peharz, R., Kapeller, G., Mowlaee, P., Pernkopf, F.: Modeling speech with sum-product networks: Application to bandwidth extension. In: International Conference on Acoustics, Speech and Signal Processing, pp. 3699–3703. IEEE (2014)
Peharz, R., Tschiatschek, S., Pernkopf, F., Domingos, P.: On theoretical properties of sum-product networks. The Journal of Machine Learning Research (2015)
Poon, H., Domingos, P.: Sum-product network: a new deep architecture. In: NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning (2011)
Rahman, T., Kothalkar, P., Gogate, V.: Cutset networks: a simple, tractable, and scalable approach for improving the accuracy of chow-liu trees. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part II. LNCS, vol. 8725, pp. 630–645. Springer, Heidelberg (2014)
Ridgeway, G.: Looking for lumps: Boosting and bagging for density estimation. Computational Statistics & Data Analysis 38(4), 379–392 (2002)
Rooshenas, A., Lowd, D.: Learning sum-product networks with direct and indirect variable interactions. In: Proceedings of the 31st International Conference on Machine Learning, pp. 710–718. JMLR Workshop and Conference Proceedings (2014)
Roth, D.: On the hardness of approximate reasoning. Artificial Intelligence 82(1–2), 273–302 (1996)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Vergari, A., Di Mauro, N., Esposito, F. (2015). Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning. In: Appice, A., Rodrigues, P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science(), vol 9285. Springer, Cham. https://doi.org/10.1007/978-3-319-23525-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-23525-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23524-0
Online ISBN: 978-3-319-23525-7
eBook Packages: Computer ScienceComputer Science (R0)