Abstract
We demonstrate the application of Widening to learning performant Bayesian Networks for use as classifiers. Widening is a framework for utilizing parallel resources and diversity to find models in a hypothesis space that are potentially better than those of a standard greedy algorithm. This work demonstrates that widened learning of Bayesian Networks, using the Frobenius Norm of the networks’ graph Laplacian matrices as a distance measure, can create Bayesian networks that are better classifiers than those generated by popular Bayesian Network algorithms.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We freely mix the use of “solution space” and “hypothesis space” throughout this paper, referring essentially to the same space, but drawing attention to whether it is the evaluation of the hypothesis or the hypothesis itself that is important.
- 2.
In this application, it would be correctly termed “l-dispersion-min-sum,” but the notation is written here as “p” to be consistent with the literature.
- 3.
- 4.
- 5.
References
Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974)
Akbar, Z., Ivanova, V.N., Berthold, M.R.: Parallel data mining revisited. better, not faster. In: Hollmén, J., Klawonn, F., Tucker, A. (eds.) IDA 2012. LNCS, vol. 7619, pp. 23–34. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34156-4_4
Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kötter, T., Ohl, P., Sieb, C., Thiel, K., Wiswedel, B.: KNIME: the konstanz information miner. In: Preisach, C., Burkhardt, H., Schmidt-Thieme, L., Decker, R. (eds.) Data Analysis, Machine Learning and Applications. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 319–326. Springer, Heidelberg (2007)
Bielza, C., Larranaga, P.: Discrete Bayesian network classifiers: a survey. ACM Comput. Surv. (CSUR) 47(1), 5 (2014)
Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 52–60. Morgan Kaufmann Publishers Inc., Los Angeles (1991)
Carvalho, A.M.: Scoring functions for learning Bayesian networks. Technical report INESC-ID Technical report 54/2009, Instituto superior TĂ©chnico, Technical University of Lisboa, April 2009
Cheng, J., Bell, D.A., Liu, W.: An algorithm for Bayesian belief network construction from data. In: Proceedings of AI & STAT 1997, pp. 83–90 (1997)
Chickering, D.M.: Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, 507–554 (2002)
Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)
De Campos, C.P., Zeng, Z., Ji, Q.: Structure learning of Bayesian networks using constraints. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 113–120. ACM (2009)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, New York (1973)
Erkut, E.: The discrete p-dispersion problem. Eur. J. Oper. Res. 46(1), 48–60 (1990)
Fillbrunn, A., Berthold, M.R.: Diversity-driven widening of hierarchical agglomerative clustering. In: Fromont, E., Bie, T., Leeuwen, M. (eds.) IDA 2015. LNCS, vol. 9385, pp. 84–94. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24465-5_8
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)
Golub, G.H., van Loan, C.F.: Matrix Computations, 4th edn. The Johns Hobpkins University Press, Baltimore (2013)
Hamming, R.W.: Error detecting and error correcting codes. Bell Syst. Tech. J. 29(2), 147–160 (1950)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)
Ivanova, V.N., Berthold, M.R.: Diversity-driven widening. In: Tucker, A., Höppner, F., Siebes, A., Swift, S. (eds.) IDA 2013. LNCS, vol. 8207, pp. 223–236. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41398-8_20
Koski, T.J., Noble, J.M.: A review of Bayesian networks and structure learning. Math. Applicanda 40(1), 53–103 (2012)
Larrañaga, P., Karshenas, H., Bielza, C., Santana, R.: A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf. Sci. 233, 109–125 (2013)
Lichman, M.: UCI Machine Learning Repository (2013)
Lowerre, B.T.: The HARPY speech recognition system. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, USA (1976)
Maron, M.E., Kuhns, J.L.: On relevance, probabilistic indexing and information retrieval. J. ACM (JACM) 7(3), 216–244 (1960)
Meinl, T.: Maximum-score diversity selection. Ph.D. thesis, University of Konstanz, July 2010
Nielsen, J.D., Kočka, T., Peña, J.M.: On local optima in learning Bayesian networks. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, pp. 435–442. Morgan Kaufmann Publishers Inc., San Francisco (2003)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)
Pernkopf, F.: Bayesian network classifiers versus k-NN classifier using sequential feature selection. In: AAAI, pp. 360–365 (2004)
Robinson, R.W.: Counting unlabeled acyclic digraphs. In: Little, C.H.C. (ed.) Combinatorial Mathematics V. LNM, vol. 622, pp. 28–43. Springer, Heidelberg (1977)
Sampson, O., Berthold, M.R.: Widened KRIMP: better performance through diverse parallelism. In: Blockeel, H., Leeuwen, M., Vinciotti, V. (eds.) IDA 2014. LNCS, vol. 8819, pp. 276–285. Springer, Heidelberg (2014). doi:10.1007/978-3-319-12571-8_24
Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)
Sierra, B., Larrañaga, P.: Predicting the survival in malignant skin melanoma using Bayesian networks. an empirical comparison between different approaches. Artif. Intell. Med. 14(1–2), 215–230 (1998)
Sprites, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, Cambridge (1993)
Jiang, S., Zhang, H.: Full Bayesian network classifiers. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 897–904. ACM (2006)
Suzuki, J.: A construction of Bayesian networks from databases based on an MDL principle. In: Proceedings of the Ninth International Conference on Uncertainty in Artificial Intelligence, pp. 266–273. Morgan Kaufmann Publishers Inc., San Francisco (1993)
Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Sampson, O.R., Berthold, M.R. (2016). Widened Learning of Bayesian Network Classifiers. In: Boström, H., Knobbe, A., Soares, C., Papapetrou, P. (eds) Advances in Intelligent Data Analysis XV. IDA 2016. Lecture Notes in Computer Science(), vol 9897. Springer, Cham. https://doi.org/10.1007/978-3-319-46349-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-46349-0_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46348-3
Online ISBN: 978-3-319-46349-0
eBook Packages: Computer ScienceComputer Science (R0)