Skip to main content
SpringerLink
Log in

Mining Bayesian Networks from Direct Marketing Databases with Missing Values

  • Chapter
Intelligent and Evolutionary Systems

Part of the book series: Studies in Computational Intelligence ((SCI,volume 187))

Abstract

Discovering knowledge from huge databases with missing values is a challenging problem in Data Mining. In this paper, a novel hybrid algorithm for learning knowledge represented in Bayesian Networks is discussed. The new algorithm combines an evolutionary algorithm with the Expectation-Maximization (EM) algorithm to overcome the problem of getting stuck in sub-optimal solutions which occurs in most existing learning algorithms. The experimental results on the databases generated from several benchmark network structures illustrate that our system outperforms some state-of-the-art algorithms. We also apply our system to a direct marketing problem, and compare the performance of the discovered Bayesian networks with the response models obtained by other algorithms. In the comparison, the Bayesian networks learned by our system outperform others.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Jensen, F.V.: An Introduction to Bayesian Network. University of College London Press (1996)

    Google Scholar 

  2. Andreassen, S., Woldbye, M., Falck, B., Andersen, S.: MUNIN: A Causal Probabilistic Network for Interpretation of Electromyographic Findings. In: Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp. 366–372 (1987)

    Google Scholar 

  3. Cheeseman, P., Kelly, J., Self, M., Stutz, J., Taylor, W., Freeman, D.: AutoClass: a Bayesian classification system. In: Proceedings of the Fifth International Workshop on Machine Learning, pp. 54–64 (1988)

    Google Scholar 

  4. Heckerman, D., Horvitz, E.: Inferring Informational Goals from Free-Text Queries: A Bayesian Approach. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 230–237 (1998)

    Google Scholar 

  5. Heckerman, D., Wellman, M.P.: Bayesian Networks. Communications of the ACM 38(3), 27–30 (1995)

    Article  Google Scholar 

  6. Cheng, J., Greiner, R., Kelly, J., Bell, D., Liu, W.: Learning Bayesian Networks from Data: An Information-Theory Based Approach. Artificial Intelligence 137, 43–90 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  7. Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, Cambridge (2000)

    Google Scholar 

  8. Cooper, G., Herskovits, E.: A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning 9(4), 309–347 (1992)

    MATH  Google Scholar 

  9. Heckerman, D.: A Tutorial on Learning Bayesian Networks. Tech. Rep. MSR-TR-95-06. Microsoft Research Adv. Technol. Div., Redmond, WA (1995)

    Google Scholar 

  10. Lam, W., Bacchus, F.: Learning Bayesian belief networks: an approach based on the MDL principle. Computational Intelligence 10(4), 269–293 (1994)

    Article  Google Scholar 

  11. Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R., Kuijpers, C.: Structure Learning of Bayesian Network by Genetic Algorithms: A Performance Analysis of Control Parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(9), 912–926 (1996)

    Article  Google Scholar 

  12. Larrañaga, P., Kuijpers, C., Mura, R., Yurramendi, Y.: Learning Bayesian Network Structures by Searching for The Best Ordering with Genetic Algorithms. IEEE Transactions on System, Man and Cybernetics 26(4), 487–493 (1996)

    Article  Google Scholar 

  13. Wong, M.L., Lam, W., Leung, K.S.: Using Evolutionary Programming and Minimum Description Length principle for data mining of Bayesian networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(2), 174–178 (1999)

    Article  Google Scholar 

  14. Wong, M.L., Leung, K.S.: An Efficient Data Mining Method for Learning Bayesian Networks Using an Evolutionary Algorithm-Based Hybrid Approach. IEEE Transactions on Evolutionary Computation 8(4), 378–404 (2004)

    Article  Google Scholar 

  15. Schafer, J.L., Graham, J.W.: Missing data: Our View of the State of the Art. Psychological Methods 7(2), 147–177 (2002)

    Article  Google Scholar 

  16. Ramoni, M., Sebastiani, P.: Efficient Parameter Learning in Bayesian Networks from Incomplete Databases. Tech. Rep. KMI-TR-41 (1997)

    Google Scholar 

  17. Ramoni, M., Sebastiani, P.: The Use of Exogenous Knowledge to Learn Bayesian Networks from Incomplete Databases. Tech. Rep. KMI-TR-44 (1997)

    Google Scholar 

  18. Friedman, N.: Learning Belief Networks in the Presence of Missing Values and Hidden Variables. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 125–133 (1997)

    Google Scholar 

  19. Friedman, N.: The Bayesian Structural EM Algorithm. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 80–89 (1998)

    Google Scholar 

  20. Peña, J.M., Lozano, J.A., Larrañaga, P.: An Improved Bayesian Structural EM Algorithm for Learning Bayesian Networks for Clustering. Pattern Recognition Letters 21, 779–786 (2000)

    Article  Google Scholar 

  21. Peña, J.M., Lozano, J.A., Larrañaga, P.: Learning Recursive Bayesian Multinets for Data Clustering by Means of Constructive Induction. Machine Learning 47, 63–89 (2002)

    Article  MATH  Google Scholar 

  22. Myers, J., Laskey, K., DeJong, K.: Learning Bayesian Networks from Incomplete Data using Evolutionary Algorithms. In: Proceedings of the First Annual Conference on Genetic and Evolutionary Computation Conference, pp. 458–465 (1999)

    Google Scholar 

  23. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo (1998)

    Google Scholar 

  24. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society(B) 39(1), 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  25. Lauritzen, S.: The EM Algorithm for Graphical Association Models with Missing Data. Computational Statistics and Data Analysis 19, 191–201 (1995)

    Article  MATH  Google Scholar 

  26. Huang, C., Darwiche, A.: Inference in Belief Networks: a Procedural Guide. International Journal of Approximate Reasoning 15(3), 225–263 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  27. LibB, http://compbio.cs.huji.ac.il/LibB/

  28. Bayesware Discoverer, http://www.bayesware.com/frontpage.html

  29. Norsys Bayes Net Library, http://www.norsys.com/net_library.htm

  30. Chickering, D.M.: Learning Equivalence Classes of Bayesian Network Structures. Journal of Machine Learning Research 2, 445–498 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  31. Beaumont, G.P., Knowles, J.D.: Statistical Tests: An Introduction with MINITAB Commentary. Prentice-Hall, Englewood Cliffs (1996)

    Google Scholar 

  32. Zahavi, J., Levin, N.: Issues and Problems in Applying Neural Computing to Target Marketing. Journal of Direct Marketing 11(4), 63–75 (1997)

    Article  Google Scholar 

  33. Bhattacharyya, S.: Direct Marketing Response Models using Genetic Algorithms. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 144–148 (1998)

    Google Scholar 

  34. Cabena, P., Hadjinian, P., Stadler, R., Verhees, J., Zanasi, A.: Discovering Data Mining: From Concept to Implementation. Prentice-Hall, Englewood Cliffs (1997)

    Google Scholar 

  35. Petrison, L.A., Blattberg, R.C., Wang, P.: Database Marketing: Past, Present, and Future. Journal of Direct Marketing 11(4), 109–125 (1997)

    Article  Google Scholar 

  36. Bhattacharyya, S.: Evolutionary Algorithms in Data Mining: Multi-Objective Performance Modeling for Direct Marketing. In: Proceedings of the Sixth International Conference on Knowledge Discovery and Data Mining, pp. 465–473 (2000)

    Google Scholar 

  37. Zahavi, J., Levin, N.: Applying Neural Computing to Target Marketing. Journal of Direct Marketing 11(4), 76–93 (1997)

    Article  Google Scholar 

  38. Ling, C.X., Li, C.H.: Data Mining for Direct Marketing: Problems and Solutions. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 73–79 (1998)

    Google Scholar 

  39. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29, 131–163 (1997)

    Article  MATH  Google Scholar 

  40. Rud, O.P.: Data Mining Cookbook: Modeling Data for Marketing, Risk and Customer Relationship Management. Wiley, New York (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing and Decision Sciences, Lingnan University, Tuen Mun, Hong Kong

    Yuan Yuan Guo & Man Leung Wong

Authors
  1. Yuan Yuan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Man Leung Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Waseda University, Kitakyushu, Japan

    Mitsuo Gen

  2. Monash University, Clayton, VIC, Australia

    David Green

  3. Kyoto University, Kyoto, Japan

    Osamu Katai

  4. Seoul National University, Seoul, Korea

    Bob McKay  & Byoung-Tak Zhang  & 

  5. National Defense Academy of Japan, Yokosuka, Japan

    Akira Namatame

  6. UNSW@ADFA, Campbell, NSW, Australia

    Ruhul A. Sarker

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Guo, Y.Y., Wong, M.L. (2009). Mining Bayesian Networks from Direct Marketing Databases with Missing Values. In: Gen, M., et al. Intelligent and Evolutionary Systems. Studies in Computational Intelligence, vol 187. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-95978-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-95978-6_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-95977-9

  • Online ISBN: 978-3-540-95978-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation