Skip to main content
SpringerLink
Log in

Software Foundation Libraries for Intelligent Systems

  • Conference paper
Book cover Neural Nets WIRN VIETRI-98

Part of the book series: Perspectives in Neural Computing ((PERSPECT.NEURAL))

  • 130 Accesses

Abstract

We describe a set of object-oriented software foundation libraries for the design and modeling of complex and intelligent systems. The basic thesis underlying the libraries is that the language required to represent and design all such systems in a scalable way is the language of sparse recursive graphs with the corresponding local message passing propagation algorithms. We provide some formal arguments to support this thesis going from decision theory, to Bayesian statistics, to probabilistic graphical models. The current version of the libraries consist of 6 different modules: ObjectCost, ObjectStat, ObjectNet, ObjectComp, ObjectData, and ObjectGraph. The libraries support most of the standard models used in artificial intelligence and machine learning, such as neural networks, hidden Markov models, and Bayesian networks. The libraries are being used in real data mining applications and products, and are actively being developed and extended.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for Boltzmann machines. Cognitive Science, 9:147–169, 1985.

    Article  Google Scholar 

  2. S. M. Aji and R. J. McEliece. The generalized distributive law. Technical Report, Department of Electrical Engineering, California Institute of Technology, 1997.

    Google Scholar 

  3. B. C. Arnold and S. J. Press. Compatible conditional distributions. Journal of the American Statistical Association, 84:152–156, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  4. P. Baldi and S. Brunak. Bioinformatics: the machine learning approach. MIT Press, Cambridge, MA, 1998. In press.

    Google Scholar 

  5. P. Baldi and Y. Chauvin. Hybrid modeling, HMM/NN architectures, and protein applications. Neural Computation, 8(7): 1541–1565, 1996.

    Article  Google Scholar 

  6. P. Baldi, Y. Chauvin, T. Hunkapillar, and M. McClure. Hidden Markov models of biological primary sequence information. Proc. Natl. Acad. Sci. USA, 91:1059–1063, 1994.

    Article  Google Scholar 

  7. P. Baldi and P. Frasconi. Parsing graphical models. Net-ID/DSI Technical Report, April 1998.

    Google Scholar 

  8. Y. Bengio and P. Frasconi. An input-output HMM architecture. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances in Neural Information Processing Systems, volume 7. Morgan Kaufmann, San Francisco, CA, 1995. (IEEE).

    Google Scholar 

  9. J. O. Berger. Statistical decision theory and Bayesian analysis. Springer-Verlag, New York, 1985.

    MATH  Google Scholar 

  10. J. Besag. Spatial interaction and the statistical analysis of lattice systems. J. Royal Statis. Soc. B, 36:192–225, 1974.

    MathSciNet  MATH  Google Scholar 

  11. W. Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Trans. on Knowledge and Data Eng., 8:195–210, 1996.

    Article  Google Scholar 

  12. E. Charniak. Bayesian networks without tears. AI Mag., 12:50–63, 1991.

    Google Scholar 

  13. G. F. Cooper. The computational complexity of probabilistic inference using Bayesian belief networks. Art. Intell., 42:393–405, 1990.

    Article  MATH  Google Scholar 

  14. A. P. Dawid. Applications of a general propagation algorithm for probabilistic expert systems. Stat. and Comp., 2:25–36, 1992.

    Article  Google Scholar 

  15. P. Dayan, G. E. Hinton, R. M. Neal, and R. S. Zemel. The Helmholtz machine. Neural Computation, 7(5):889–904, 1995.

    Article  Google Scholar 

  16. A. Gelman and T. P. Speed. Characterizing a joint probability distribution by conditionals. Journal of the Royal Statistical Society B, 55(1): 185–188, 1993.

    MathSciNet  MATH  Google Scholar 

  17. S. Geman and D. Geman. Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. and Machine Intell., 6:721–741, 1984.

    Article  MATH  Google Scholar 

  18. Z. Ghahramani. Learning dynamic Bayesian networks. In M. Gori and C. L. Giles, editors, Adaptive Processing of Temporal Information. Lecture Notes in Artifical Intelligence. Springer Verlag, Heidelberg, 1998. In press.

    Google Scholar 

  19. Z. Ghahramani and M. I. Jordan. Factorial hidden Markov models. Machine Learning, 1997.

    Google Scholar 

  20. W. R. Gilks, A. Thomas, and D. J. Spiegelhalter. A language and program for complex Bayesian modelling. The Statistician, 43:69–78, 1994.

    Article  Google Scholar 

  21. John C. Harsanyi. Rational behavior and bargaining equilibrium in games and social situations. Cambridge University Press, Cambridge, UK, 1977.

    Book  MATH  Google Scholar 

  22. D. Heckerman. Bayesian networks for data mining. Data Mining and Knowl. Discov., 1:79–119, 1997.

    Article  Google Scholar 

  23. G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal. The wake-sleep algorithm for unsupervised neural networks. Science, 268(5214):1158–1161, 1995.

    Article  Google Scholar 

  24. V. Isham. An introduction to spatial point processes and Markov random fields. Internat. Statist. Rev., 49:21–43, 1981.

    Article  MathSciNet  MATH  Google Scholar 

  25. T. S. Jaakkola and I. Jordan. Recursive algorithms for approximating probabilities in graphical models. In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9, pages 487–493. MIT Press, Cambridge, MA, 1997.

    Google Scholar 

  26. F. V. Jensen, S. L. Lauritzen, and K. G. Olesen. Bayesian updating in causal probabilistic networks by local computations. Comput. Statist. Quart., 4:269–282, 1990.

    MathSciNet  Google Scholar 

  27. M. I. Jordan, Z. Ghahramani, and L. K. Saul. Hidden Markov decision trees. In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9, pages 501–507. MIT Press, Cambridge, MA, 1997.

    Google Scholar 

  28. S. L. Lauritzen, A. P. Dawid, B. N. Larsen, and H. G. Leimer. Independence properties of directed Markov fields. Networks, 20:491–505, 1990.

    Article  MathSciNet  MATH  Google Scholar 

  29. S. L. Lauritzen and D. J. Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. J. Royal Statis. Soc. B, 50:157–224, 1988.

    MathSciNet  MATH  Google Scholar 

  30. D. J. C. MacKay, R. J. McEliece, and J. F. Cheng. Turbo decoding as an instance of Pearl’s belief propagation algorithm. IEEE J. Sel. Areas Commun., 1997. In press.

    Google Scholar 

  31. J. M. Modestino and J. Zhang. A Markov random field model-based approach to image interpretation. IEEE Trans. Pattern Anal. Machine Intell., 14:606–615, 1992.

    Article  Google Scholar 

  32. R. M. Neal. Probabilistic inference using Markov chain Monte Carlo methods. Technical report. Department of Computer Science, University of Toronto, 1993.

    Google Scholar 

  33. Radford M. Neal. Connectionist learning of belief networks. Artificial Intelligence, 56:71–113, 1992.

    Article  MathSciNet  MATH  Google Scholar 

  34. J. Pearl. Probabilistic reasoning in intelligent systems. Morgan Kaufmann, San Mateo, CA, 1988.

    Google Scholar 

  35. J. W. Pratt, H. Raiffa, and R. Schlaifer. Introduction to statistical decision theory. MIT Press, Cambridge, MA, 1995.

    Google Scholar 

  36. L. K. Saul and M. I. Jordan. Exploiting tractable substructures in intractable networks. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8, pages 486–492. MIT Press, Cambridge, MA, 1996.

    Google Scholar 

  37. R. D. Schachter. Probabilistic inference and influence diagrams. Operation Research, 36:589–604, 1988.

    Article  Google Scholar 

  38. R. D. Schachter, S. K. Anderson, and P. Szolovits. Global conditioning for probabilistic inference in belief networks. In Proceedings of the Uncertainty in AI Conference, pages 514–522, San Francisco, CA, 1994. Morgan Kaufmann.

    Google Scholar 

  39. P. Smyth, D. Heckerman, and M. I. Jordan. Probabilistic independence networks for hidden Markov probability models. Neural Computation, 9:227–267, 1997.

    Article  MATH  Google Scholar 

  40. D. J. Spiegelhalter, A. P. Dawid, S. L. Lauritzen, and R. G. Cowell. Bayesian analysis in expert systems. Stat. Sci., 8:219–283, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  41. F. Spitzer. Markov random fields and Gibbs ensembles. Am. Math. Monthly, 78:142–154, 1971.

    Article  MathSciNet  MATH  Google Scholar 

  42. R. E. Tarjan and M. Yannakakis. Simple linear-time algorithms to test the chordality of graphs, test acyclicity of hypergraphs. and selectively reduce acyclic hypergraphs. SIAM Journal on Computing, 13(3):566–579, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  43. J. von Neumann and O. Morgenstern. Theory of games and economic behavior. Princeton University Press, Princeton, NJ, 1953. third edition (first edition published in 1944).

    MATH  Google Scholar 

  44. J. Whittaker. Graphical models in applied multivariate statistics. John Wiley & Sons, New York, 1990.

    MATH  Google Scholar 

  45. L. Xu. A unified learning scheme: Bayesian-Kullback Ying-Yang machine, 1996.

    Google Scholar 

  46. J. York. Use of the Gibbs sampler in expert systems. Artif. Intell., 56:115–130, 1992.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Net-ID and Netcore, Los Angeles, CA, 90042, USA

    Pierre Baldi

  2. Net-ID and Netcore, San Francisco, CA, 94107, USA

    Yves Chauvin

  3. Net-ID and Netcore, Los Angeles, CA, 90025, USA

    Van Mittal-Henkle

Authors
  1. Pierre Baldi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yves Chauvin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van Mittal-Henkle
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Scienze Fisiche “E.R. Caianiello”, Università di Salerno, 84081, Baronissi (SA), Italy

    Maria Marinaro

  2. Dipartimento di Informatica ed Applicazioni “R.M. Capocelli”, Università di Salerno, 84081, Baronissi (SA), Italy

    Roberto Tagliaferri

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag London Limited

About this paper

Cite this paper

Baldi, P., Chauvin, Y., Mittal-Henkle, V. (1999). Software Foundation Libraries for Intelligent Systems. In: Marinaro, M., Tagliaferri, R. (eds) Neural Nets WIRN VIETRI-98. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0811-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0811-5_3

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-1208-2

  • Online ISBN: 978-1-4471-0811-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation