Cognitive Algorithms and Systems: Reasoning and Knowledge Representation

  • Artur S. d’Avila GarcezEmail author
  • Luis C. Lamb
Part of the Springer Series in Cognitive and Neural Systems book series (SSCNS)


This chapter reviews recent advances in computational cognitive reasoning and their underlying algorithmic foundations. It summarises the neural-symbolic approach to cognition and computation. Neural-symbolic systems integrate two fundamental phenomena of intelligent behaviour: reasoning and the ability to learn from experience. The chapter illustrates how to represent, learn and compute several expressive forms of symbolic knowledge using neural networks. The goal is to provide computational models with integrated reasoning capabilities, where the neural networks offer the machinery for cognitive reasoning and learning while symbolic logic offers explanations to the neural models facilitating the necessary interaction with the world and other systems.


Modal Logic Temporal Logic Translation Algorithm Symbolic Knowledge Nonclassical Logic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. F. Baader, D. Calvanese, D. L. McGuinness, D. Nardi, and P. F. Patel-Schneider, editors. The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, Cambridge, 2003.Google Scholar
  2. S. Bader, A. d’Avila Garcez, and P. Hitzler. Computing first-order logic programs by fibring artificial neural networks. In Proceedings of the AAAI International FLAIRS Conference, pages 314–319, 2005.Google Scholar
  3. S. Bader, P. Hitzler, S. Holldobler, and A. Witzel. A fully connectionist model generator for covered first-order logic programs. In Proceedings of the International Joint Conference on Artificial Intelligence IJCAI-07, pages 666–671, Hyderabad, India, 2007. AAAI.Google Scholar
  4. B. Bennett. Spatial reasoning with propositional logics. In Proceedings of the Fourth International Conference on Principles of Knowledge Representation and Reasoning KR-94, pages 51–62, 1994.Google Scholar
  5. P. Blackburn, J. van Benthem, and F. Wolter, editors. Handbook of Modal Logic. Studies in Logic and Practical Reasoning. Elsevier, Amsterdam, 2006.Google Scholar
  6. G. Bologna. Is it worth generating rules from neural network ensembles? Journal of Applied Logic, 2(3):325–348, 2004.CrossRefGoogle Scholar
  7. R. V. Borger, A. d’Avila Garcez, and L. Lamb. A neural-symbolic perspective on analogy. Behavioral and Brain Sciences, 31(4):379–380, 2008.Google Scholar
  8. K. Broda, D. Gabbay, L. Lamb, and A. Russo. Labelled natural deduction for conditional logics of normality. Logic Journal of the IGPL, 10(2):123–163, 2002.CrossRefGoogle Scholar
  9. K. Broda, D. Gabbay, L. Lamb, and A. Russo. Compiled Labelled Deductive Systems: A Uniform Presentation of Non-classical Logics. Studies in Logic and Computation. Research Studies Press/Institute of Physics Publishing, Baldock, UK, Philadelphia, PA, 2004.Google Scholar
  10. A. Browne and R. Sun. Connectionist inference models. Neural Networks, 14:1331–1355, 2001.CrossRefPubMedGoogle Scholar
  11. I. Cloete and J. Zurada, editors. Knowledge-Based Neurocomputing. MIT, Cambridge, MA, 2000.Google Scholar
  12. A. d’Avila Garcez. Fewer epistemological challenges for connectionism. In S. B. Cooper, B. Lowe, and L. Torenvliet, editors, Proceedings of Computability in Europe, CiE 2005, volume LNCS 3526, pages 139–149, Amsterdam, The Netherlands, June 2005. Springer, Berlin.Google Scholar
  13. A. d’Avila Garcez and D. Gabbay. Fibring neural networks. In Proceedings of 19th National Conference on Artificial Intelligence (AAAI-04), pages 342–347, San Jose, CA, 2004.Google Scholar
  14. A. d’Avila Garcez and P. Hitzler, editors. Proceedings of IJCAI International Workshop on Neural-Symbolic Learning and Reasoning NeSy09, Pasadena, California, USA, 2009.Google Scholar
  15. A. d’Avila Garcez and L. Lamb. Reasoning about time and knowledge in neural-symbolic learning systems. In S. Thrun, L. Saul, and B. Schoelkopf, editors, Advances in Neural Information Processing Systems 16, Proceedings of  NIPS 2003, pages 921–928. MIT, Cambridge, MA, 2004.Google Scholar
  16. A. d’Avila Garcez and L. Lamb. Neural-symbolic systems and the case for non-classical reasoning. In S. Artëmov, H. Barringer, A. d’Avila Garcez, L. Lamb, and J. Woods, editors, We Will Show Them! Essays in Honour of Dov Gabbay, pages 469–488. College Publications, International Federation for Computational Logic, UK, 2005.Google Scholar
  17. A. d’Avila Garcez and L. Lamb. A connectionist computational model for epistemic and temporal reasoning. Neural Computation, 18(7):1711–1738, 2006.Google Scholar
  18. A. d’Avila Garcez and G. Zaverucha. The connectionist inductive learning and logic programming system. Applied Intelligence Journal, Special Issue on Neural Networks and Structured Knowledge, 11(1):59–77, 1999.Google Scholar
  19. A. d’Avila Garcez, K. Broda, and D. Gabbay. Symbolic knowledge extraction from trained neural networks: A sound approach. Artificial Intelligence, 125:155–207, 2001.Google Scholar
  20. A. d’Avila Garcez, K. Broda, and D. Gabbay. Neural-Symbolic Learning Systems: Foundations and Applications. Perspectives in Neural Computing. Springer, Berlin, 2002a.Google Scholar
  21. A. d’Avila Garcez, L. Lamb, and D. Gabbay. A connectionist inductive learning system for modal logic programming. In Proceedings of the 9th International Conference on Neural Information Processing ICONIP’02, pages 1992–1997, Singapore, 2002b. IEEE.Google Scholar
  22. A. d’Avila Garcez, L. Lamb, K. Broda, and D. Gabbay. Distributed knowledge representation in neural-symbolic learning systems: A case study. In Proceedings of AAAI International FLAIRS Conference, pages 271–275, St. Augustine, FL, 2003a. AAAI.Google Scholar
  23. A. d’Avila Garcez, L. Lamb, and D. Gabbay. Neural-symbolic intuitionistic reasoning. Frontiers in Artificial Intelligence and Applications Vol. 104, pages 399–408. IOS, 2003b.Google Scholar
  24. A. d’Avila Garcez, D. Gabbay, and L. Lamb. Argumentation neural networks. In Proceedings of the 11th International Conference on Neural Information Processing, ICONIP’04, volume 3316 of Lecture Notes in Computer Science, pages 606–612. Springer, New York, 2004a.Google Scholar
  25. A. d’Avila Garcez, D. Gabbay, and L. Lamb. Towards a connectionist argumentation framework. In Proceedings of the 16th European Conference on Artificial Intelligence, ECAI 2004, including Prestigious Applicants of Intelligent Systems, PAIS 2004, Valencia, Spain, August 22–27, 2004, pages 987–988, 2004b.Google Scholar
  26. A. d’Avila Garcez, L. Lamb, K. Broda, and D. Gabbay. Applying connectionist modal logics to distributed knowledge representation problems. International Journal on Artificial Intelligence Tools, 13(1):115–139, 2004c.Google Scholar
  27. A. d’Avila Garcez, D. Gabbay, and L. Lamb. Value-based argumentation frameworks as neural-symbolic learning systems. Journal of Logic and Computation, 15(6):1041–1058, 2005.Google Scholar
  28. A. d’Avila Garcez, L. Lamb, and D. Gabbay. Connectionist computations of intuitionistic reasoning. Theoretical Computer Science, 358(1):34–55, 2006a.Google Scholar
  29. A. d’Avila Garcez, L. Lamb, and D. Gabbay. A connectionist model for constructive modal reasoning. In Advances in Neural Information Processing Systems 18, Proceedings of  NIPS 2005, pages 403–410. MIT, 2006b.Google Scholar
  30. A. d’Avila Garcez, D. M. Gabbay, O. Ray, and J. Woods. Abductive reasoning in neural-symbolic systems. TOPOI: An International Review of Philosophy, 26:37–49, 2007a.Google Scholar
  31. A. d’Avila Garcez, L. Lamb, and D. Gabbay. Connectionist modal logic: Representing modalities in neural networks. Theoretical Computer Science, 371(1–2):34–53, 2007b.Google Scholar
  32. A. d’Avila Garcez, L. Lamb, and D. Gabbay. Neural-Symbolic Cognitive Reasoning. Cognitive Technologies. Springer, Berlin, 2009.Google Scholar
  33. J. Elman. Finding structure in time. Cognitive Science, 14(2):179–211, 1990.CrossRefGoogle Scholar
  34. R. Fagin, J. Halpern, Y. Moses, and M. Vardi. Reasoning About Knowledge. MIT, Cambridge, MA, 1995.Google Scholar
  35. D. Gabbay. Elementary Logics: a Procedural Perspective. Prentice Hall, London, 1998.Google Scholar
  36. D. Gabbay. Fibring Logics. Oxford University Press, Oxford, 1999. Oxford Logic Guides, Vol. 38.Google Scholar
  37. D. M. Gabbay and A. Hunter. Making inconsistency respectable: Part 2 – meta-level handling of inconsistency. In Symbolic and Quantitative Approaches to Reasoning and Uncertainty ECSQARU’93, volume LNCS 747, pages 129–136. Springer, Berlin, 1993.Google Scholar
  38. D. Gabbay and J. Woods. A Practical Logic of Cognitive Systems, Volume 2: The reach of abduction: Insight and trial. Elsevier, New York, 2005.Google Scholar
  39. D. Gabbay, I. Hodkinson, and M. Reynolds. Temporal logic: mathematical foundations and computational aspects, volume 1. Oxford University Press, Oxford, 1994. Oxford Logic Guides, Vol. 28.Google Scholar
  40. D. Gabbay, A. Kurucz, F. Wolter, and M. Zakharyaschev. Many-dimensional Modal Logics: Theory and Applications, volume 148 of Studies in Logic and the Foundations of Mathematics. Elsevier Science, Amsterdam, The Netherlands, 2003.Google Scholar
  41. P. Gärdenfors. Conceptual Spaces: The Geometry of Thought. MIT, Cambridge, MA, 2000.Google Scholar
  42. J. Halpern. Reasoning About Uncertainty. MIT, Cambridge, MA, 2003.Google Scholar
  43. S. Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall, New Jersey, 1999.Google Scholar
  44. G. E. Hinton, S. Osindero, and Y. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527–1554, 2006.CrossRefPubMedGoogle Scholar
  45. P. Hitzler, S. Holldobler, and A. K. Seda. Logic programs and connectionist networks. Journal of Applied Logic, 2(3):245–272, 2004. Special Issue on Neural-Symbolic Systems.Google Scholar
  46. S. Hölldobler. Automated inferencing and connectionist models. Postdoctoral Thesis, Intellektik, Informatik, TH Darmstadt, 1993.Google Scholar
  47. S. Hölldobler and Y. Kalinke. Toward a new massively parallel computational model for logic programming. In Proceedings of the Workshop on Combining Symbolic and Connectionist Processing, ECAI 1994, pages 68–77, 1994.Google Scholar
  48. M. Huth and M. Ryan. Logic in Computer Science: Modelling and Reasoning About Systems. Cambridge University Press, Cambridge, 2000.Google Scholar
  49. H. Jacobsson. Rule extraction from recurrent neural networks: A taxonomy and review. Neural Computation, 17(6):1223–1263, 2005.CrossRefGoogle Scholar
  50. L. Lamb, R. Borges, and A. d’Avila Garcez. A connectionist cognitive model for temporal synchronisation and learning. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence AAAI 2007, pages 827–832. AAAI, 2007.Google Scholar
  51. H. Leitgeb. Neural network models of conditionals: an introduction. In X. Arrazola and J. M. Larrazabal et al., editors, Proceedings of ILCLI International Workshop on Logic and Philosophy of Knowledge, Communication and Action, pages 191–223, Bilbao, 2007.Google Scholar
  52. A. Lozowski and J. Zurada. Extraction of linguistic rules from data via neural networks and fuzzy approximation. In I. Cloete and J. Zurada, editors, Knowledge-Based Neurocomputing, pages 403–417. MIT, Cambridge, 2000.Google Scholar
  53. J. McCarthy. Epistemological challenges for connectionism. Behavioral and Brain Sciences, 11(1):44, 1988.Google Scholar
  54. M. Mendler. Characterising combinatorial timing analysis in intuitionistic modal logic. Logic Journal of the IGPL, 8(6):821–852, 2000.CrossRefGoogle Scholar
  55. R. Mooney and D. Ourston. A multistrategy approach to theory refinement. In R. Michalski and G. Teccuci, editors, Machine Learning: A Multistrategy Approach, volume 4, pages 141–164. Morgan Kaufmann, San Mateo, CA, 1994.Google Scholar
  56. H. Nunez, C. Angulo, and A. Catala. Rule based learning systems for support vector machines. Neural Processing Letters, 24(1):1–18, 2006.CrossRefGoogle Scholar
  57. M. Orgun and W. Ma. An overview of temporal and modal logic programming. In Proceedings of the International Conference on Temporal Logic ICTL’94, volume 827 of Lecture Notes in Artificial Intelligence, pages 445–479. Springer, Berlin, 1994.Google Scholar
  58. M. Page. Connectionist modelling in psychology: A localist manifesto. Behavioral and Brain Sciences, 23:443–467, 2000.CrossRefPubMedGoogle Scholar
  59. S. Pinker. The Stuff of Thought: Language as a Window into Human Nature. Viking, New York, 2007.Google Scholar
  60. S. Pinker, M. A. Nowak, and J. J. Lee. The logic of indirect speech. Proceedings of the National Academy of Sciences USA, 105(3):833–838, 2008.CrossRefGoogle Scholar
  61. A. Pnueli. The temporal logic of programs. In Proceedings of 18th IEEE Annual Symposium on Foundations of Computer Science, pages 46–57, 1977.Google Scholar
  62. A. Rao and M. Georgeff. Decision procedures for BDI logics. Journal of Logic and Computation, 8(3):293–343, 1998.CrossRefGoogle Scholar
  63. D. Rumelhart, G. Hinton, and R. Williams. Learning internal representations by error propagation. In D. Rumelhart and J. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1, pages 318–362. MIT, Cambridge, 1986.Google Scholar
  64. R. Setiono. Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Computation, 9:205–225, 1997.CrossRefPubMedGoogle Scholar
  65. L. Shastri. Advances in SHRUTI: a neurally motivated model of relational knowledge representation and rapid inference using temporal synchrony. Applied Intelligence Journal, Special Issue on Neural Networks and Structured Knowledge, 11:79–108, 1999.Google Scholar
  66. L. Shastri. Shruti: A neurally motivated architecture for rapid, scalable inference. In B. Hammer and P. Hitzler, editors, Perspectives of Neural-Symbolic Integration, pages 183–203. Springer, Heidelberg, 2007.CrossRefGoogle Scholar
  67. L. Shastri and V. Ajjanagadde. From simple associations to semantic reasoning: A connectionist representation of rules, variables and dynamic binding. Technical report, University of Pennsylvania, 1990.Google Scholar
  68. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, 2004.Google Scholar
  69. P. Smolensky. On the proper treatment of connectionism. Behavioral and Brain Sciences, 44:1–74, 1988.CrossRefGoogle Scholar
  70. P. Smolensky and G. Legendre. The Harmonic Mind: From Neural Computation to Optimality-Theoretic Grammar. MIT, Cambridge, MA, 2006.Google Scholar
  71. K. Stenning and M. van Lambalgen. Human reasoning and cognitive science, 2008.Google Scholar
  72. R. Sun. Robust reasoning: integrating rule-based and similarity-based reasoning. Artificial Intelligence, 75(2):241–296, 1995.CrossRefGoogle Scholar
  73. R. Sun. Theoretical status of computational cognitive modeling. Cognitive Systems Research, 10(2):124–140, 2009.CrossRefGoogle Scholar
  74. J. Taylor. Cognitive computation. Cognitive Computation, 1(1):4–16, 2009.CrossRefGoogle Scholar
  75. S. Thrun. Extracting provably correct rules from artificial neural networks. Technical report, Institut für Informatik, Universität Bonn, 1994.Google Scholar
  76. G. Towell and J. Shavlik. Knowledge-based artificial neural networks. Artificial Intelligence, 70(1):119–165, 1994.CrossRefGoogle Scholar
  77. L. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, 1984.CrossRefGoogle Scholar
  78. L. Valiant. A neuroidal architecture for cognitive computation. Journal of the ACM, 47(5):854–882, 2000.CrossRefGoogle Scholar
  79. L. Valiant. Three problems in computer science. Journal of the ACM, 50(1):96–99, 2003.CrossRefGoogle Scholar
  80. L. Valiant. Knowledge infusion: In pursuit of robustness in artificial intelligence. In Proceedings of the 28th Conference on Foundations of Software Technology and Theoretical Computer Science, pages 415–422, Bangalore, India, 2008.Google Scholar
  81. J. van Benthem. Correspondence theory. In D. Gabbay and F. Guenthner, editors, Handbook of Philosophical Logic, chapter II.4, pages 167–247. D. Reidel Publishing Company, Dordrecht, 1984.Google Scholar
  82. D. van Dalen. Intuitionistic logic. In D. Gabbay and F. Guenthner, editors, Handbook of Philosophical Logic, volume 5. Kluwer, Dordrecht, 2nd edition, 2002.Google Scholar
  83. M. Vardi. Why is modal logic so robustly decidable. In N. Immerman and P. Kolaitis, editors, Descriptive Complexity and Finite Models, volume 31 of Discrete Mathematics and Theoretical Computer Science, pages 149–184. DIMACS, 1997.Google Scholar
  84. M. Wooldridge. Introduction to Multi-agent Systems. Wiley, New York, 2001.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of ComputingCity UniversityLondonUK

Personalised recommendations