From Logic Programming to Machine Ethics

Living reference work entry
Part of the Springer Reference Geisteswissenschaften book series (SPREFGEIST)


This chapter investigates the appropriateness of Logic Programming-based reasoning to machine ethics, an interdisciplinary field of inquiry that emerges from the need of imbuing autonomous agents with the capacity for moral decision making. The first part of the chapter aims at identifying morality viewpoints, as studied in moral philosophy and psychology, which are amenable to computational modeling, and then mapping them to appropriate Logic Programming-based reasoning features. The identified viewpoints are covered by two morality themes: moral permissibility and the dual-process model. In the second part, various Logic Programming-based reasoning features are applied to model these identified morality viewpoints, via classic moral examples taken off-the-shelf from the literature. For this purpose, our qualm system mainly employs a combination of the Logic Programming features of abduction, updating, and counterfactuals. These features are all supported jointly by Logic Programming tabling mechanisms. The applications are also supported by other existing Logic Programming-based systems, featuring preference handling and probabilistic reasoning, which complement qualm in addressing the morality viewpoints in question.

Throughout the chapter, many references to our published work are given, providing further examples and details about each topic. Thus, this chapter can be envisaged as an entry point survey on the employment of Logic Programming for knowledge modelling and technically implementing machine ethics.


Logic Programming Machine Ethics Abduction Updating Counterfactuals 



LMP acknowledges support from FCT/MEC NOVA LINCS PEst UID/CEC/04516/2013.


  1. Alferes, José Júlio, Antonio Brogi, João Alexandre Leite, and Luís Moniz Pereira. 2002. Evolving logic programs. In JELIA 2002, volume 2424 of LNCS, 50–61. Springer, Berlin, Heidelberg. Springer.Google Scholar
  2. Alferes, José Júlio, Luís Moniz Pereira, and Terrance Swift. 2004. Abduction in well-founded semantics and generalized stable models via tabled dual programs. Theory and Practice of Logic Programming 4(4): 383–428.Google Scholar
  3. Anderson, Michael, and Susan Leigh Anderson. 2008. EthEl: Toward a principled ethical eldercare robot. In Proceedings of AAAI Fall 2008. Symposium on AI in Eldercare.Google Scholar
  4. Anderson, Michael, and Susan Leigh Anderson, eds. 2011. Machine Ethics. Cambridge University Press, New York.Google Scholar
  5. Anderson, Michael, Susan Leigh Anderson, and Chris Armen. 2005. AAAI fall symposium on machine ethics.
  6. Boissier, Olivier, Gregory Bonnet, and Catherine Tessier. 2012. 1st workshop on rights and duties of autonomous agents (RDA2).
  7. Bringsjord, Selmer, Konstantine Arkoudas, and Paul Bello. 2006. Toward a general logicist methodology for engineering ethically correct robots. IEEE Intelligent Systems 21(4): 38–44.Google Scholar
  8. Cushman, Fiery, Liane Young, and Joshua D. Greene. 2010. Multi-system moral psychology. In The moral psychology handbook, ed. John M. Doris. Oxford University Press, Oxford.Google Scholar
  9. Dell’Acqua, Pierangelo, and Luís Moniz Pereira. 2007. Preferential theory revision. Journal of Applied Logic 5(4): 586–601.Google Scholar
  10. Epstude, Kai, and Neal J. Roese. 2008. The functional theory of counterfactual thinking. Personality and Social Psychology Review 12(2): 168–192.Google Scholar
  11. Foot, Phillipa. 1967. The problem of abortion and the doctrine of double effect. Oxford Review 5: 5–15.Google Scholar
  12. Ganascia, Jean-Gabriel. 2007. Modelling ethical rules of lying with answer set programming. Ethics and Information Technology, 9(1): 39–47.Google Scholar
  13. Gelfond, Michael, and Vladimir Lifschitz. 1988. The stable model semantics for logic programming. In 5th international logic programming conference. El Paso, Texas, USA MIT Press.Google Scholar
  14. Han, The Anh, Carroline D. Kencana Ramli, and Carlos Viegas Dam’asio. 2008. An implementation of extended P-log using XASP. In Proceedings of 24th international conference on logic programming (ICLP’08), volume 5366 of LNCS. Udine, Italy. Springer.Google Scholar
  15. Han, The Anh, Ari Saptawijaya, and Luís Moniz Pereira. 2012. Moral reasoning under uncertainty. In LPAR-18, volume 7180 of LNCS, 212–227. Springer, Berlin, Heidelberg.Google Scholar
  16. Hauser, Marc, Fiery Cushman, Liane Young, R. Kang-Xing Jin, and John Mikhail. 2007. A dissociation between moral judgments and justifications. Mind and Language, 22(1): 1–21.Google Scholar
  17. Higgins, Chris. 2014. US Navy funds morality lessons for robots.
  18. Kakas, Antonis, Robert Kowalski, and Francesca Toni. 1993. Abductive logic programming. Journal of Logic and Computation 2(6): 719–770.Google Scholar
  19. Kamm, Frances Myrna. 2006. Intricate ethics: Rights, responsibilities, and permissible harm. Oxford University Press, New York.Google Scholar
  20. Kowalski, Robert. 2011. Computational logic and human thinking: How to be artificially intelligent. Cambridge University Press, Cambridge.Google Scholar
  21. Lopes, Gonçalo, and Luís Moniz Pereira. 2006. Prospective programming with ACORDA. In ESCoR 2006 workshop, IJCAR’06. Springer, Berlin, Heidelberg.Google Scholar
  22. Lopes, Gonçalo, and Luís Moniz Pereira. 2010a. Prospective storytelling agents. In PADL 2010, volume 5937 of LNCS. Springer, Berlin, Heidelberg.Google Scholar
  23. Lopes, Gonçalo, and Luís Moniz Pereira. 2010b. Visual demo of “Princess-saviour Robot”. Available from
  24. Mallon, Ron, and Shaun Nichols. 2010. Rules. In The moral psychology handbook, ed. John M. Doris. Oxford University Press, Oxford.Google Scholar
  25. Markman, Keith D., Igor Gavanski, Steven J. Sherman, and Matthew N. McMullen. 1993. The mental simulation of better and worse possible worlds. Journal of Experimental Social Psychology 29: 87–109.Google Scholar
  26. McCloy, Rachel, and Ruth M. J. Byrne. 2000. Counterfactual thinking about controllable events. Memory and Cognition, 28: 1071–1078.Google Scholar
  27. McIntyre, Alison. 2004. Doctrine of double effect. In The stanford encyclopedia of philosophy, ed. Edward N. Zalta. Center for the Study of Language and Information, Stanford University, Fall 2011 edn.
  28. Migliore, Simone, Giuseppe Curcio, Francesco Mancini, and Stefano F. Cappa. 2014. Counterfactual thinking in moral judgment: An experimental study. Frontiers in Psychology 5: 451.Google Scholar
  29. Newman, Jon O. 2006. Quantifying the standard of proof beyond a reasonable doubt: A comment on three comments. Law, Probability and Risk 5(3–4): 267–269.Google Scholar
  30. Otsuka, Michael. 2008. Double effect, triple effect and the trolley problem: Squaring the circle in looping cases. Utilitas 20(1): 92–110.CrossRefGoogle Scholar
  31. Pereira, Luís Moniz, and The Anh Han. 2009. Evolution prospection. In Proceedings of KES international conference on intelligence decision technologies, Himeji, Japan. vol. 199, 139–150.Google Scholar
  32. Pereira, Luís Moniz, and Ari Saptawijaya. 2011. Modelling Morality with Prospective Logic. In Machine ethics, eds. Michael Anderson and Susan Leigh Anderson, 398–421. Cambridge University Press, New York.Google Scholar
  33. Pereira, Luís Moniz, and Ari Saptawijaya. 2015. Bridging two realms of machine ethics. In Rethinking machine ethics in the age of ubiquitous technology, eds. Jeffrey B. White and Rick Searle. IGI Global, Hershey PA, USA.Google Scholar
  34. Pereira, Luís Moniz, and Ari Saptawijaya. 2016. Programming Machine Ethics. Springer, Berlin, Heidelberg.Google Scholar
  35. Pereira, Luís Moniz, and Ari Saptawijaya. 2017. Counterfactuals, logic programming and agent morality. In Applied Formal Philosophy: Some Reflections on the Program, volume 14 of Logic, Argumentation & Reasoning (Interdisciplinary Perspectives from the Humanities and Social Sciences), eds. Gillman Payette and Rafal Urbaniak. Springer, Cham. Available from
  36. Pereira, Luís Moniz, Pierangelo Dell’Acqua, Alexandre Miguel Pinto, and Gonçalo Lopes. 2013. Inspecting and preferring abductive models. In The handbook on reasoning-based intelligent systems, eds. Kazumi Nakamatsu and Lakhmi C. Jain, 243–274. World Scientific Publishers, Singapore.Google Scholar
  37. Powers, Thomas M. 2006. Prospects for a Kantian machine. IEEE Intelligent Systems 21(4): 46–51.CrossRefGoogle Scholar
  38. Russell, Stuart, Daniel Dewey, and Max Tegmark. 2015. Research priorities for robust and beneficial artificial intelligence. AI Magazine 36(4): 105.Google Scholar
  39. Saptawijaya, Ari, and Luís Moniz Pereira. 2013a. Incremental tabling for query-driven propagation of logic program updates. In LPAR-19, volume 8312 of LNCS, 694–709. Springer.Google Scholar
  40. Saptawijaya, Ari, and Luís Moniz Pereira. 2013b. Tabled abduction in logic programs (Technical communication of ICLP 2013). Theory and Practice of Logic Programming, Online Supplement 13(4–5).
  41. Saptawijaya, Ari, and Luís Moniz Pereira. 2014. Joint tabling of logic program abductions and updates (Technical communication of ICLP 2014). Theory and Practice of Logic Programming, Online Supplement, 14(4–5). Available from
  42. Saptawijaya, Ari, and Luís Moniz Pereira. 2015. TABDUAL: A tabled abduction system for logic programs. IfCoLog Journal of Logics and their Applications 2(1): 69.Google Scholar
  43. Scanlon, Thomas M. 1998. What we owe to each other. Harvard University Press, Cambridge.Google Scholar
  44. Stanovich, Keith E. 2011. Rationality and the reflective mind. Oxford University Press, Oxford.Google Scholar
  45. Swift, Terrance. 1999. Tabling for non-monotonic programming. Annals of Mathematics and Artificial Intelligence 25(3–4): 201–240.CrossRefGoogle Scholar
  46. The Economist. 2012. Morals and the machine. Main front cover and leaders (page 13), June 2nd–8th 2012.Google Scholar
  47. The Future of Life Institute. 2015. International grant competition for robust and beneficial AI.
  48. Thomson, Judith Jarvis. 1985. The trolley problem. The Yale Law Journal 279: 1395–1415.CrossRefGoogle Scholar
  49. van Gelder, Allen, Kenneth A. Ross, and John S. Schlipf. 1991. The well-founded semantics for general logic programs. Journal of the ACM 38(3): 620–650.Google Scholar
  50. Wallach, Wendell, and Colin Allen. 2009. Moral machines: Teaching robots right from wrong. Oxford University Press, Oxford.Google Scholar
  51. White, Jeffrey B., and Rick Searle, eds. 2015. Rethinking machine ethics in the age of ubiquitous technology. Hershey: IGI Global.Google Scholar
  52. Wiegel, Vincent. 2007. SophoLab: Experimental computational philosophy. PhD thesis, Delft University of Technology.Google Scholar

Authors and Affiliations

  1. 1.Faculty of Computer ScienceUniversitas IndonesiaDepokIndonesia
  2. 2.NOVA Laboratory for Computer Science and Informatics (NOVA LINCS), Departamento de Informática, Faculdade de Ciências e TecnologiaUniversidade Nova de LisboaCaparicaPortugal

Personalised recommendations