Abstract
This Chapter analyses who (or what legal entity) should be held responsible for behaviours by Autonomous Weapons Systems (AWS) that, were they enacted by a human agent, would qualify as internationally wrongful acts. After illustrating the structural problems which make ascription of responsibility for AWS’ activities particularly difficult, when not impossible, the alternative routes proposed to solve the ensuing responsibility gap will be assessed. The analysis will focus, in the first place, on the international criminal responsibility of the individuals who, in one way or another, are involved in the process of production, deployment and activation of the AWS. The possibility to hold the deploying State accountable for AWS’ wrongdoings will then be gauged. Subsequently, attention will be paid to the responsibility of the corporations manufacturing and/or programming the AWS. It will be observed that these options may solve some responsibility problems more effectively than critics of AWS are ready to admit. At the same time, it will be shown that, unless a no-fault liability regime is adopted, autonomy in weapons systems is bound to magnify the risk that no one may be held to answer for acts which are objectively in contrast with international legal prescriptions. Also, it will be argued that, given the complementary relationship among the various forms of responsibility under international law, proposals aimed at focusing solely on one of these at the expense of others are incapable of leading to satisfying results.
Paper submitted on 23 February 2018. Although the Authors equally share the responsibility for the entire work, just for evaluation purposes, Sects. 2, 3, 4 and 6 should be attributed to Daniele Amoroso, while the remaining sections should be attributed to Benedetta Giordano.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Academic discussion on this topic was kicked off by Sparrow (2007). Since then, this problem has been dealt with in several articles, speeches and reports, of uneven quality indeed, which we will recall—as comprehensively as possible—throughout the analysis.
- 2.
- 3.
The GGE was established after a three-year cycle of informal meetings, with the mandate of exploring possible recommendations on Lethal AWS to be submitted to CCW State parties. See the Final Document of the Fifth Review Conference, 23 December 2016, UN Doc. CCW/CONF.V/10, Decision 1.
- 4.
Report of the 2017 Group of Governmental Experts on Lethal Autonomous Weapons Systems, 20 November 2017, UN Doc. CCW/GGE.1/2017/CRP.1, para. 16(c).
- 5.
- 6.
Amoroso and Tamburrini (2017), pp. 3–4.
- 7.
US Department of Defense, “Autonomy in Weapons Systems” Directive 3000.09 (21 November 2012), p. 2.
- 8.
ICRC (2016), p. 8.
- 9.
Remarkably enough, by adopting a definition of this kind, it is possible to qualify some presently operating weapons systems as autonomous, their limited target baskets notwithstanding, including stationary robotic sentinels, loitering munitions and fire-and-forget systems. For an overview of existing weapons captured by this notion of autonomy, see Amoroso and Tamburrini (2017), pp. 3–4.
- 10.
Matthias (2004).
- 11.
Ibid., p. 182.
- 12.
Sparrow (2007).
- 13.
Ibid., p. 70.
- 14.
For an overview, see Johnson (2014), pp. 709–710.
- 15.
Ibid., p. 714.
- 16.
See, e.g., Reitinger (2015–2016), pp. 116–118.
- 17.
Arkin (2009), p. 108.
- 18.
Tamburrini (2016), pp. 127–128.
- 19.
Ibid., p. 129. The most notable exception in this respect is provided by autonomous anti-materiel defensive systems, like the Israeli Iron Dome, which operates in sufficiently structured areas pre-set by military officers on the basis of IHL-informed judgments. See Amoroso and Tamburrini (2017), p. 14.
- 20.
Thompson (1980), p. 905.
- 21.
Nissenbaum (1996), p. 29.
- 22.
Ibid., pp. 29–32.
- 23.
Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, 9 April 2013, UN Doc. A/HRC/23/47, para. 77.
- 24.
Bovens (1998), pp. 46–47.
- 25.
Ibid., pp. 50–52.
- 26.
See below Sect. 4.
- 27.
- 28.
McFarland and McCormack (2014).
- 29.
See, e.g., Article 25(c) of the Statute of the International Criminal Court (ICC).
- 30.
British Military Court, Trial of Bruno Tesch et al., 1–8 March 1946, The UN War Crimes Commission “Law reports of trials of war criminals”, Vol. I (1947), p. 93.
- 31.
McFarland and McCormack (2014), p. 370.
- 32.
Article 25(a) of the ICC Statute.
- 33.
ICC (2011), pp. 13–43.
- 34.
McFarland and McCormack (2014), pp. 376–378.
- 35.
Ibid., p. 384.
- 36.
van Sliedregt (2012), p. 1174.
- 37.
ICTY, Prosecutor v. Tadić, IT-94-1, Appeals Judgment, 15 July 1999, paras. 185–229. The JCE doctrine has been subsequently endorsed by the International Criminal Tribunal for Rwanda (see below footnote 43), the Special Court for Sierra Leone (Prosecutor v. Kondewa, No. SCSL-03-12-PT, Decision and Order on Defence Preliminary Motion for Defects in the Form of the Indictment, 27 November 2003, para. 9) and the East Timorese Special Panel for Serious Crimes (Prosecutor v. Perreira, No. 34/2003, Judgment, 27 April 2005, pp. 19–20).
- 38.
International Criminal Tribunal for Rwanda, Prosecutor v. Ntakirutimana et al., ICTR-96-17, Appeals Judgment, 13 December 2004, para. 465.
- 39.
See, e.g., Badar (2006), who wittingly dubbed the doctrine as “Just Convict Everyone”.
- 40.
Article 30 of the ICC Statute.
- 41.
Prosecutor v. Galić, IT-98-29-T, Trial Judgment, 5 December 2003, paras. 57–58. The Trial Chamber uses, quite interchangeably, the civil law notion of dolus eventualis and the common law one of “recklessness”, although only the former can be properly characterized as a form of intent.
- 42.
ICRC (1987), p. 994.
- 43.
Gaeta (2016), p. 45.
- 44.
Ohlin (2013).
- 45.
Ibid., p. 113.
- 46.
- 47.
ICRC (2005), p. 558.
- 48.
- 49.
HRW and IHRC (2015), pp. 21–22.
- 50.
US Military Court, Trial of Wilhelm von Leeb et al. (The German High Command Trial), 30 December 1947–28 October 1948, The UN War Crimes Commission “Law reports of trials of war criminals”, Vol. XII, p. 76.
- 51.
See, generally, Ambos (2013), pp. 197–228.
- 52.
Corn (2016), p. 221.
- 53.
See, e.g., HRW and IHRC (2015), p. 13.
- 54.
Gaeta (2016), p. 45.
- 55.
Melzer (2013), p. 40.
- 56.
Conforti (2018), pp. 410–413.
- 57.
For a similar conclusion, see Geiss and Lahmann (2017), pp. 386–387.
- 58.
- 59.
Conforti (2018), pp. 412–413.
- 60.
In this sense, see Geiss and Lahmann (2017), pp. 390–391.
- 61.
ICRC (2005), p. 56.
- 62.
Ronen (2009), pp. 185–186.
- 63.
Ohlin (2013), pp. 115–116.
- 64.
See, e.g., ICRC (2014), pp. 8, 23 and 47.
- 65.
For an overview, see Kaeb (2016).
- 66.
See, above all, Article 21 of the ICC Statute, which rules out the Court’s jurisdiction over “legal persons”, including corporations. It has been carefully demonstrated (Clapham 2000), however, that this limitation was due to reasons other than the their (alleged) lack of legal personality under international criminal law.
- 67.
See, also for further references, Kaeb (2016), p. 396.
- 68.
Ibid., p. 385.
- 69.
Stephens (2002).
- 70.
Lin (2012), p. 8.
- 71.
- 72.
Quite remarkably, the 1977 European Convention on Products Liability in regard to Personal Injury and Death was signed by only 4 States and ratified by none.
- 73.
See, for instance, American Law Institute 1998, Restatement of the Law, Third, Torts: Products Liability, para. 3 (“Circumstantial Evidence Supporting Inference of Product Defect”).
- 74.
See, e.g., ibid., para. 2(b) (“[A product] is defective in design when the foreseeable risks of harm posed by the product could have been reduced or avoided by the adoption of a reasonable alternative design”); Directive 85/374/EEC, Article 7 (“The producer shall not be liable […] if he proves: […] e) that the state of scientific and technical knowledge at the time when he put the product into circulation was not such as to enable the existence of the defect to be discovered”).
- 75.
Beck (2015), p. 475.
- 76.
487 U.S. 500 (1988).
- 77.
976 F.2d 1328 (1992), 1337.
- 78.
- 79.
Kiss and Shelton (2007).
- 80.
Ibid., p. 1148.
- 81.
Crootof (2016), p. 1395.
- 82.
Prosecutor v. Furundžija, IT-95-17/1-T, Trial Judgment, 10 December 1998, para. 146.
- 83.
- 84.
See, for references, Amoroso and Tamburrini (2017), pp. 1–2.
- 85.
On a more optimistic note, it is worth adding that Germany has recently joined the list of countries that will not use AWS. see “German military has no plans to acquire robot weapons”, Reuters (15 February 2018).
- 86.
Article 36, “Killer Robots: UK Government Policy on Fully Autonomous Weapons” (April 2013).
- 87.
United Nations Institute for Disarmament Research, The Weaponization of Increasingly Autonomous Technologies: Considering how Meaningful Human Control Might Move the Discussion Forward, UNIDIR Resources No. 2, 2014.
- 88.
For a first (and admittedly embryonic) attempt to define the MHC requirement on a principled ethical and legal basis, see Amoroso and Tamburrini (2017), pp. 13–14.
References
Ambos, Kai. 2013. Treatise on International Criminal Law. Volume I: Foundations and General Part. Oxford: Oxford University Press.
Amoroso, Daniele, and Guglielmo Tamburrini. 2017. The Ethical and Legal Case Against Autonomy in Weapons Systems. Global Jurist (on-line): 1–20.
Arkin, Ronald. 2009. Governing Lethal Behavior in Autonomous Robots. Boca Raton: CRC Press.
Badar, Mohamed. 2006. “Just Convict Everyone!” – Joint Perpetration: From Tadić to Stakić and Back Again. International Criminal Law Review 6: 293–302.
Beck, Susanne. 2015. The Problem of Ascribing Legal Responsibility in the Case of Robotics. AI & Society 31: 473–481.
Bianchi, Andrea. 2009. State Responsibility and Criminal Liability of Individuals. In The Oxford Companion to International Criminal Justice, ed. Antonio Cassese, 16–24. Oxford: Oxford University Press.
Bovens, Mark. 1998. The Quest for Responsibility. Accountability and Citizenship in Complex Organisations. Cambridge: Cambridge University Press.
Chengeta, Thompson. 2016. Accountability Gap: Autonomous Weapon Systems and Modes of Responsibility in International Law. Denver Journal of International Law and Policy 45: 1–50.
Clapham, Andrew. 2000. The Question of Jurisdiction Under International Criminal Law Over Legal Persons: Lessons from the Rome Conference on the International Criminal Court. In Liability of Multinational Corporations Under International Law, ed. Menno Y. Kamminga and Saman Zia-Zarifi, 139–195. The Hague: Kluwer Law International.
Conforti, Benedetto. 2018. In Diritto internazionale, ed. Massimo Iovane. Napoli: Editoriale Scientifica.
Corn, Geoffrey S. 2016. Autonomous Weapons Systems: Managing the Inevitability of “Taking the Man Out of the Loop”. In Autonomous Weapons Systems: Law, Ethics, Policy, ed. Nehal Bhuta et al., 209–242. Cambridge: Cambridge University Press.
Crootof, Rebecca. 2016. War Torts: Accountability for Autonomous Weapons. University of Pennsylvania Law Review 164: 1347–1402.
Gaeta, Paola. 2016. Autonomous Weapon Systems and the Alleged Responsibility Gap. Speaker’s Summary. In Expert Meeting, Autonomous Weapon Systems Implications of Increasing Autonomy in the Critical Functions of Weapons (Versoix, Switzerland, 15–16 March 2016), ed. ICRC, 44–45. Geneva: ICRC.
Geiss, Robin, and Henning Lahmann. 2017. Autonomous Weapons Systems: A Paradigm Shift for the Law of Armed Conflict? In Research Handbook on Remote Warfare, ed. Jens D. Ohlin, 371–404. Cheltenham: Edward Elgar Press.
Hammond, Daniel M. 2015. Autonomous Weapons and the Problem of State Accountability. Chicago Journal of International Law 15: 652–687.
Human Rights Watch (HRW) and Harvard’s International Human Rights Clinic (IHRC). 2015. Mind the Gap. The Lack of Accountability for Killer Robots.
ICC. 2011. Elements of Crimes.
ICRC. 1987. In Commentary on the Additional Protocols of 8 June 1977 to the Geneva Conventions of 12 August 1949, ed. Yves Sandoz, Christophe Swinarski, and Bruno Zimmermann. Geneva: Martinus Nijhoff.
———. 2005. In Customary International Humanitarian Law, ed. Jean-Marie Henckaerts and Louise Doswald-Beck. Cambridge: Cambridge University Press.
———, ed. 2014. Expert Meeting. Autonomous Weapon Systems. Technical, Military, Legal and Humanitarian Aspects (Geneva, Switzerland, 26 to 28 March 2014). Geneva: ICRC.
———. 2016. Summary Report. In Expert Meeting, Autonomous Weapon Systems Implications of Increasing Autonomy in the Critical Functions of Weapons (Versoix, Switzerland, 15–16 March 2016), ed. ICRC, 7-22. Geneva: ICRC.
Johnson, Deborah G. 2014. Technology with No Human Responsibility? Journal of Business Ethics 127: 707–715.
Kaeb, Caroline. 2016. The Shifting Sands of Corporate Liability Under International Criminal Law. The George Washington International Law Review 49: 351–403.
Kiss, Alex, and Dinah L. Shelton. 2007. Strict Liability in International Environmental Law. In Law of the Sea, Environmental Law and Settlement of Disputes: Liber Amicorum Judge Thomas A. Mensah, ed. Tafsir Malick Ndiaye and Wolfrum Rüdiger. The Hague: Brill.
Krishnan, Armin. 2009. Killer Robots: Legality and Ethicality of Autonomous Weapons. Farnham: Ashgate.
Lin, Patrick. 2012. Introduction to Robot Ethics. In Robot Ethics. The Ethical and Social Implications of Robotics, ed. Patrick Lin, Keith Abney, and George A. Bekey, 3–16. Cambridge, MA: MIT Press.
Margulies, Peter. 2017. Making Autonomous Weapons Accountable: Command Responsibility for Computer-Guided Lethal Force in Armed Conflicts. In Research Handbook on Remote Warfare, ed. Jens D. Ohlin, 405–442. Cheltenham: Edward Elgar Press.
Matthias, Andreas. 2004. The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata. Ethics and Information Technology 6: 175–183.
McFarland, Tim, and Tim McCormack. 2014. Mind the Gap: Can Developers of Autonomous Weapons Systems Be Liable for War Crimes? International Law Studies 90: 361–385.
Melzer, Nils. 2013. Human Rights Implications of the Usage of Drones and Unmanned Robots in Warfare. Bruxelles: EU Directorate-General for External Policies.
NATO Joint Air Power Competence Centre (JAPCC). 2016. Future Unmanned System Technologies. Legal and Ethical Implications of Increasing Automation. Kalkar (Germany): JAPCC.
Nissenbaum, Helen. 1996. Accountability in a Computerized Society. Science and Engineering Ethics 2: 25–42.
Ohlin, Jens D. 2013. Targeting and the Concept of Intent. Michigan Journal of International Law 35: 79–130.
Reitinger, Nathan. 2015–2016. Algorithmic Choice and Superior Responsibility: Closing the Gap Between Liability and Lethal Autonomy by Defining the Line Between Actors and Tools. Gonzaga Law Review 51: 79–119.
Ronen, Yaël. 2009. Avoid or Compensate? Liability for Incidental Injury to Civilians Inflicted During Armed Conflict. Vanderbilt Journal of Transnational Law 42: 181–225.
Schmitt, Michael N. 2013. Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics. Harvard National Security Journal Features 1: 1–37.
Sparrow, Robert. 2007. Killer Robots. Journal of Applied Philosophy 24: 62–77.
Stephens, Beth. 2002. Translating Filártiga: A Comparative and International Law Analysis of Domestic Remedies for International Human Rights Violations. Yale Journal of International Law 27: 1–57.
Tamburrini, Guglielmo. 2016. On Banning Autonomous Weapon Systems: From Deontological to Wide Consequentialist Reasons. In Autonomous Weapons Systems: Law, Ethics, Policy, ed. Nehal Bhuta et al., 122–142. Cambridge: Cambridge University Press.
Thompson, Dennis F. 1980. Moral Responsibility of Public Officials: The Problem of Many Hands. The American Political Science Review 74: 95–106.
van Sliedregt, Elies. 2012. The Curious Case of International Criminal Liability. Journal of International Criminal Justice 10: 1171–1188.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Amoroso, D., Giordano, B. (2019). Who Is to Blame for Autonomous Weapons Systems’ Misdoings?. In: Carpanelli, E., Lazzerini, N. (eds) Use and Misuse of New Technologies. Springer, Cham. https://doi.org/10.1007/978-3-030-05648-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-05648-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05647-6
Online ISBN: 978-3-030-05648-3
eBook Packages: Law and CriminologyLaw and Criminology (R0)