Make Way for the Robots! Human- and Machine-Centricity in Constituting a European Public–Private Partnership
This article is an analytic register of recent European efforts in the making of ‘autonomous’ robots to address what is imagined as Europe’s societal challenges. The paper describes how an emerging techno-epistemic network stretches across industry, science, policy and law to legitimize and enact a robotics innovation agenda. Roadmap is the main metaphor and organizing tool in working across the disciplines and sectors, and in aligning these heterogeneous actors with a machine-centric vision along a path to make way for ‘new kinds’ of robots. We describe what happens as this industry-dominated project docks in a public–private partnership with pan-European institutions and a legislative initiative on robolaw. Emphasizing the co-production of robotics and European innovation politics, we observe how well-known uncertainties and scholarly debates about machine capabilities and human–machine configurations, are unexpectedly played out in legal scholarship and institutions as a controversy and a significant problem for human-centered legal frameworks. European robotics are indeed driving an increase in speculative ethics and a new-found weight of possible futures in legislative practice.
KeywordsAutonomy Robotics ELSA Public–private partnership Imaginary Rights Innovation Co-production
Introduction: A Networking of Visions, Disciplines and Policy
Robotics development has always been intimately connected with visions of autonomous machines, in particular, any human-like intelligent appearances like those found in mythology, folklore and the science fiction genre (e.g., Hephaestus’ golden assistants; the Golem; Asimov’s robot visions or Disney Pixar’s animations). Arguably, such visions have been integral to ongoing efforts towards a better understanding of life through artificially recreating it (Riskin 2007). Increasingly, policymakers and industrialists are seizing upon the vision of autonomous robots in more pro-active ways. The idea is no longer primarily about a better understanding of life or its fictional representations, but has become a social-technical imaginary to enhance life, welfare, jobs and well-being in society, advance Europe’s competitiveness and open new markets for its industry.
A type of human robot, a Humanoid, is expected to work together with human partners in our living environment, and it will share the same working space and will experience the same thinking and behavior patterns as a human being.1
The autonomous robot imaginary takes on new configurations and roles within the institutional organization of innovation in Europe. Future robots are collectively envisioned to become more capable of learning, flexibility and social interaction, compared with previous generations of (mainly) industrial robots. They will move from the closed-off confines of factories into people’s daily environments: communities, workplaces, homes and public spaces. The expectation is that they will become co-workers and providers of services to humans, including intimate relations of care and companionship.
Public–private conglomerates now coordinate European developments in this area under headings such as the Fourth Industrial Revolution and elaborate policy agendas for industry and innovation. European Technology Platforms (ETP), Joint Technology Initiatives (JTI) and Public–Private Partnerships (PPP) have been key instruments in shaping long term strategies and implementing so-called strategic research agendas (EC 2005). These are industry-led programs and partnerships jointly supported by public and private funds and the platforms have been particularly instrumental in strategic visioning and defining the research agendas (e.g., EUROP 2009; euRobotics 2013). As argued by Fuchs (2018), such initiatives now form part of major efforts to re-make European societies in the image of socially intelligent, learning and interacting machines. The prime mover in these developments is industry as demonstrated by its power to influence research and innovation policies oriented to industrial production. But industry cannot act alone in shaping innovative visions of the future of robotics, and becomes increasingly reliant on mediators to help make and stabilize new relations and environments. Funding needs to be brought in to support ailing companies and law, regulation and public institutions become important in supporting market-making and stabilizing actors’ expectations.
This paper explores prominent features of this ongoing story in Europe, especially how the agenda of autonomous robots for social, economic and environmental betterment is appropriated, coordinated and enacted in and across industry, science, politics and law. In so doing, our paper adds to a growing literature on imaginaries as an intake to the study of the institutionalization of complex networks of innovation actors (Felt et al. 2007). Jasanoff and Kim (2009) describe how socio-technical imaginaries project desirable collective futures and social orders: common meanings that inspire and enable actors to coordinate their efforts across institutional, professional and disciplinary backgrounds: opening up possibilities and enabling certain constellations of actors to form. This conceptual approach has been joined by studies focused on economic relations and market-making (Levidow 2013), and applied to governance aspects of science, technology and innovation more generally (cf. Jasanoff and Kim 2015). Recently, it has also been expanded to critical studies of innovation as innovation imaginaries (Pfotenhauer and Jasanoff 2017), including in the digital domains (Gunnarsdóttir and Rommetveit 2017; Rommetveit and Wynne 2017).
Innovation imaginaries performatively mix up previous distinctions between life and market, private and public, human and machine, science and politics. They are amalgamations in which the making of European identity, creation of new markets, and the solution to pressing problems (Pfotenhauer and Jasanoff 2017), come together in new ways. The agenda to make autonomous robots to address societal problems has performatively funneled collective future-making. through digital innovation. Digital technologies are imbued with a universalizing logic of connecting across the physical, biological and social (cf. Bowker 1993), with application everywhere, at any time, to anything or any societal practice. Increasingly, social studies of robotics are addressing this upscaled political, institutional and economic role (cf. Pfadenhauer 2014; Matsuzaki and Hindemann 2016), including roboethics (Haselager 2005; Veruggio 2006; Nagenborg et al. 2008) and ‘robolaw’ (de Cock Buning et al. 2012; Bertolini and Palmerini 2014; Palmerini et al. 2014). Science and Technology Studies (STS) research focused on imaginaries and co-production can strengthen and contribute to social studies of robotics by situating the institutional and legal performativity of the autonomous robotics agenda within a broader political innovation economy.
Major questions resounding from these approaches pertain to the characterization of imaginaries as both disruptive and continuous with regard to previous forms of social life (Jasanoff and Kim 2009), and how they spread and stabilize (or not) over time. In this paper we focus on how the autonomous robots imaginary is appropriated, operationalized and aligned in and across various practices. We highlight two dimensions central to this work: first, the ongoing strategic effort to anchor the imaginary within the public institutions, described in terms of subsequent stages: the making of a technology platform (2005), a coordination action (2010) and a public–private partnership (2014). Second, we highlight the mobilization of legal scholarship and Parliamentary Procedure as a strategic resource in stabilizing future expectations in an area marked by high levels of technology- and market-related insecurities (Matsuzaki and Hindemann 2016).
We describe how these efforts at mutually alignment (cf. Te Kulve and Rip 2011) across technical, normative and institutional domains, contribute to constituting a techno-epistemic network. The main characteristic of this network(ing) resides in how it mobilizes and orders sources of knowledge and authority for the purpose of innovating for a societal purpose.2 Although there is great diversity among the constituent actors in our case, and their obligations differ, they share a commitment to realize a technoscientific application domain which is the sine qua non of a techno-epistemic network in the making. We emphasize the ‘epistemic’ work as an axiomatic element, given that the main currency in ongoing attempts at stabilizing cross-domain relations consists in committing the relevant networks of expertise and authority. For this purpose they need to convincingly muster representations of their epistemic ordering into structured interdisciplinary or cross-sectoral interactions, collaborative research agendas, innovation partnership organograms, and organizational procedures and practices.3 Hence the central role of fictions such as ‘sentient machines,’ ‘robot companions,’ ‘electronic personhood,’ ‘machine intelligence’ and ‘autonomy,’ all of which are subsumed under the key policy problematic in question, robot autonomy.
The main organizing tool and metaphor in this co-construction work is that of the roadmap whose main characteristics stem from the peculiar fact that it depicts a road that has yet to be built (cf. Scott 1998), and that eventually will be the single road on the map. The autonomous robot imaginary remains vague and open-ended, serving to attract, mobilize and inspire different actors into action. It serves to project and will a road into existence, rendering the innovation object operational and actionable in a series of (relatively) concrete steps. It represents the work that needs doing, by incorporating the required knowledge and expertise to connect physical, social, organizational and digital/virtual worlds. For instance, it guides robotics developments through a set of ‘application scenarios’ toward concrete ‘product visions’. The roadmap is also a crucial aligning device (Te Kulve and Rip 2011), coordinating the clustering together of law, ethics, science, industry, publics, shaping the network along a commonly laid out path, and assigning each epistemic practice its concrete task. In attending to this strategy, we describe an emerging division of labor: Science provides new knowledge to overcome gaps by building bridges and technoscientific enablers, whereas Ethics, Law and Sociology (ELS) scholars sweep the road of any potential ELS obstacles.
The insecurities against which these measures are hedged range from the narrowly-technical matter of industrial and strategic competitiveness to the broadly public matter of acceptability. As regards competitiveness, European robotics is assessed to be in a position of relative strength for civilian robotics (esp. manufacture and care), however, perceived as constantly at risk of losing out to global competitors. There are highly complex technical issues in play here (such as control architectures) and legal issues (especially liability) relating to potential release of autonomously acting and learning machines into unstructured environments: with increased efficiency gains and expansions of use into new domains, legal and technical uncertainties increase. Institutionally, these problems are reflected in complex negotiations and entanglements, considering that the creation of the network is already embedded in European governance of innovation. If seen from the point of view of rational decision-making, the gaps and obstacles may appear insurmountable. However, when seen from an STS co-production perspective (Shapin and Schaffer 1985; Jasanoff 2004), technical and legal problems serve to drive problem solving along the institutional chain of influence and impact.4
As we describe, institutional and constitutional problems result from the underlying machine-centrism and speculative character of the agenda. Firstly, the machine-centric agenda treats the entrenched public fear and skepticism of autonomous machines as obstacles and barriers, constituting an obstacle-model of public problems (Rommetveit and Wynne 2017). These public problems are not swept off the path, however, but pushed into the future. Second, concerning law, the sustained machine-centric focus on robot autonomy touches upon the core of human-centric European constitutions (Nagenborg et al. 2008), where it encounters resistance and controversy. We will highlight efforts to qualify concepts such as ‘agency,’ ‘autonomy’ and ‘personhood’ used to describe machine capabilities, in legal terms. We will follow how these discussions feed into an initiative to influence the European legislature: a legal pondering of possible futures, situating speculative legal scholarship in the co-production of robotics and society.
In the three main sections of this paper, we address (second section) the shaping of EU robotics networks, visions and strategies, (third section) the makings of robot autonomy as a topic for ongoing exploration, perplexity and debate by industry, science and law, and (fourth section) the kinds of co-production enabled (or not) by these actions of the emerging techno-epistemic network.
Note on Method
This paper is built on a case study on robotics,5 exploring the imaginary of robot autonomy as a vision seeking practical achievements, by: 1. identifying the main networks involved in the development of robotics in Europe, following the documentary trail of research networks and associated expert groups; 2. identifying the relevant policy discourse and implementation strategies concerning industry, research and innovation, along with other activities with bearing on regulatory and legislative outcomes; and, 3. interviewing practitioners working within these networks, attending their conferences and inviting them to a two-day workshop. Taking our cue from constructive technology assessment (Te Kulve and Rip 2011), we dedicated our workshop to a discussion of ‘robot autonomy’ as a public–political, ethical-legal and technological project, inviting representatives from different core-groups within the wider network (Rommetveit et al. 2015).
The majority of our data consist of publicly available documents (n>300), considering also documented developments of significant relevance after our case study completed in 2015 to capture the fate of some of the work which was intended to influence (and educate) legislators. The documents in this database classify as ‘visionary’ (mission statements, strategic agendas, roadmaps), project applications and project deliverables, technical reports, high-level expert group (HLG) reports, ELS assessments, policy recommendations, Green and White Papers, parliamentary committee opinions, minutes (expert hearings), presentations, handbooks, websites and press releases. Other data are interviews (n=7) and our own documentary practices in capturing observations at the events we organized and attended.
Our analytic focus concerns content and argumentation in this discourse, considering the strategic relevance and practical use of the subset of documents we sampled for this paper and compared with the interviews and our own documentation of real-life events. As Ten Have explains, all such documents are specimens of their own type and can be considered some form of evidence, produced and reproduced to assist an activity and bring it to attention (Ten Have 2004). Conceived in this way, documents are social facts, deliberately produced, distributed and shared in organized ways for one or other reason (e.g., Silverman 2006). The investigator should thus be sensitive to the conditions of producing them, the conditions for making them available, their practical purposes in how they both ‘fix’ and re-frame certain aspects of events, actions, ideology, reasoning, planning and strategy. To this effect, we follow the footsteps of studies of imaginaries and their institutionalization (cf. Jasanoff and Kim 2009; Pfotenhauer and Jasanoff 2017), i.e., in considering the practical relevance of documentation and how the document trail is evidence of innovation-in-practice, including the implementation of agendas, legislative activities and other efforts that respond to, as well as commit, industry, research and innovation.
We use this approach to map alignments in the shaping of a techno-epistemic network around the vision of autonomous robots, and we pay attention to the performative nature of the documents in aiming to produce certain effects and outcomes, e.g., realizing the innovation imaginary, orchestrating a plurality of voices, aligning different epistemic contributions to an innovation goal, resisting machine-centrism, seeking clarifications, and so on. In other words, we analyze these documents as steps in reaching goals and testifying on behalf of the practices they are assisting, i.e., in terms of content and argumentation but also structure and (cross-)references to other documents.
In approaching our own documentation, we were listening for characterizations of ‘robot autonomy’ as a practical professional achievement as well as a complex public innovation project with ethical, legal and social implications and an uncertain future. We listened for tensions and complexities, alignments and misalignment in expressions about practice, especially, as collaborations are sought out across professional and institutional boundaries. In analyzing these data, and in comparison with the results of our document analyses, we focus on the framing of problem domains and research interests, frictions and tensions therein, and the value-laden logics we observe in the argumentation at work in communications of and with different groupings within the wider network (cf. Te Kulve and Rip 2011).
The Constitution of the Techno-Epistemic Network for EU Robotics
The institutional endorsement of an autonomous robot imaginary marks the gradual emergence of the techno-epistemic network settling within EU institutions.6 In the shaping of the network, the actors perform a vision of society in which all sorts of everyday human tasks are delegated to machines. These developments are projected as beneficial to society, markets and technology users alike. Initially quite vague and underspecified, the imaginary has gradually taken on a more tangible and institutionalized form, e.g., in project communications and media discourse on new roles for robots in care and companionship, household chores, security, policing and commerce. The imaginary prepares publics and professionals for domesticated robots, embracing a vision of a robot-integrated society in ways that do not invoke fear and stigma. The imaginary foregrounds also new means of production, transport, environmental and infrastructural management, preparing for a new industrial era in which Europe will have a competitive edge.
Instrumental to robotics development at the time was the work of the advisory group to the Information Society Technologies (IST) Program of the European Commission, ISTAG. The group was made up of representatives from ICT-related industries and academe, and counseled the Commission between 1999 and 2012 on investment priorities in the IST/ICT Work Programs. ISTAG included robotic companions in 2004, as one of the visionary technologies that could both address economic needs and societal challenges in the future (ISTAG 2004). The group also suggested robot companions in 2009, as a domain of innovation that could achieve Flagship status under the Future and Emerging Technologies (FET) scheme (ISTAG 2009).
We need to achieve higher economic growth through more innovation and higher productivity, whilst creating more jobs. We need also to address many societal challenges, the ageing population, the well being of our society, and the need for security … Robotics will contribute to these challenges (Reding 2005).
The 6th Framework made possible the drafting of EUROP’s 2009 strategic research agenda (and roadmap), ‘Robot Visions,’ as a so-called Coordinated Action (CA) project, called CARE. This effort consisted of 125 partners from the European robotics industry. Then in 2010, another CA project, euRobotics, was initiated in which the industry-driven network (EUROP) and the EU-funded research excellence network (EURON) came together. This new coordination action, euRobotics, has since morphed into a PPP contract with the European Commission (Bischoff et al. 2010)—a partnership that strategically supports innovation in robotics along the lines endorsed by Commissioner Reding, and establishes contractual relations between public institutions (i.e., the European Commission) and an industry-led network (see Figure 1 in the electronic supplementary material).
In approaching the various ways in which different actors have positioned themselves in relation to the robotic imaginary in Europe, we highlight the following:
Industry has increasingly appropriated the roadmapping metaphor, using it as a control device to ‘coordinate’ the actions of other networks. The roadmap refers to an organized strategy embedded in strategic vision documents (e.g., EUROP 2009, euRobotics 2013–2014), or more specific technical and ELSA documents (Guhl and Zhang 2011; Leroux and Labruto 2012). Among key actions is forging a common language for all practices involved and a vision for guidance of pan-European activities: “industry will be the main driver behind these targeted stimulations because its needs for innovation and strong positioning in the worldwide robotics market are the greatest” (EUROP 2009, our italics).
As concerns robotics as a science and an academic endeavor, we followed the story of a multistage project proposal to the FET Flagship Initiatives scheme, called Robot Companions for Citizens (RCC 2012). The proposal came out of previous strategic work and collaborations in ISTAG and EURON. It made it to the last round of six projects, but was not one of the two that eventually achieved Flagship status in 2014. The RCC did, however, mobilize most of the academic robotics community with contributions to offer to adaptable, sociable robots for care and companionship, and was a kind of a ‘project of projects’ in this area (with partners from 73 academic institutions).
With respect to law (and ethics), we observe two key outputs from ELS activities coordinated by euRobotics, with funding from the EC under Coordination and Support Action (CSA): 1. a proposal for a Green Paper to stimulate discussion which might give rise to a legislative proposal on robotic technologies, hereafter the ‘Green Paper’ (Leroux et al. 2012: 2). A proposal for a White Paper, for launching a stakeholder debate on how to legislate and regulate robotics within the EU, hereafter the ‘White Paper.’7 The Green Paper was meant to foreshadow the White Paper (called a “forerunner”),8 yet, when the White Paper finally came out, it was not worked out within the confines of the industry strategy but mainly by legal scholars from Robolaw, an EU-funded collaborative research project.9 The findings of the Robolaw project were presented to the Committee for Legal Affairs and Internal Market of the European Parliament (the JURI Committee), which created a Working Group on the ‘Legal Questions related to the Development of Robotics and Artificial Intelligence.’ They eventually formulated a Motion for a European Parliament Resolution on ‘Civil Law Rules on Robotics’ which can be seen to incorporate elements from both the Green and White Papers (EP 2017).
The Makings of Autonomy
In this section, we study how industry, science and law have engaged the imaginary of autonomous intelligent robots in society: how they perceive of the challenges and construct the visions, goals and strategies for realizing these developments. Our main focus is on the variety of concepts deployed to align different parts and actors of the network. The role of metaphors and analogies is particularly powerful here: In the case of industry, metaphors are taken from the assembly line of the traditional factory, and extended to the workings of society; in the case of science we observe the imaginary of the natural biological ‘machine’ which becomes a metaphor in describing assistive systems for users in need of help, and to address ‘societal challenges’; In law, the key concepts in legal terms refer to basic attributes ascribed to natural persons, such as personhood, agency and autonomy, the yardstick against which such attributes can or cannot describe non-human entities.
Industry: Re-making and Extending the Assembly Line
Robots were introduced to manufacture, mainly of automobiles, in the early 1960s, with the first robots on the market from companies such as KUKA and ABB in the early 1970s. But it was not until the 1980s that robots became mainstay in industrial production. The key concept describing their function is robotic and the classic assembly line of robots referred to as automation (cf. Suchman 2007). Emblematically, robot arms are installed in factory and assembly lines to which a host of formerly routine human actions are delegated such as sorting, distributing, welding, assembling, bolting and painting, one task leading deterministically to the next. ‘Degrees of machine freedom’ occur within a three-dimensional geometric field, literally shackled to the shop floor. More recently, however, different robots have been imagined and experimented with: ‘cobots’ for work environments beyond manufacture, e.g., in logistics, warehousing and healthcare; and, experiments with drones and driverless cars operating autonomously in complex semi-structured and unstructured environments. This shift in thinking what robots can achieve is emblematic of the gradual blurring of industrial, service and assistive robotics (euRobotics 2013–2014). Shifts in enabling technologies are projected to take society from robotic automaton in closed-off secure settings to that of the autonomous robot in living social and working environments (EUROP 2009); not just automation but autonomy (Floridi and Sanders 2004); not just repetitive movement, but flexibility, adaptability and learning. (Siciliano and Khatib 2008). The industry roadmap projected how ‘[w]ith increased flexibility and ease of use, robots are at the dawn of a new era, turning them into ubiquitous helpers to improve our quality of life by delivering efficient services in our homes, offices and public spaces’ (EUROP 2009: 7, cf. also euRobotics 2013–2014: 15).
The anticipation of autonomy is mobilized together with ‘application requirements’ such as adaptation, positioning, human–robot interaction, robot-robot interaction and dependability, to be translated into more concrete ‘product visions’ (Ibid.). Although robot autonomy is not the same as human autonomy (Haselager 2005), it is seen to share some of the same virtual or presumed characteristics: a regulative principle or meta-property to steer actions and strategies not identifiable in any single body part or application. It is not a technical specification of machines; rather, it projects and expects an evolving relation between humans and machines: “Autonomy is the system’s ability to independently perform a task, a process or system adjustment. The level of autonomy can be assessed by defining the necessary degree of human intervention…” (EUROP 2009: 22).
Signs of developing service and supply chains would evidence a growing market of useful and usable robots, however, all sorts of potential application domains cannot be directly assessed because they remain to be built (cf. Scott 1998). Central to that construction task is the need to influence legislation, regulation and standards and strengthen the common language used in the robotics communities, e.g., through the circulation of strategic documents, roadmaps, newsletters, job announcements, events, etc. Roboticists are actively fostering a community, for example, with robot competitions and other outward projections and promotions of the promise of autonomous machines aimed at investors, policymakers and publics.
The robotics market is not only composed of end user applications and robot technology suppliers but also of service and supply chains which add value. The early stage nature of the robotics market means that these are not yet fully developed (euRobotics 2013-2014: 27).
In this sense the progression of industry, extending assembly line robots to societal robot integration, is inseparable from acceptability of robot autonomy. But for the expansion of Europe-owned and controlled technologies, the way ahead is seen as one of incremental improvements upon core enabling technologies and regulations. Here, the industrial strategy mobilizes and implements an “economic imaginary” (Jessop 2009), giving “meaning and shape to the economic field,” at the same time performatively suggestive of the regulatory environment and technological economy required for its realization. It singles out new domains and value chains in citizens’ living and working environments, including an imagined community of interest (Levidow 2013). This is a Europe attached to its ownership (mainly German) of certain technological and scientific domains of highly competitive advantage (Fuchs 2018), yet vulnerable to international competition. Against this background, the roadmap envisions and orders the way forward toward new ‘partnerships’ of public and private enterprise, humans and machines.
The obstacles for robots have to be investigated. There is competition with Korea and Japan. .. ELS issues need to be investigated that hinder solutions. European robotics industry has to be made world leader … so that social science related to robotics has leadership of opinion in the world. This is why the Green paper was produced (author’s notes).
Academic Research: Making Nature’s Friendly Helpers?
Visions of autonomous machines have historically captured the imagination, and are at the heart of ongoing efforts to better understand life and unique human characteristics (Riskin 2007). This, one could say, is a genuinely academic and experimental endeavour, and a philosophical one, at arm’s length from the industry’s practical reasoning. The RCC proposal is an instantiation of what happens when researchers and engineers are granted the opportunity to dream big. The FET Flagship Initiatives scheme held out a promise of large funds and prestige, of revealing nature’s secrets while offering solutions to Europe’s societal, economic and existential problems. The RCC proposal played along, portraying Europe’s high standards of living as an object of global envy: “democracy, advanced economies, social inclusion and quality of life” (RCC 2012: 3), yet, these standards being threatened by man-made and natural disasters, the economic downturn, trade imbalances, and a dwindling industrial base. The primary challenge was demographic, since “never before in human history have older citizens made up such a large proportion of the European populace” (RCC 2012: 92). For this purpose, the RCC’s primary goal was the making of friendly helpers for care and companionship. The RCC identified a gap between citizens’ expectations and their capabilities to live within the means of available resources. This gap was projected as “the challenge of sustainable welfare,” to be met by “a whole new class of machines to overcome the limitations of today’s machines, new machines based on a whole new science” (ibid.). Autonomous machines were thus posited as a direct response to a European welfare challenge.
Typically, roboticists do not address the topic of autonomy head-on, but rather work by way of conceptual and experimental detours,10 in the case of the RCC proposal articulated as sentience, “the ability to integrate across perception, affect, cognition and action” (ibid.: 32). The building of sentient machines was the main scientific challenge, termed the robotics bottleneck. It stated how present robotics are advancing toward adaptable and learning machines, capable of acting in unstructured environments (Dario et al. 2011; RCC 2012), but so far not fully delivering. This lack of knowledge to deliver robot capabilities to think, build and act, fits well within the industry’s identification of gaps, and a promise to overcome them in one paradigm-changing leap. In the RCC proposal, filling gaps was branded as a challenge of enablement, to be met by building “a clear bridge between basic science, technology and society” and between “scientific vision and its implementation into concrete innovation and engineering objectives” (RCC 2012: 11). The pillars of the bridge were five: Simplexity, Morphological Computation, Novel Fabrication Technologies, Sentience and Society. A biomimetic approach that studies and models the adaptive mechanisms developed by living beings (micro-scale, invertebrate, vertebrate and human) over millions of years, underpinned and unified the design of this bridge. Seeing nature as an ‘engineer,’ these sorts of ‘mechanisms’ were turned into fundamental design principles taken to drive the evolution of bodies and brains.
Here we can see the differences between the academic and industrial robotics networks, but also their interactions and attempts at adjustment. Industry actors deliberately identify and map, to the greatest possible extent, the 66 knowledge gaps, leading to a set of recommendations for gradual improvements (Guhl and Zhang 2011). Academics, on the other hand, plunge themselves into the unknown unknowns of Nature’s secrets, seeing the gaps as constitutive of the road-building enterprise. To them, industry’s incremental approach is likely to lead to “a gradual loss of controllability and robustness, and … ultimately …to a substantial cost in efficiency and safety” (RCC 2012: 5; cf. Bekey 2005).11 They articulate and mobilize the notion of a gap, when arguing that these shortcomings of controllability, paralleled in the legal problem of liability, can only be overcome in a paradigmatic leap to sentience. Here, scientific ingenuity is crucial. The challenge is accommodated within the broader roadmap strategy where concepts and metaphors such as ‘bridging gaps,’ and ‘overcoming limits and bottlenecks’ have a central role.
At its core, the RCC vision was similar to the industry vision in being deeply hybrid, a socio-technical imaginary (Jasanoff and Kim 2009) of a new society, driven and assisted by robot companions for humans and robots working alongside them in most walks of life. Importantly, both are machine-centric in their suggestions of economic and social change. However, the academically-driven consortium left relatively little room for input from industry and for its mediating role to deliver actual products to markets. This was also paralleled in an initial lack of attention to ethical, legal and societal issues, however, when the relevant expertise finally came on board, it was generously included in the discovery engine. The careful attention to community relations however, characteristic of the industry roadmap, was largely absent. Thus, the main identifier and basis for collective action within the RCC consortium, was the fascination with building and exploring things that move, act, think and feel, without a clear pathway to innovation to deliver societal goods and products to market.
Legal Studies: Qualifying as Man’s Friendly Helper or Self-standing Person?
The emergence of robots capable of autonomous decisions is seen as a challenge to human-centric constitutions and possibly resulting in a paradigmatic shift in legal thinking, considering the speculative character of machine autonomy (De Cock Buning et al. 2012). In this section we describe the work of two legal networks on this topic, the machine-centric Green Paper developed within the industry roadmap of euRobotics, and the human-centric White Paper of the Robolaw project. We highlight their different positions on robot autonomy in relation to legal frameworks, existing laws and charters, the former proposing electronic personhood for robots. From those basic positions different problem-frames and strategies follow, which we attend to in the section “Public Realignments in Co-producing the Partnership”.
Green Paper: From Human Tools to Electronic Persons
The Green Paper declares itself to be the first European effort at bringing together the robotics and legal communities, supported by an ELS assessment document (Leroux and Labruto 2012). The roadmap metaphor is up-front in the ELS assessment, reminding the reader that the general objective of euRobotics is “to act and find ways to favour the development of European robotics,” and deal with potential “worries about the consequences of introducing robots into society” (Ibid.: 5). It appears as if ELS issues are “hindering the development of robotics in Europe,” hence the “roadmap to overcome them.”12 This framing of ELS issues provides the starting point for interdisciplinary collaboration. ELS issues are presented as ‘barriers’ or ‘obstacles’ that need to be removed, preferably before they arise and the contributions of legal scholars, ethicists, social scientists and engineers must aim at this road-sweeping task.
Arguably, the communities of engineers and lawyers should get to “know each other” through this work, “share common language, vision and objective” (Leroux et al. 2012: 8) and, predominantly, share the concept of autonomy: “It is precisely that “interdisciplinary” collaboration that is the main reason for the current debate on the meaning of the word autonomy” (Ibid.: 11–12). The Green Paper provides different meanings of ‘autonomy’ in law, engineering and ethics. Yet, the main authors are industry representatives and the problems in getting on with collaboration are seen primarily from an engineering perspective.13 The paper blurs human–machine categories and disciplinary relations, since differing conceptions are themselves among the main obstacles. It also sidestepped an Ethics Roadmap created at the time (Veruggio 2006), and the results of the Ethicbots project (Tamburrini 2009), both of which took a human-centric approach.
The main normative obstacles pertain to issues concerning human autonomy like human rights, which are framed as ethical issues, not ‘legal issues.’ An important feature of the method the authors use is to only focus on issues that (speculatively) are specific to autonomous robots and not technologies in general. “This approach means that we always try to guess if ELS issues disappear or not when we replace the word “robot” by “device”, “robotics” by “technology” (Leroux and Labruto 2012: 9, our italics). In the analysis of different robotics fields (assistive, security, toy and sex robots, to exo-skeletons), ethical issues like privacy, equality and dignity turn out not to be robo-specific, since some analogy can always be found in an adjacent technological field. These human autonomy-focused issues thus disappear from further analysis.
Having set aside the ethical issues, the Green Paper turns to major legal obstacles to robot autonomy. These touch upon the core of legal systems: the fundamental distinction between legal subjects with agency and legal objects as physical entities. Robots are positioned as hybrids of the two. Depending on the legal regime in question (intellectual property rights, liability, legal capacity), certain autonomy-related qualities are attributed to robots (creativity, making choices, being a person), which serves to attribute legal qualities (authorship, liability, personhood), previously reserved for humans.
The Green Paper distinguishes between robots as physical entities (i.e., objects) and robots as a kind of agent, i.e. as quasi-subjectivities, and ventures into speculative terrains with respect to the second. This tendency is clearly seen in the chapter on intellectual property rights (IPRs). Following a broad outline of existing IPR laws concerning robots as objects of appropriation, the mode of analysis switches to the possibility of ‘robot generated works,’ whereby the machine is considered the subject, or author, of works worthy of IPR protections.
The overarching obstacle, however, remains the issue of liability (De Cock Buning 2012). Nobody wants to invest in robots if there is uncertainty about who is liable for damages caused by their behaviors. Until now, robots have been regarded as physical objects, but this conception might prove problematic for robots with self-adaptive and decision-making abilities (Boscarato 2011). The Green Paper offers a gradient legal analysis in which different forms of non-contractual liability are tailored according to a robot’s increased capability: starting from behaviors determined by producers, the analysis proceeds to machines that can “move freely in the surrounding space.” When a robot leaves the confines of its owner and causes harm or damage, it could be qualified as an ‘animal’ in the sense of article VI. 3:203 of the European Civil Code, and the custodian is liable. When robots possess decision-making and learning skills however, leading to behaviors not intended by producers or programmers, they may be qualified as a ‘child’ in the sense of article VI. 3:104 of the European Civil Code. This qualification becomes especially relevant when robots are modeled after animals, as in the RCC’s bio-mimetic approach. The ‘parents’ or ‘guardians’ of the robot would be held liable for damage or harm if their supervision has been deficient. Note that in such reasoning by analogy, certain actors are foregrounded (e.g., owners and guardians) whereas others disappear or are less relevant (e.g., producers), thus facilitating a transfer of liability.
Based on this, the Green Paper goes on to argue that robot autonomy is a central challenge to human-centric charters and judiciaries (method and practice), stating how “strict differentiation between man and machine (“man-machine-dualism”) is no longer acceptable” and that “man and machine should be considered simultaneously and their actions should be seen as cooperation” (p. 58). In other words, robot autonomy is positioned as a hybrid agency no longer bound by the passive concept of legal objectivity. This framing opens up for legal innovation, i.e., in exploring the self-admittedly speculative possibility of turning robots into ‘electronic persons.’ Robots should be granted “a special legal category” (Ibid.: 61) that would for instance allow them to be held directly liable for any damage they cause. This category of ‘electronic personhood’ builds upon notions of software agents in AI, presumptions that embodiment will make software agents more intelligent, and analogy with the legal personhood of corporations which suggests that electronic personhood is a solution to the problem of liability and responsibility, i.e., by a “bundling of capacities, material and financial responsibilities” (Ibid.: 61).
The Green Paper also provides an account of European legal structures, the main barrier being the sheer complexity and many-layered character of European law (Ibid.: 13–14, cf. 66). The Paper argues the need to “harmonize the legislation concerning robotics in Europe. Industries are confronted to different regulation and legal constraints which represent barriers making difficult the emergence of new markets.” Here, electronic personhood as a novel construct could prove to be useful. If the liability problem can be solved on a cross-jurisdictional basis, future relations and expectations of participants and investors on the European digital market would be provided with legal certainty.
White Paper: Putting Humans and Society First?
The machine-centrism of the Green Paper comes more clearly into view when contrasted with the legal work delivered by Robolaw. One of the main objectives of the project was to investigate how emerging technologies in the field of robotics have a bearing on the content, meaning and setting of laws and how they will affect existing legal categories and qualifications. Analytically, the authors do not prioritize that robots are about to enter society. Rather, they prioritize the conglomerate of legal regulations across Europe dealing with robotics. The mapping they do is one of a barely chartered terrain, whereby, mapping entails a comparative analysis of legislation in several different countries. New-emerging robotics are presented in the paper as an a-territorial and cross-boundaries phenomena, and that legally hybrid entities are moving between different legal systems, which calls for a development of a “specific European approach … characterized by core “European values” deriving from the main European sources of law.”
The goal is to avoid robots becoming disruptive to societal structures and to ensure that their benefits are fully exploited. Such a functional perspective “means to put rights – and fundamental rights as recognized by the European Union – first […],” requiring that “the impact the single application may have on society and fundamental rights shall be guiding the choice” (Bertolini and Palmerini 2014: 169).
The analysis in the White Paper orients to existing legal safeguards in society, e.g., how human rights could be affected by future robotics. What occurs in the Green Paper as an ‘obstacle’ is articulated in the White Paper as the premise on which roles for machines in society can be assessed and implemented. This focus on fundamental rights puts humans first, since “human rights are in fact an essential apparatus to deploy in order to promote and guarantee responsible advances in science and technology.” Human rights draw the boundaries for robotics development by indexing the ‘intangible’ zones that cannot be infringed. They function as “a test-bed for the desirability of robotic applications, since they can serve to identify the main goals and achievements expected by advancements in robotic research and industrial applications” (Ibid.: 176–177).
In this section we focused on mediations accomplished through strategic and intellectual work performed by industry, science and law. Whereas this work is premised on the imaginary of autonomous robots, each practice remains dependent on its own epistemic home-base and styles of reasoning, although, stretching fundamental discipline-specific concepts to meet new goals. This illustrates our analytic point that roadmaps are used to operationalize imaginaries and as aligning devices across networks, practices and institutions. Yet, the focus and overall goal of using a roadmap remains controversial: industrial and academic robotics remain machine-centric (although in different ways) which is transmitted to the legal and regulatory networks (Green Paper), but that also becomes subject to controversy and contradiction by a human-centric legal and constitutional approach (White Paper, cf. Nagenborg et al. 2008; see Figure 2 in the electronic supplementary material).
The role of institutions and networks in innovation is a key theme in STS perspectives on co-production, insofar as STS researchers shift attention away from the scientific and technological problem domains, toward the social and institutional relations and networks in which the co-production is embedded and takes shape. In this story the ‘embedding’ is visible as a boundary crossing activity, much like shifting in and out of situational frames of reference. We will now draw upon this theme to address the co-production of science, industry, law, and politics in the making of partnerships and robot autonomy.
Public Realignments in Co-producing the Partnership
When assessing the innovation object, the developments we have described could be seen as failures. The main strategy of roadmapping is to overcome the ‘gaps’ between industry and academic science, between engineering and law, and to ‘remove the obstacles’ separating robotics and society. Yet, the gaps persist, as reported in technical reports and communicated to us by practicing roboticists. Economically, the gaps have widened: In 2013 and 2016, two flagships of European robotics (Aldebaran and KUKA) were sold off to Asian companies, triggering ‘business angst in Europe’s economic powerhouse’ (DW 2018). Such developments cannot be separated from the question of whether robotics could actually deliver as promised, and of who would bear the long-term costs of the promissory enterprise. One roboticist told us that the European program to address problems of aging societies through robots is, “a disaster based in erroneous ideas of what robotics can and cannot deliver” (Rommetveit et al. 2015). Scientifically, biomimetic approaches are riddled with shortcomings of their own, and the persistent underlying assumptions about realizing (‘strong’) embodied intelligence could be criticized as unrealistic. Legally, ambiguity and controversy persist over how to accommodate robots within human-centric legal frameworks in the service of human rights and wider society. During our workshop we learned from lawyers that their practices come under pressure from industry, science and engineering (Rommetveit et al. 2015). We also learned that preexisting epistemic differences are deeply entrenched, hindering collaboration and understanding within the wider network (cf. Bischoff et al. 2010).
Yet, when we view these developments as co-productions of science and social order (Shapin and Schaffer 1985), success in merely scientific or technical terms is not the only thing of relevance. The making of a Public–Private Partnership is a main achievement through which European industry can promote its innovation object, and shape a large techno-epistemic network comprising economic, legal, political and market relations. The various gaps and challenges identified in the euRobotics roadmap become means toward this end, namely to partner with the public and European governance (cf. EPEC 2016). Here, however, the industry platform runs into new actors: lawyers, ethicists and politicians. It is precisely at the intersections where the public/governance side of the partnership enters the picture, that the imaginary is significantly complexified.
For the European Union, this path to innovation is part of its ongoing voyage to deliver an Innovation Union. Framing future Europe as built on humanistic values, reinforces the construction of a European identity with implications for innovation: on the one hand, by embracing the imaginary of autonomous robots for societal purposes and, on the other hand, by seeking to shape it according to European value-laden logics and modes of operation (cf. Jasanoff 2005). The presumed identity, however, diverges from dominant innovation discourse (and policy discourse on industry and innovation), of competitiveness, disruption, efficiency (displacing labor), and profit in markets.
This new pathway for robotics also has its own barriers. Confronted with a plethora of national legislation developed by Member States, “discrepancies are expected to create obstacles for an effective development of robotics. Due to the fact that this technology has cross-border implications, the best legislative option is a European one” (EP 2017: 28). The subsidiarity principle is raised here to authorize the Union’s legislative road-sweeper. Hence, the legislators mirror and embrace the technical problem-solving strategy of the industry roadmap, at the same time re-creating it in their own image. We now describe some ways in which this entanglement is manifested through a series of realignments of framing advanced robotics/AI, the suggested directions and paths to innovation.
First, the framing of a pan-European research agenda is placed within the grasp of the PPP. During our workshops, roboticists lamented how they now have to ‘follow the money’ to pursue their research, meaning, the EC investment priorities in robotics.14
Second, whereas the main goal of a PPP, according to the stakeholders, is “to connect the science base to the marketplace, a connection that ultimately benefits society,”15 the societal goal of research under a PPP contract (cf. EPEC 2016) is slightly more ample and must by necessity frame prospective developments as “addressing specific societal challenges” (European Parliament and Council” 2013, art. 25).
Third, the Parliament’s Motion for ‘Civil Law Rules on Robotics’ subscribes to the generally beneficial role of advanced robotics for society, while also highlighting potentially significant negative impacts (on work, privacy, human relations, etc.). The negative concerns are conceived largely in terms of soft impacts on human rights and societal values. Importantly, these issues have to be addressed through an elaborate new ethical framework, comprising a code of conduct for engineers, a code for research ethics committees, model licenses for designers and users, privacy by design, and more. Such responsibilization is institutionally fortified by suggesting a new ethical expert body to advise EU institutions about potential issues and systemic challenges raised by developments in robotics and AI.
Fourth, the speculative frame of the robotic imaginary also takes hold of the legislative imagination and process. The Parliament’s Motion is noteworthy for the fact that its introductory clause cites works of science fiction, and this futuring is paralleled by another novelty for the legislative process. The Parliament’s Working Group commissioned the application of STOA’s16 newly developed Scientific Foresight methodology (van Woensel 2016). This was one of the first applications of the method by the Parliament, and perhaps a first step to institutionalize in support of legislative activities. Compared to the creation and enactment of a roadmap, this foresight approach is a different vehicle for futuring and it operationalizes the imaginary of autonomous robots differently, even though the imaginary itself is generally accepted.
So, fundamental values become the measure here for robotics innovation, with serious implications for future designs.
The road from the industrial sector to the civil society environment obliges a different approach on these technologies, […] ensuring that a set of core fundamental values is translated into every stage of contact between robots, AI and humans. In this process, special emphasis should be given to human safety, privacy, integrity, dignity and autonomy (Ibid.: 27).
To summarize, we observe (A) a realignment of the innovation goals, the primary aim being to solve Europe’s societal challenges instead of relying mostly on bootstrapping new markets; (B) the re-positioning of milestones for measuring progress against fundamental values (a task for ethicists), rather than simply realizing a technological potential like autonomous self-learning robots; (C) a modulation of futuring introduced with foresight studies instead of simply following the roadmapping exercise. On the one hand, the legislative activities incorporate the imaginations already emerging within the techno-epistemic network; on the other hand, we also see that Parliamentary actors seeks distance by instituting their own futuring method. In so doing, the legislature is gradually incorporating speculative thinking and approach, potentially driven towards increasingly remote horizons. The articulation of possible futures is enrolled as a means to stabilize expectations in the present and as a basis to inform current legislative initiatives on future legal frameworks. Simultaneously, speculative ethics and ‘responsibilization’ become institutionalized to evaluate increasingly far-sighted innovation projections, sometimes to the detriment of normative issues more nearby in the present (Nordman and Rip 2009). This speculative turn testifies to the uncertainty, unpredictability and complexity of an increasingly distributed and networked innovation practice. Whether intended or not, the PPP is an instantiation of what Pellizzoni has termed “governance through disorder” since Parliamentary actions appear “attuned to the productive regard on indeterminacy and speculation that constitutes a trademark feature of neoliberal governmentality” (Pellizzoni 2015: 171).
Conclusions: Back to the Present?
In writing this paper, our point of departure was the increasingly complex social and institutional relations in Europe in steering and strategizing an innovation agenda for robotics and addressing the question of how to govern future robots. Roboticists within industry and academe joined forces and legal scholarship was called upon to institutionalize (and institute) imaginaries of autonomous intelligent machines and stabilize expectations through legal conceptions of ‘electronic personhood’ and ‘sentient machines.’ We observe here the emergence of a techno-epistemic network entangled in public governing institutions in the making of a Public–Private Partnership.
In framing these developments as a co-production of robotics and society, we have drawn attention to how different actors come together to build epistemic, institutional and political legitimacy in new and more intense ways. The actors within these networks, implicitly or explicitly, take on an identity of being co-producers: industry has opinions on agent autonomy (previously the prerogative of philosophers and ethicists); lawyers (and ethicists) take a position not only on how to guide and steer robots into society, but also on the properties that robots should possess; scientists aim to build ethics, law and ‘society’ into their (imagined) projects and artefacts. Put this way, we provide a wide perspective on the processes of constructing what we call a techno-epistemic network of European robotics.
We have highlighted the crucial role of imaginaries and futuring in this work: how the imagined-possible seeps into legislative practice. The autonomous robot imaginary has triggered response by legislative actors, who have introduced a range of ‘societal’ perspectives and emphasized the human-centeredness of laws and charters. Yet this public agenda is constructed on top of a machine-centric ideology whose aim is to overcome public fears and skepticism by recasting them as obstacles. The robot imaginary is predicated on fascination and willfulness, enabling concepts and visions to travel across networks and institutions. When ‘gaps,’ ‘obstacles’ and other conceptions become the means to political, policy and market-making ends, the character and purpose of the innovation practice changes. For example, both scientists and lawyers complain that their professional practices are under pressure, having to bend their contributions under an industry-dominated discovery engine along with political and market-making ends such as ‘autonomous robots for care and companionship’ and ‘saving Europe’s economy.’ Legal scholars contribute their expertise in matters of agency, autonomy, personhood and responsibility against the unknown of conflicts that may arise between human-centric legal systems that safeguard these phenomena and machine-centric designs pushing at their boundaries. This work appears to intensify speculation and disorder in governance, and perplexity about autonomous robots, what they are capable of and what meanings to attach to them.
The developments observed in this paper might well go beyond robotics, as technologically addressing Europe’s societal challenges has its own set of challenges. Large industry actors competing in global markets progressively attain a tight embrace of these hybrid processes by which practitioners of different backgrounds come together to anticipate future problems. This highlights the need to shift the attention away from the object of innovation (machines, technology) to societal and institutional relations, and from the future back to the present to attend to more immediate human and public concerns.
Sources of inspiration for this concept are Haas (1992) and Stengers (2005). Our approach differs from Haas’ notion of an ‘epistemic community’ by foregrounding the strong driving force of technology in the making of new relations across ontological, disciplinary professional and institutional domains.
We have called such epistemic ordering devices ‘epigrams’ (van Dijk 2019).
Jasanoff describes how “the EU has found it necessary to specify the problems it wants to solve in order to legitimate its political existence” (2005: 93).
EPINET (FP7-SiS, 05/2012 – 04/2015, Grant nr. 288971).
Since, only ‘when the originator’s «vanguard vision» comes to be communally adopted … does it rise to the status of an imaginary’ (Jasanoff 2015: 4).
Since these documents are not official documents of the European Commission, they are strictly speaking not Green or White Papers, but proposals to be considered as such.
In: “ELS issues in Robotics” workshop description, at ERF 2012, Odense, Denmark.
Described by our workshop participants through concepts such as adaptability, intelligence, rationality and relative autonomy (Rommetveit 2015).
Based on existing science, ‘machine autonomy’ may entail a contradiction. As explained by one roboticist, a persistent tendency is that ‘When machine intelligence goes up user friendliness goes down’ (Rommetveit 2015).
In our workshop ethicists argued that industry and roboticists persistently tended to blur distinctions between human and machine autonomy, whereas these are very different things (Rommetveit 2015).
A robotics researcher argued that “we will see all sorts of tensions arise between the different actors who are now involved in taking the research agendas forward” (Rommetveit 2015).
STOA: European Parliament’s Science and Technology Options Assessment Panel.
- Bekey, George A. 2005. Autonomous Robots. From Biological Inspiration to Implementation and Control. Massachusetts, MA: The MIT Press.Google Scholar
- Bertolini, Andrea, and Erica Palmerini. 2014. Regulating Robotics: A Challenge for Europe. In Legal Affairs Committee, UPCOMING ISSUES OF EU LAW. Brussels: European Parliament.Google Scholar
- Bischoff, Rainer et al. 2010. euRobotics—Shaping the future of European robotics. ISR/ROBOTIK: 728–735.Google Scholar
- Boscarato, Chiara. 2011. Who is responsibe for a robot’s actions? In Technologies on the Stand. Legal and Ethical Questions in Neuroscience and Robotics, eds. Bibi van den Berg and Laura Klaming, 383–402. Nijmegen: Wolf Legal Publishers.Google Scholar
- De Cock Buning, Madeleine et al. 2012. Mapping the Legal Framework for the Introduction into Society of Robots as Autonomous Intelligent Systems. In The Law of the Future and the Future of Law (Vol II), eds. S. Muller et al., 195-210. The Hague: Torkel Opsahl Academic Epublisher. Cambridge, MA.Google Scholar
- DW. 2018. Changes at German robotics firm Kuka raise questions over Chinese intentions. Deutsche Welle Nov. 28. 2018.Google Scholar
- [EC] European Commission. 2005. Report on European Technology Platforms and Joint Technology Initiatives: Fostering Public–Private R&D Partnerships to Boost Europe’s Industrial Competitiveness. SEC(2005) 800, http://ec.europa.eu/research/fp7/pdf/tp_report_council.pdf. Accessed 25 May 2019.
- [EP] European Parliament. 2017. Report with recommendations to the Commission on Civil Law Rules on Robotics (2015/2103(INL)). Brussels: European Parliament. http://www.europarl.europa.eu/doceo/document/A-8-2017-0005_EN.html. Accessed 25 May 2019.
- [EP] European Parliament. 2018. General Principles of EU Industrial Policy (F. Gouardères & S. Horl). Fact Sheets on the European Union, 05/18. http://www.europarl.europa.eu/factsheets/en/sheet/61/general-principles-of-eu-industrial-policy. Accessed 25 May 2019.
- EPEC. 2016. World Bank Group: The Guide to Guidance. How to Prepare, Procure and Deliver PPP Projects. https://ppp.worldbank.org/public-private-partnership/library/epec-guide-guidance-how-prepare-procure-and-deliver-ppp-projects. Accessed 25 May 2019.
- EUROP. 2009. Robotics Visions to 2020 and Beyond - The Strategic Research Agenda for Robotics in Europe. Coordination Action for Robotics in Europe (CARE). https://ec.europa.eu/research/industrial_technologies/pdf/robotics-ppp-roadmap_en.pdf. Accessed 25 May 2019.
- euRobotics. 2013. Robotics 2020. Strategic Research Agenda for Robotics in Europe. https://www.eu-robotics.net/cms/upload/topic_groups/SRA2020_SPARC.pdf. Accessed 25 May 2019.
- euRobotics. 2015. Robotics 2020: Multi-Annual Roadmap For Robotics in Europe Call 2 ICT24 (2015)—Horizon 2020 Release B 06/02/2015. https://www.eu-robotics.net/cms/upload/downloads/ppp-documents/Multi-Annual_Roadmap2020_ICT-24_Rev_B_full.pdf. Accessed 25 May 2019.
- European Parliament and Council. 2013. Regulation No 1291/2013 establishing Horizon 2020. Official Journal of the European Union. L 347/104.Google Scholar
- Felt, Ulrike, et al. 2007. Taking European Knowledge Society Seriously: Report of the Expert Group on Science and Governance to the Science, Economy and Society Directorate. Luxembourg: European Commission.Google Scholar
- Guhl, Tim, and Zichao Zhang. 2011. euRobotics: The European Robotics Initiative. Report on identified gaps. Deliverable D1(1): 1.Google Scholar
- ISTAG. 2004. Grand Challenges in the Evolution of the Information Society. Luxemburg: Report EU Commission.Google Scholar
- ISTAG. 2009. Orientations for Work programme 2011-2013. ISTAG Report. November 2009.Google Scholar
- Jasanoff, Sheila, and Sang-Hyun Kim (eds.). 2015. Dreamscapes of Modernity: Sociotechnical Imaginaries and the Fabrication of Power. Chicago, IL: University of Chicago Press.Google Scholar
- Jessop, Bob. 2009. Cultural political economy and critical policy studies. Critical Policy Studies 3(3–4): 336–356.Google Scholar
- Leroux, Christophe, and Roberto Labruto. 2012. Ethical Legal and Societal issues in robotics D3.2.1. euRobotics.Google Scholar
- Leroux, Christophe et al. 2012. Suggestion for a green paper on legal issues in robotics. euRobotics. https://www.unipv-lawtech.eu/files/euRobotics-legal-issues-in-robotics-DRAFT_6j6ryjyp.pdf. Accessed 25 May 2019.
- Palmerini, Erica et al. 2014. Guidelines on Regulating Robotics. Robolaw Project D6.2. http://www.robolaw.eu/RoboLaw_files/documents/robolaw_d6.2_guidelinesregulatingrobotics_20140922.pdf. Accessed 25 May 2019.
- Pfadenhauer, Michaela. 2014. On the Sociality of Social Robots. A Sociology-of-Knowledge Perspective. Science, Technology & Innovation Studies 10(1): 135–153.Google Scholar
- Pellizzoni, Luigi. 2015. Ontological Politics in a Disposable World. The New Mastery of Nature. New York: Routledge.Google Scholar
- Rommetveit, Kjetil et al. 2015. Report from workshop: Making robotic autonomy through science and law? Project deliverable. https://eprints.lancs.ac.uk/id/eprint/74258/ Accessed 25 May 2019.
- Reding, Viviane. 2005. Advanced Robotics: why Europe needs to catch up in this strategic sector. Presentation at official launch of the Technology Platform EUROP. http://www.europa.eu/rapid/press-release_SPEECH-05-583_en.pdf. Accessed 25 May 2019.
- RCC Consortium (2012) MANIFESTO. More than machines. Robot Companions for Citizens. http://www.robotcompanions.eu. Accessed 25 May 2019.
- Riskin, J. (ed.). 2007. Genesis Redux. Essays in the History and Philosophy of Artificial Life. Chicago and London: Chicago University Press.Google Scholar
- Scott, James C. 1998. Seeing Like a State. How Certain Schemes to Improve the Human Condition Have Failed. New Haven and London: Yale University Press.Google Scholar
- Shapin, Steven, and Simon Schaffer. 1985. Leviathan and the Air Pump. Princeton NJ: Princeton Univ. Press.Google Scholar
- Siciliano, Bruno, and Oussama Khatib (eds.). 2008. Springer Handbook of Robotics. New York: Springer.Google Scholar
- Silverman, D. 2006. Interpreting Qualitative Data: Methods for Analyzing Talk, Text and Interaction. London: Sage Publications.Google Scholar
- Suchman, Lucy A. 2007. Human–Machine Reconfigurations: Plans and Situated Actions, 2nd ed. Cambridge: Cambridge University Press.Google Scholar
- Tamburrini, G. 2009. Robot Ethics: A view from the philosophy of science. In Ethics and Robotics, eds. Raphael Capurro and Michael Nagenborg, 11–22. Amsterdam-Heidelberg: IOS Press/AKA.Google Scholar
- van Dijk, N. 2019. A Constitutional Ecology of Practices and the Role of Epigrams. Perspectives in Science. Accepted 2019.Google Scholar
- Van Woensel, L. 2016. Ethical Aspects of Cyber-Physical Systems. European Parliament: Scientific Foresight Unit.Google Scholar
- Veruggio, Gianmarco. 2006. The EURON Roboethics Roadmap. http://www.roboethics.org/atelier2006/docs/ROBOETHICS%20ROADMAP%20Rel2.1.1.pdf. Accessed 25 May 2019.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.