Keywords

1 Introduction

Technological artefacts such as intelligent personal assistants are increasingly employed within everyday contexts. Systems such as CALO [1] and its popular spin-off Siri accompany users throughout their daily lives, supporting a wide array of leisure activities and professional obligations. Consequently, they have to operate across a broad range of diverse, hardly predictable and sometimes chaotic contexts.

This diversification of contexts of use calls into question the utility of traditional patterns of lab-based research. Many of the classical tools used when evaluating and developing said technologies such as controlled lab studies or evaluations relying on predetermined tasks do not fare well when faced with the uncertainties and ambiguities of the everyday [2, 17, 29, 31, 51].

When developing for everyday contexts, a methodological toolset is required that supports construction, description, and evaluation activities unfolding over extended periods of time within environments that we as researchers cannot control or design beforehand. The dynamic patterns of communication present within these contexts thus put a high strain on our capabilities both of scientific analysis and prospective design, challenging proven methods in respective fields.

Methodological Diversity

The field of human-computer-interaction (HCI) has acknowledged and responded to these novel challenges with a diversification of its methodological base, theoretical framings and epistemological groundings:

In-the-wild methods [30, 40, 44, 46], methodologies informed by ethnomethodology [15, 55], grounded theory [47], or ethnography [16], frameworks such as Embodied Interaction [18] and Thoughtful Design [39], have established themselves within the increasingly diverse methodological landscape of the field [45].

In reaction to challenges to the cognitive paradigm, researchers have sought to establish novel encompassing theoretical frameworks. Activity Theory (AT) was adopted as a promising candidate for informing development activities within rich real-world contexts [42].

A different approach lies in identification of incommensurable paradigms [22, 33], waves [10], and thus different communities of academic practice. An especially well received and sophisticated approach of this kind is the concept of Third paradigm HCI developed by Harrison, Sengers, and Tatar [21]. Drawing on Kuhn [32], the authors argue for the presence of three distinct academic communities operating within individual systems of epistemological commitments and methodological procedures. Accordingly, knowledge claims remain partially incommensurable, allowing mutual respect and precluding unnecessary discussion between practitioners operating within differing paradigms.

Intra-Team Diversity

When building cognitive artefacts for everyday contexts, project teams frequently find themselves in situations where stakeholders are indepted to differing paradigms or perspectives. Media psychologists employ the cognitive paradigm, interaction designers subscribe to a practice paradigm, computer scientists exercise methodological indifference, while ethnographers champion a constructive construal of reality. Most of the approaches mentioned above leave the question open how methodological coordination within interdisciplinary teams is possible that operate across paradigms, or do not achieve consensus on one of the discussed theoretical umbrella frameworks.

We argue that an interesting candidate for coordination within interdisciplinary teams is provided by a theoretical vocabulary based on the notion of complexity. On the level of methods, appropriation of techniques used within practice-based research are proposed in order to contain development and evaluation complexities. Instead of proposing an integrated theoretical-methodological stance, that all participants have to agree on, or a compartmentalisation of researcher activity into distinct paradigms, we are formulating a set of concepts that allow for processes of ongoing conflictual negotiation during development projects.

2 Theoretical Framing

Choice of a theoretical base for an interdisciplinary framework is no easy endeavour. Often, academic communities of practice remain entrenched within idiosyncratic theoretical vocabularies, nurturing theories specific to their interests, while seeking to establish discourses of their own in order to differentiate and isolate themselves from bordering disciplines.

Contact areas between disciplines do exist, however, allowing for specification of theories intelligible to multiple communities. We have identified the notion of complexity as an intellectual ‘plexus’ of this kind, connecting various disciplines that otherwise remain weakly interlinked on the level of theoretical discourse. As a concept complexity stretches disciplinary boundaries; the signifier is known within the domains of psychology, computer-science, human-computer-interaction, sociology, science and technology studies, among others. As goes without saying, such a diverse pattern of intellectual proliferation does not reproduce without effecting equivocations and ambiguities in usage: Computational complexity theory [23, 50] might understand the concept very differently from sociological systems theory [38], science and technology studies [35, 37], or design theory [51].

However, as a conceptual starting point, the concept remains useful. Where it is not accompanied by clear or congruent concepts, it at least triggers associations relating to relevant notions, which subsequently can be employed in order to bootstrap a discussion process.

2.1 Everyday Complexity

In order to give an account of complexity within everyday situations conducible to the requirements of human-computer interaction, we discuss an approach developed by interaction designer Ron Wakkary. In his text Framing complexity, design and experience: a reflective analysis Wakkary frames the problematic of everyday situations within the theoretical vocabulary of complexity. In order to describe the specific requirements of addressing the everyday within HCI projects, Wakkary highlights the necessity of approaching rich interactional networks present within domestic, cultural, and leisure contexts through practice.

As is often the case, complexity is applied as qualification in order to signify the inability of achieving an analytic solution, of not being able to break up a problem into its constituent parts or variables. Building on HCI discourse and insights derived from his own projects, Wakkary identifies this quality within everyday situations.

Being faced with a complex problem has direct implications for possible design strategies: As comprehensive analysis is impossible, design actions must operate on the basis of imperfect, provisional, and ambiguous information. The resulting problem space has to be explored through practice in a situated manner.

In response to this necessity, Wakkary proposes an approach similar to dead reckoning in navigation [51, pp. 7, 10]. Thereby, he sketches an incremental design methodology responding to the problematic of complexity. It consists of setting a general direction for the design process, which is modified and corrected during every design decision. At every point within the design process, a course correction is performed, whose direction is marked in reference to the last.

Implications for Interdisciplinary Development of Cognitive Artefacts. The concept of complexity was introduced as a possible device for regulating interdisciplinary discourse. The detailed account Wakkary provides of everyday complexity facilitates a precise description of the necessities of adopting an iterative and practice-based approach towards interdisciplinary development of artefacts performing within everyday contexts.

There is an inherent conflict between a conception of complexity entailing non-representability and a construal of cognitive artefacts as providers of representations. As a consequence, cognitive science and interaction design strategies will at times be at odds: The former aims at generalisability beyond the exigencies of specific situations, while the latter wants to explore the particularities of said situations, moulding constructed artefacts into the interactional niches reproduced within users’ life-worlds.

2.2 Complication

A further conceptual differentiation provides a positive account of the objects of cognitive and computer sciences. Science and technology studies scholar Bruno Latour proposes distinguishing between complexity and complication: Complicated phenomena can be analysed in the form of a countable set of variables, while complex phenomena either consist of non-countable sets of variables or resist description through variables altogether [35, 36].

Distinguishing complexity and complication in this fashion, allows for a characterisation of computing/cognitive level processes as well as social complexity, employing a theoretical vocabulary already received within the field of HCI [8, 14, 19, 20].

3 Boundary Objects

When operating across paradigms, discourse within a development team inevitably exhibits a high degree of diversity. Respective stakeholders frame their results within the languages of complexity or complication, construing relevant phenomena in a way specific to their perspectives. In the case of cognitive and computing artefacts, resulting systems can be highly formalised, while sociological analyses of complex phenomena entail a high amount of theoretical prerequisites. We discuss boundary objects as a conceptual aid for facilitating productive discussion among these diverse communities of practice.

Social scientist Susan Leigh Star and philosopher James R. Griesemer introduce the notion of boundary objects in their seminal study on institutional ecology [49]. They identify the boundary object as a conceptual phenomenon occurring within cooperative work contexts. Boundary objects present at least two instances of themselves, a well specified one, and a family of versions open to interpretation. Participants within a collaborative setting are able to alternate between both instances, There is a resulting discursive back-and-forth movement between well-specified and appropriated versions of the boundary object. As a result, disciplinary communities dependent on more formalised versions of the boundary object find requisite exactness within the object, while neighbouring communities can adapt and reconstruct it according to their respective disciplinary language games. The resulting communicative dynamic allows for cooperation to unfold in the absence of consensus [3].

The problematic can be exemplified with respect to the concept of narration. In “Storytelling as a Means to Transfer Knowledge via Narration” [53] Wuttke et. al. describe design elements of a cognitive agent capable of narration. The developed conception is quite different from conceptions formulated within the field of humanities [43], yet it provides a boundary object facilitating coordination. Consequently, it can be brought into productive yet conflictual discourse regarding the relationship of narration and interactive artefacts [27].

4 Methodological Framework

In response to the challenges posed by developing at the intersection of complexity and complication, we propose an iterative development methodology. It is based on the observation that fundamental differences concerning disciplinary methodological commitments usually cannot be overcome, but can be rendered productive by establishing an ongoing process of conflictual negotiation [25].

4.1 Development Context

Methodological building blocks outlined were developed in the course of interdisciplinary development projects, tasked with development of intelligent agents [52, 54] and interactive installations for cultural education [24, 26, 27, 41]. They aim at facilitating prototyping activities through provision of shared vocabularies [7] as well as providing incentives on the level of design methods [6]. Discursive devices developed are grounded within a conceptual analysis of disciplinary differences regarding the notions of interaction [9] and materiality [28].

During cooperation with qualitative social research huge obstacles had to be overcome on the levels of language and concept building [25]. On a superficial level, cooperation between media psychology and computer-science is faced with much lower obstacles. Both disciplines share a common paradigm, being centered on information processing, rooted within the traditions of science. Consequently, none of the misunderstandings between incommensurable language games occur.

However, comparing cooperation with cognitive science also foregrounds some of the similarities between computer-science and qualitative social research which were not apparent before. Cooperating social researchers were using grounded-theory methodology, thereby employing an iterative methodology. Consequently, it was possible to align iterative development methodologies within computer-science [11, 13, 34, 48, 48] to those of social research.

4.2 Iterative Development Style

In order to facilitate artefact centric modes of cooperation and minimise the detrimental impact of misunderstandings, an iterative development style is adopted:

With respect to the interdisciplinary nature of developments it is important to adopt a shared practice of conducting iterations. While iterative modes of development are common within computer-science and HCI, they are less common within the domains of cognitive science and media psychology. Since extensive lab studies require a large amount of planning, supervision, and evaluation, proceeding within a large number of small iterations is not always an option.

Consequently, it is important to interleave the more waterfall-like structured aspects with practices of continuous iterative technology development. This entails acknowledging a difference in the mode of knowledge construction: while practice-based knowledge is constructed within an ongoing process of interpretation, Interaction-paradigm lab-study data arrives in larger chunks.

Irrespective of this acknowledgement, cognitive scientists play an integral continuous role throughout the practice-based design process. Their input is needed in order to continuously renegotiate the status of previously formulated theories. Since lab-based knowledge has to be situated within extensive processes of contextualisation and renegotiation [31], respective processes of situation proceed iteratively, even if hypothesis testing according to psychological protocol does not.

4.3 Switching Perspectives – Dialogue in Practice

Participants are encouraged to translate theoretical and empirical results into design. Computer-scientists are encouraged to design experiments and studies, psychologists are encouraged to specify technological prototypes.

4.4 Continuous Renegotiation

Drawing on Wakkary’s approach concerning complexity [51], we adopt a methodological pattern, inspired by what he likens to ‘dead reckoning’: Instead of seeking agreement concerning a fixed stipulated endpoint in the form of a system or artefact, we agree on a general direction within the project. Development proceeds in this agreed direction, while future iterations perform ‘course corrections’ in reference to the outcome of the antecedent iteration.

This approach is necessary when targetting complex everyday situations: As Wakkary argues [51], the specifics of relevant problem descriptions themselves cannot be known in advance.

Accordingly, the status of produced artefacts and respective epistemological claims within the project remain the subject of continuous discussion and renegotiation. Not only do methods employed and designs constructed to reach a certain goal change during the course of a development project, the goals itself are subject to evolution and renegotiation.

When employing an iterative process such as this, it becomes especially important to impose limits on goal flexibility in order for the process not to devolve into a random free-for-all but instead retain structure and momentum. On the other hand, the need of fixing goals in advance does not do justice to the complex renegotiations necessary when adressing the problematic of everyday complexity: The dynamic of emerging design situations cannot be known in advance, relevant relationships are only discovered iteratively.

4.5 Theoretical Framing

Theoretical framings possess a special status within the methodology: No common framing is aimed for, since the conflicting nature of respective disciplinary epistemological commitments and discourse practices have to be acknowledged. Instead, the methodology calls for individual readings of a shared theory, in order to provide a shared vocabulary for processes of conflict, negotiation, and development. All sides are familiar with a specific theory, though they need not subscribe to its claims within their own practice.

4.6 Process vs. Outcome

Even if the utility of lab-based studies is severely limited concerning the realm of everyday complexity, insights can still be wrested from participating in respective processes of complication. In order to create controlled environments conducible to hypothesis testing, psychologists have to perform requisite complications. They construct environments in which a large number of repeatable, homogeneous interactions can be observed.

Artefacts should be able to afford a wide range of diverse activities, not constricting users to repetitive or inflexible procedures. At the same time, digital artefacts exhibit a specific coded mechanics, limiting their flexibility. They always act as complicating material frames. Accordingly, knowing the feasible and infeasible pathways for complication is necessary in order to successfully design digital artefacts for complex situations.

In this sense, observing the failures of cognitve-science complications tells constructive computer scientists as much as learning from knowledge derived from successful experiments and studies.

The insight just formulated has to be read against the backdrop of its practical conditions: it is valid only in relationship to everyday complexity. Within research focussing on inherently complicated processes, such as issues relating to perception and physiology, the results of quantitative research again become more central.

5 Discussion and Conclusion

We have provided an analysis of the status of cognitive artefacts within everyday contexts. There are considerable challenges when employing cognitive technology within everyday contexts, especially in non-work, non-task oriented ways. The dynamicity of everyday contexts necessitates novel ways of interrelating cognitive science, computer-science and design practices.

The field of cognitive theory itself provides interesting impulses to this effect in the form of ecological psychology and situated cognition approaches. The unfolding discussion need not devolve into a culture clash between disciplines, opposing scientific cognitive psychology and hermeneutic/constructive interaction design techniques. Psychological theory itself provides theoretical frameworks that go well beyond the simplistic focus on task times and error rates that Carroll and Kellog warned us from in the 1980s [12, p. 13].

We have tried to establish a discursive framework, able to bridge the disciplinary gap between cognitive science and practice-based researchers. Drawing on the concept of complexity we have outlined a methodology facilitating interdisciplinary development of cognitive artefacts for everyday contexts. In doing so, we have been consciously operating outside the epistemological safety margins afforded by respective theories.