1 Introduction

The creation and production processes of audiovisual content, including video in mobile applications and on demand systems, are rapidly changing with technological convergence, bringing together previously distinct areas, such as Human Computer Interaction (HCI) and Audience (or Media) Studies [1,2,3]. This integration demands a theoretical relation that is not currently contemplated by these fields individually [4, 5].

The communicational and methodological model of Audiovisual Design (AD) brings together relevant methods and concepts from those two normally separate disciplines, but with similar profiles. In short, AD has two practical purposes to attain. First, the model provides instruments to analyze audiovisual productions developed for interactive systems, using tools afforded by both HCI and Media Studies fields. Second, the theory provides methods to predict interactions and to propose innovative audiovisual systems, suitable to be used by different profiles described in the model. The main characteristic of this model is planning the interaction, or interactivity, in the process of audio, video and software production. AD describes four roles to be assumed by individuals and four Lines of Design, to guide and to model the creative development process [4, 5].

The design of interactive computer systems had its initial focus on problems, tasks and functions, but it gradually changed its scope and also incorporated perspective of new possibilities, meanings and emotions [6,7,8]. Nowadays, individual’s behavior is a central part in design projects. Moments of passive fruition, for example, gain relevance when the final object of an interactive system contains audiovisual content. For example, we can mention video on demand systems, accessed both in smart TV sets and mobile devices, with search resources and evaluation tools [5].

A similar phenomenon can be observed from the point of view of audiovisual consumption. The experience using software has become as relevant as the quality of movies, series, songs or online videos. Different digital systems to access video content have one feature in common: interaction using software-based interfaces. Electronic programming guides of digital TV, search tools for specific titles in movie applications, or recommendation tools of videos on social media, are examples in which the viewer’s experience mixes an active posture (browsing or searching for information) with moments of passive fruition or enjoyment (video watching). The simple act of choosing and watching an audiovisual show may require the individual to perform different roles, with degrees of greater or lesser interaction or participation. Consequently, a revision of theories and methods supporting the development of these contents becomes relevant, especially in the field of HCI, in which the general notion of user has a limited role. This concept does not contemplate total immersion in the different media, especially those based on audiovisual content. It is necessary, therefore, to understand that this mediatized process incorporates content, technique, aesthetics and meaning, aspects relevant to the complete processes of communication and fruition by individuals [2, 9, 10].

To adequately act in response to this new scenario relative to production and fruition of audiovisual content, where audios, videos and software are integrated into a single product, AD defines four roles: the Audience, the Synthesizer, the Modifier, and the Producer [4, 5]. Each role has elements that permit individuals to extrapolate common behaviors, then becoming the enhanced role ‘Player’. In this article, we present and classify the competencies related to each of these roles that an individual (sometimes identified as user, interactor or spectator) can assume. In the model, individuals need to have certain competencies, represented as a set of knowledge, skills, capabilities and behaviors, to be able to act within each role.

Mastering new competencies is a key element to modifying behaviors in certain roles [11,12,13]. It is a theoretical and conceptual proposal that contributes to the scaling of AD’s reach and predictability, a discussion that allows both researchers to evaluate media interactions and producers to design environments and meanings of interaction.

2 Communication Models

Aiming to describe communicative acts and the flow of information between people and technologies involved in communication, the process can be organized in theoretical and conceptual models. In the communication field, different models have been suggested since the beginning of Media Studies, including both transmission and reception technologies, as well as content and meaning of the message [14]. Recently, Jenkins, Ford and Green [2] proposed three simple models to describe the different scenarios of current communication ecosystem: one sender for multiple receivers, as Broadcast; online communication, in which individuals take initiative to search content, as Stickiness; and Spreadable, when content reaches the audience through the action of individuals, usually using digital tools. Although different, in practice there is some complementarity among those models. Often content shared within the propagation logic was produced by television stations (Broadcast), released in a controlled way (Stickiness) and is shared in an uncontrolled way by action of individuals (Spreadable). Digital interfaces, of which Stickiness and Spreadable models depend, have been occupying the space of traditional media and are becoming, themselves, new media elements.

In this way, audiovisual content producers need to incorporate interaction tools as essential elements inherent to production process, which changes the experience of audiovisual fruition. While producers maintain some control over the product, they encourage, at the same time, some audience engagement.

Of course, since many people will only enjoy content at a basic level, it would be a mistake if engagement tools decreased the quality of Broadcast content to passive audiences. Moreover, the interface design should consider not anonymous users, but individuals who, at some point, will use the content as a reference to compose their identity in social networks.

Despite the relevance and appropriateness of those models in Media Studies field, they do not adequately describe the interaction of the individual, nor do they provide tools to analyze, or to create, audiovisual products that depend on software to be watched. Jenkins, Ford and Green [2] describe the communicational processes in information exchange and different forms of media consumption. However, the authors do not contemplate technical-creative process of audiovisual production, where subjectivity of audiovisual narrative interrelates with objectivity of interaction demands. Interface problems, or poor user experience, can compromise the entire audiovisual product.

One example can be represented by affordances of interaction interfaces. Affordance refers to possibilities of interaction and use that the system offers or allows for users [8]. This idea, although longstanding in HCI, does not exist in audiovisual production process, including theoretical, conceptual or methodological descriptions in Media Studies. Unlike audiovisual fruition, traditionally considered as a simple process, with no need for any technological domain [14], the development of software depends on potential actions intrinsic to the perception that individuals have about an object or technology, when interacting with the system [5].

2.1 The Audiovisual Design Model

From this scenario, previous studies have identified the need for a theoretical and methodological model that integrates both software development and audiovisual creation processes. As an outcome of the intersections between disciplines of Media Studies and HCI, Audiovisual Design is represented by a graphic model that allows understanding the dynamic flow of audiovisual production, including software and considering different scenarios and roles played by individuals [4].

The horizontal arrows (Fig. 1) indicate the starting points for understanding interaction scenarios. While Media Studies allow inferring behaviors from people in front of audiovisual devices, HCI offers consistent theories and methods for the producer, or software developer, to design interactions and uses. In this way, mechanisms such as user surveys, formative evaluations of interfaces and usability tests help to identify the requirements, demands and needs of the product. In this model, the same individual can play different roles at different times: Audience, Synthesizer, Modifier and Producer. Every role has levels of fruition, with minimum and maximum activities supported in each one. When the individual extrapolates characteristics of each role, but without changing the position, an enhanced role emerges, characterized as Player-Audience, Player-Synthesizer, Player-Modifier and Player-Producer. For analytic purposes, in this article the Player is analyzed separately, to facilitate identification of competencies necessary to act in this enhanced role.

Fig. 1.
figure 1

(Source: Audiovisual Design Research Group).

Flow diagram of the process in the Audiovisual Design (AD)

Full size image

Audience: this is the basic role an individual can assume, denoting low level or absence of interactions in media consumption. This is the passive behavior associated with the Broadcast model [2, 14], whose relationship with digital interfaces are channel tuning, video search and playback, channel subscription, etc. In this way, these people are understood together through audience ratings, which give only a collective view of preferences. The relationship of individuals with content is through personal choices, and remains relatively private.

Synthesizer: the concept was brought in by Jenkins, Ford and Green from an idea developed by Bradley Horowitz [2]. Individuals have skills to compile, to classify, to comment, to recommend and to share audiovisual products they like, often creating an “identity”, that is, a profile staged in a network with other people. The role of the Synthesizer involves notion of engagement, or a deep emotional connection that allows individuals to express something about they from the content they share.

Modifier: this role is a subset of Synthesizers, with other competencies and skills: they know how to use software to manipulate and recreate contents with which they identify, going beyond the idea of engagement towards the idea of appropriation. Some examples of Modifier activities are: to improve content, when individuals appropriate content to enhance something they do not like; to remix, when individuals create new productions from the original content; to participate, when individuals send messages and interact with television shows, mainly live, providing information useful for directors and producers. In many cases, participants’ messages are screened during the shows.

Producer: one person or a group of people who creates original product, even if inspired by content already existing in other media, autonomously and independently or as part of large media corporations. Although every Producer is a Synthesizer (the distribution of content is part of his competencies), the Modifier’s competencies are applied only in situations where an industrially planned workpiece is an adaptation of an existing one, so the overlap between these roles is only partial.

Player: Refers to the individual who fully uses the interaction tools available within each level, being an ‘enhancement’ within each role. They seek content that will bring challenges and make them, even in isolation, think and do something. This role is identified with “early adopters” or “early users”, i.e. people who take the risks of using a new technology and, thus, contribute to its development. As examples, we mention a Producer creating new distribution channels (Player-Producer); a Modifier creating a new product based on different media, such as a cartoon from images other people have taken (Player-Modifier); a Synthesizer acting as a fan and searching posts in many blogs, sites or social media to share them with their own social media contacts (Player-Synthesizer); an individual who looks for easter eggs or other hidden content in off-line media, such as DVD or Blu-ray discs, and engages in its content (Player-Audience).

The relationships inherent in these roles occur through four Lines of Design: (a) Content, with relationships that vary (and add up) according to the role played by the individual: subjective relations at Audience level; engagement in Synthesizer level; appropriation in Modifier level; creation for Producer; and challenges for Player; (b) Identity, which bases the personal relationship with content; (c) Motivation, which leads to engagement; and (d) Experience, which links fruition to advanced use of interactive features, which should provide practical experience with content and interfaces.

3 Competencies Related to the Roles

As we can see in the AD workflow description, the way designers or producers direct their production to different user levels, and how they correlate to each other, depends on articulation of contents, techniques and interfaces that accompany goals defined by the individuals. These objectives correspond directly to the way individuals engage with media (privately or publicly, passively or actively, as spectator or co-producer). To do this, individuals need to have a “competency set” appropriate to technological affordances available for audiovisual product and its interfaces.

In a very general first approach, the main affordances of contemporary media ecosystem are: navigation, search and retrieval possibilities; interactivity; adaptability of content; possibility of being appropriable (both tools and content); openness to new forms of distribution (such as sharing); to be a facilitator for data mining about usage. This list can be expanded inexhaustibly with specific items, since new affordances can emerge at each technological and/or social convergence process.

The competencies, as we understand them here, is a term borrowed from the field of Business Administration and People Management. It refers mainly to behavioral repertoire and intentions demonstrated by an individual to efficiently perform a given task [11, 12]. Nevertheless, this notion is not limited to the workplace, since people must dedicate intentions, actions and behaviors in all interactions in which they have intention to engage. A competency, or the ability to perform a task competently, involves knowledge, skills and capabilities of individuals, ordered from their behaviors and their intentions [11, 15].

We identify the subset of individual’s characteristics that allow them to perform an intentional operation as skills. These skills are acquired and developed through the individual’s intentions to act or react to a situation, thus shaping their behavior during performance in an engaged manner. Capabilities, on the other hand, are potential skills, present at an unconscious level and not yet developed. Capabilities are inherent in the individual’s learning process, intermediate stage between acquisition of knowledge and putting it into operation.

Considering new communication affordances created within contemporary media ecosystem, it is natural for individuals to demonstrate new competencies, or new sets of skills, knowledge, capabilities and behaviors, or even modify those already ingrained both psychologically and culturally. We present in this paper a panoramic description of competencies related to each user role within AD, however, some observations are necessary.

Firstly, the elements that belong to specific competencies may have socioeconomic, cultural, technical or technological characteristics. Within AD, we consider them related to: physical-financial conditions to access the content; fruition, appropriation and cultural use of messages transmitted in a communicative process; the way this fruition, appropriation, and use occur; and knowledge of technologies necessary to apply them to a particular use. We do not offer a description of socioeconomic or cultural characteristics in this article; we believe that this requires a greater degree of specificity, which must necessarily be related to each context (geographic, political and economic).

Secondly, a competency only becomes effective when skills, knowledge, and behaviors activate the affordances from available communication technology. Thus, it is not just a theoretical appropriation or notion of the operation that ensures a person has actually acquired the skills necessary to be classified within an AD role. To do so, the individual must apply the respective competencies in the process of media fruition. Only then it is possible to state, in fact, the individual has moved in between roles. This proviso is necessary because we reinforce that there is an opening for progression or regression of each individual in between roles, which is always confirmed by the Line of Identity: the same person can use all tools at his disposal to engage more fully with a product or a workpiece, which touches him deeply, and can be a passive spectator of another, with which it relates only for the satisfaction of entertainment.

A third observation refers to actions the Producer can take to enable acquisition of knowledge and skills by individuals, which may sustain behaviors that apply the necessary skills to change from one role to another. It is an intrinsic part of designing complex interactive audiovisual pieces. Affordances need to be adaptable and learnable by target audience, in an intuitive and harmonious way with general narrative universe. In other words, if the Producer wants (controllable) groups of Synthesizers and Modifiers in a participatory and collaborative logic, it is very likely that they will have to enable people to learn how to develop those roles.

3.1 Competencies Framework

In order to facilitate the understanding of how competencies are applied to different roles, we present a list divided by role individuals can act in the AD model. We emphasize that not always will all skills be necessary for a person to be identified acting within a role. It depends on production characteristics and how individuals interact. However, at least some of the characteristics should be present. In addition, since the logic of Audiovisual Design workflow is based on the notion of “sets”, where one group is contained in another, competencies accumulate from one level to another.

This description is not exhaustive of the competencies of each role, since they may vary according to proposed work. The items described initially come from profiles and skills traditionally identified by HCI and Media Studies from authors such as: [1, 6, 10, 16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. A detailed review was conducted looking for competencies necessary to act minimally in each role, and how to extrapolate this actuation, both to change among roles and to become an enhanced role, as analyzed in Player’s competencies description. With the combination of these elements proposed by AD, common attributes were established to individuals and explained according to their level of interaction. What we intend, with this framework, is to open a new discussion about elements currently involved in design of complex audiovisual products. We recognize that these observations are, at least, fairly generic to describe all possible processes.

As summarized in Table 1, the first level of roles of individual clearly requires the simplest and most easily appropriable competencies. For the viewer who only watch audiovisual content in Broadcast model, cultural and linguistic competencies for understanding and interpretation of the message are basically enough. However, when we think about the increasing number of individuals using alternative fruition forms to the Broadcast, but still for passive consumption - such as legal or illegal download, on demand, fragments, and so on - skills accumulate.

Table 1. Competencies per individual role in Audiovisual Design
Full size table

Knowing the functioning and limitations of internet itself is still a limiting factor in many countries, which is being rapidly overcome as we observe the growing number of population with access to computer and to network connections. After surpassed this stage, the competencies listed here make possible two central phenomena: individual’s identification with the produced content (theme, characters, narrative, ambiences, etc.), and feedback to the Producer, especially in quantitative formats (qualitative feedback is possible when the correct tools are available).

In the second role, (Synthesizer), the feedback becomes more qualitative. The Producer should observe more closely the ways in which people have engaged with their product. This analysis can direct choices for workpiece in production. Among these competencies, we emphasized those related to social media. In an attempt to create their network identities, Synthesizers are part of communities of mutual interests and act as representations of themselves, especially designed for particular situations. Thus, they become poles within social networks articulated by digital media. These competencies also make evident one’s condition as a fan, whose discussions revolve around their active participation in available content activities.

Regarding the role of the Modifier, it should be noted that appropriation of technology is fundamental, especially when Producers do not make available tools for modification. Despite that, the appropriation of content that occurs at this level is a cultural appropriation of some parts or of the whole workpiece or technology. This appropriation includes generating some meaning, sense or a different discourse from that originally manifested by the Producer. This process can be observed, for example, when an element in an audiovisual piece (a song, an emblematic scene in a film, a character in a fictional narrative) becomes both the link of a dispersed group of people and a symbol of a cause or a fight.

The Modifier gets elements from original context, modifies them, or recreates parts to transform them in a representative idea, or even in an ideology. Such means of expression represent a fundamental source of information for the Producer who, knowing how to read these signs, can correct future works or adapt ongoing demands. From a sociological point of view, it can represent a very aggressive capacity for expression, especially when the group appropriating content is a minority that uses a popular resource to make itself heard. Anyway, the Modifier is often able to create his own social network, becoming an important node of content diffusion.

The activities of the Players should be widely explored by Producers interested in knowing fruition possibilities of audiovisual productions. Players represent, however, a small part of total audience. They are best placed in the projects elaborated within the logic of achievements and rewards.

Finally, the Producers’ competencies are more complex, divided mainly between the field of technology and the ability to interpret the users’ demands. It remains to be pointed out that, except for a few authorial or amateur projects, an audiovisual production usually involves a group of people because even the director usually is not able to control all steps. In this way, when we refer to the Producer we are talking about each person who works at a project, with their own competencies within the area of expertise. This is common in large commercial products (e.g. writers, system developers, editors, graphic designers). Even if a person cannot master all technologies involved in a production, it is important to be aware that the workflow incorporates these technologies. User interfaces depend on a sense of continuity of actions, and sudden ruptures caused by the simple articulation of independent products can compromise the whole fruition process.

The success of this implementation depends, above all, on the incorporation into the system of responses to individuals’ necessities arisen from the set of observed uses, considering content characteristics and fruition process. Data to be read by Producers can be generated after the distribution of audiovisual product; in other words, after the audience has watched or heard the content, and used the interaction tools. This is the audience data and feedbacks mentioned above. In this case, Producers’ actions are limited to future works or, in the case of long-running productions, to slow transformations as the narrative unfolds. Furthermore, it is possible to use the technique of personas creation, with some previous data and with the development of scenarios, to predict some reactions. In this case, it is of great value to read the data obtained from the application of digital tools. Access and understand the data is an important tool (although not fundamental) for the Producer.

4 Conclusion

This article described core competencies needed by an individual to occupy one of the roles indicated in the Audiovisual Design model. The competences listed here may not be the only ones in certain contexts of fruition of an audiovisual workpiece. They are, however, ample descriptions that correspond to a significant part of situations which require an action or reaction by user. Each competency involves knowledge about the object of interaction, capacities (potentialities) for use of these objects, skills to engage in a particular action and behaviors in face of situations that arise during audiovisual consumption.

These competencies are relevant to the contemporary media ecosystem, which, when merging different means in the same environment, dominated by digital technologies, give rise to new affordances related to use of interfaces and computers. In this sense, the HCI field contributes to analyze situations where media consumption occurs, pointing out what these new affordances are and how audiovisual production should be organized. Hence those affordances can be incorporated in content and interface planning of interaction, as a prediction of actions taken by individuals. In other words, HCI’s problem-solving chain allows a complex use of different platforms and media by indicating potential properties within each audiovisual content distribution channel.

Such confluence of two distinct fields (Media Studies and HCI) demonstrates the need to review school core curriculum, especially in the field of audiovisual production, which today does not contemplate important topics for the activities of content producers for interactive systems.

Although the methodology of Audiovisual Design incorporates technologies available today, we must keep our mind open for assimilation of new technological or social dynamics that interfere in the proposed flow. We also understand that financial limitations may prevent the application of this methodology in its most complete version: on the production side, by the costs that may be high depending on the type of resources it proposes to use; by the audience, by the imbalance in the access to the Internet and digital tools that allow interaction at the levels foreseen here.

However, Audiovisual Design remains an appropriate methodological set, allowing to ponder over each phase of planning as an isolated process, but connected through the Lines of Design. In addition, it adds questions relating to training professionals who will produce content since the relations among system, content and individual are becoming increasingly close. Finally, it is worthwhile to emphasize the application of AD as a theoretical tool for evaluation and understanding of media usage scenarios. AD methodology encourages discussion about concrete aspects of audio and video systems use and theoretical aspects such as layers of identity, perception of value and sense people attribute during the enjoyment of the content.