1 Semiotic Approaches from Advertising to Post-advertising

The semiotics of advertising [1] and of marketing [2, 3] accompanied and monitored the developments and transformations of its object of study from the 1960s until today: semiotics followed the metamorphoses of advertising practices and discourses, starting from traditional forms (see Sect. 4), then passing through self-referential and post-modern ones [4], and finally arriving at to the use of advertising in digital and social media [5], including user-generated advertising [6, 7] and online/offline unconventional advertising [8].

The hypothesis behind this paper is that in recent years the world of advertising has been at the center of a break that is not yet entirely over. The main engine of this turn is the use of increasingly sophisticated and refined machine learning algorithms that automate both the brand’s communicative manifestations and the purchasing processes carried out by consumers. As a consequence, traditional forms of digital advertising tend to disappear, as a result of ad-blocking software products, for example. At the same time, however, corporate communication does not vanish: on the contrary, it becomes ubiquitous, deeply personalized, and radically relational. In other words, advertising loses the character of explicitly persuasive addresses to assume that of friendly and open proposals and advice, or even the simple facilitation of everyday purchasing practices. I propose to designate this incoming landscape with the term of “post-advertising” condition [9]. Starting from this hypothesis, I seek to understand if and in case with what revision the semiotic approaches developed for the analysis of traditional advertising can still be used for defining and analyzing post-advertising phenomena.

This paper is divided into three parts. In the first, I consider the transformations in the relationship between web media and advertising that have taken place over the last thirty years; I argue that digital communication is currently guided by the logic of algorithmic capitalism, of which post-advertising is an essential component and a typical manifestation. In the second part, I take as a case study the system of Voice User Interface Alexa, by Amazon, which I consider an example of a post-advertising dispositive; the semiotic analysis of two Amazon commercials focused on Alexa allows me to reconstruct the discursive identity of such dispositive. In the final part I ask more systematically if and how the semiotic tools that were used for the analysis of advertising (whether socio-semiotic or semio-pragmatic) can be applied to the study of post-advertising; in the light of my previous analysis my answer is affirmative, but on the condition that some new conceptual and methodological tools are introduced.

2 Algorithmic Capitalism and the Advent of Post-advertising

2.1 From Networked Capitalism to Sensor Capitalism

We can distinguish three major phases of the relationship between web media and digital advertising [10, 11]. In the first phase, which goes from the 1990s until about 2000, the web is considered a new space for the presence of advertising messages. Companies open their sites and buy banners, pop-ups, page takeovers, and so on – all means often considered intrusive. In some cases, companies perceive the potentialities of User Generated Content for the construction of a good reputation (for example in the case of fan pages of characters or media products or discussion in forums, chats and blogs), but the possibility of awkward reactions is always lurking – for example, that of Warner Bros against Harry Potter fans documented by Henry Jenkins [12]. In this phase (which roughly corresponds to Web 1.0) advertising remains quite traditional and tends to produce (both symbolic and economic) value by entering network spaces: we could thus speak here of “networked capitalism”.

The second phase of the relationship between web media and advertising spans from around 2000 to 2010. Many events transform the rules of the game: the advent of social media and the transition from “networked communication” to “platformed sociality” [13], the explosion and complexity of User Generated Content, and the spread of mobile media. Two main consequences emerge for web advertising. On the one hand, new forms of “wikinomics” are developed, tending to bypass corporate, institutional communication: the sharing economy and above all the spread of peer-to-peer counselling sites help to develop a suspicious attitude towards big companies and their intrusiveness. Companies respond to these trends in various ways, for example by experimenting with alternative forms of integrated communication such as branded content. In respect to these phenomena, some commentators speak about “post-capitalism” [14].

On the other hand, companies discover the usefulness of the traces left by users in their web browsing as evidence of their habits, tastes and preferences. Although we often talk about “big data”, the term is not entirely correct: the data has increased not only in terms of volume, but also of variety (it comes from different sources, in the form of both “captured” and “exhaust” data and metadata [15, 16]), velocity (the speed of acquisition and processing), veracity (the automatized assumption of data reliability) and value (the worth derived from exploiting data) [17]. In general, the term “sensors data” is preferable [18]. As a consequence, brands can better profile advertising proposals and switch from targeted to customized/tailored/personalized advertising [19, 20]. However, messages remain largely traditional, sent by e-mail or through “personalized” banners. Some speak of these phenomena in terms of “platform capitalism” [21], but I think that on the whole the most fitting term is that of “sensor-capitalism”.

2.2 From Data Capitalism to Algorithmic Capitalism

The third phase of the relationship between advertising and digital media is the one in which we find ourselves today, inaugurated at the beginning of the 2010s. The main feature of the transformations underway is the advent of a new generation of machine learning algorithms [22] within artificial intelligence studies [23,24,25], for example, those based on deep- or representation-learning neural networks [26]. These algorithms allow the machines to learn to identify patterns within data without having to write software containing predefined logical rules and instructions: data science distinguishes in this regard between supervised and unsupervised learning. There are different types of learning algorithms, inspired by different conceptions of learning processes (symbolism, connectionism, evolutionism, probabilism, analogism) [27]; all of them can in any case be described as devices able to transform a vast and disordered mass of data as input in a series of very complex but still organized and manageable models as output.

As a consequence, machine learning algorithms typically intervene in the data modelling phase (i.e. the extraction of non-obvious and useful patterns from data cubes); in practice, however, their usefulness manifests itself in all the steps of data flows. First, they proved to be very powerful in the transformation of unstructured, low-density, low-value, big data coming from sensors (so-called “raw data” [28]) into structured, high-density and high-value data, for example in machine vision, natural-language processing, and so on (see also Sect. 3.1). In other terms, these algorithms dynamically implement data cubes by constructing and transforming analytics base tables in real time. Second, they work the data inside the data cubes, transforming them into models that allow advancing reliable predictions; for example, through operations of clustering (which allows market segmentation and advertising customization), or through association-rule mining processes (which allow to identify groups of products typically purchased together, and then to advance purchasing advice). Finally, machine learning algorithms intervene in the output phase within the various forms with which the interfaces return information to users, from data visualization to interactive voice response (see Sect. 3.1).

Thanks to machine learning algorithms, interactions with data come to serve the small and big needs of every day: what information is relevant to my research (Google), which partner to choose for the evening (Tinder), which television series to watch (Netflix), which book or which detergent to buy (Amazon), and so on. Moreover, they do it through a series of “naturalized” practices that consider machines as an integral part of living and working environments. In this way, the algorithms become “culture machines [i.e.] complex assemblages of abstractions, processes, and people” [29]. In this case, the extreme importance assumed by algorithms in the processes of production and manipulation of values leads me to speak of “algorithmic capitalism”.

In this context, traditional banners and ads continue to operate in an increasingly personalized form, thanks to the algorithms of Google Ads, Facebook Ads, Instagram Ads, etc. At the same time, however, traditional advertisements tend to be less visible – due to the spread of ad-blocking software, whose availability and use have grown enormously in recent years (passing in the U.S. from a 15,7% penetration rate in 2014 to an estimated 27,5% in 2020 [30]). Conversely, the dynamics of mutual and disintermediate advice, previously delegated to peer relationships, are now primarily assumed by the algorithms themselves: these, appropriately constructed and trained, become the main agents of the guidelines for purchasing goods, services, products. In this way, we return to the phenomena of post-advertising from which we started; in fact, I consider post-advertising as the most typical form assumed by corporate communication within algorithmic capitalism.

3 The Marvelous Mrs. Alexa

3.1 Alexa as a Post-advertising Dispositive

In 2014, Amazon introduced a line of smart speakers (the Echo series), that offered the possibility of interacting with a digital assistant named Alexa. Alexa can give information in real time (for example on current weather, or traffic situation), play music from various platforms, make phone calls, activate or control home automation appliances, or place orders for goods or services. In this section, I propose to consider Alexa as a typical “post-advertising dispositive”; the analysis of some aspects of Alexa will then allow us to test semiotic tools on post-advertising objects and phenomena.

Alexa is a digital assistant. This category of appliances is the most recent evolution of Vocal User Interfaces (VUI), i.e. tools for interacting with a machine that does not use the traditional Graphic User Interface (GUI) means but instead relies on oral interaction. Typically, a VUI has three components. The first is a vocal sensor connected to human speech recognition (HSR), automatic speech recognition (ASR) and natural language understanding (NLU) software [31]; these components capture the human voice and translate it into sentences that the machine can interpret correctly. The second component is a mechanism for digging for information within a data set, which can present an output corresponding with the input request. The third component is an interface that provides the user with the required information through an Interactive Voice Response (IVR): in recent years research has provided synthetic voices with greater fluidity and emotional coloring, also linked to their gender [32]. A typical VUI application is a chatbot that, faced with a limited set of possible user requests, offers a series of answers chosen from a limited array (for example, in telephone travel booking systems, or complaint services). In some cases, a VUI device can be embedded within a home automation appliance, becoming part of the so-called Internet of Things: in this case, in addition to responding vocally, it can activate and control the appliance (switching on or off an oven, adjusting the home temperature, and so on).

Digital assistants like Alexa derive from the application of machine learning algorithms to VUI devices, a phenomenon typical of algorithmic capitalism. Artificial intelligence has particularly affected the first and second components of VUIs. First, the introduction of machine learning algorithms has significantly improved HSR, ASR, and NLU: they have made it possible to move ever faster from the “raw data” of natural speech to organized data that can be understood and managed by the machine, without having to depend longer on sets of prefixed statements. Secondly, the new machine learning algorithms make it possible to search for the answers requested by the subjects within the universe of big data: queries are no longer limited to prefixed data-sets but can range over any subject, including updates in real time.

Today, we can find four main players in the VUI field: Apple Siri (introduced in 2010), Google Now (introduced in 2013 and replaced in 2016 by Google Assistant), Microsoft Cortana (introduced in 2013), and Amazon Alexa (introduced with Echo speakers in 2014, as noted above). Each of these assistants has specific characteristics related to the company that developed them. In this context “Alexa is your almost perfect shopping assistant, at least for now. Integrating Amazon Prime for shopping, videos, and now, music has given users a straightforward choice to buy into the Amazon ecosystem.” [33: p. 10]. In other words, the Alexa system is designed to act as an elicitor or facilitator of practices of purchasing: it is a “shopping medium”. Unlike its competitors, Amazon has directly aimed at building a v-commerce (voice-commerce) system; this sector is currently limited (data shows 0.4% of sales as e-commerce in the US for 2018) but destined to expand [34].

It is worth underscoring that at the moment Amazon explicitly denies any intention to introduce advertising messages into Alexa’s interactions; traditional advertising coming from Amazon is therefore explicitly banned. Rather, users are “naturally” advised about brands and products best suited to meet their specific needs, with the tendency of enhancing the routine nature of certain consumer behaviors and thereby intensifying brand loyalty. This process takes place through a two-part mechanism. First, brands can develop “skills”, i.e. applications intended to implement Alexa’s capabilities, that users can activate; for example the Starbucks Reorder Skill allows you to “reorder your usual from one of the last 10 stores you have ordered from before […] Your Starbucks order will be minutes away when you say ‘Alexa, tell Starbucks to start my usual order.’” [35]. Second, the integration of Alexa with some home automation devices (such as refrigerators, printers, and so on) makes it possible to automatically reorder the products that are about to run out from Amazon [36].

From what we have said, we can draw three conclusions. First, Alexa appears to be not only a “voice” but rather an “apparatus,” i.e. “[something] that has in some way the capacity to capture, orient, determine, intercept, model, control, or secure the gestures, behaviors, opinions, or discourses of living beings” [37: p. 14]); or even better a “dispositive,” i.e. an assemblage of technological components, use practices embedded in wider social activities, objects and subjects’ roles, spatial and temporal determinations, plans, intentions, and desires [38]. Second, from this point of view it is possible to grasp a specific strategic component of Alexa as dispositive: Amazon rejects by policy any recourse to traditional advertising, in order to configure Alexa as a direct and natural consumer elicitor and facilitator. For this reason, I consider Alexa (more than other digital assistants) as a typical post-advertising dispositive. Finally, it must be recognized that as a dispositive Alexa is not just a set of technologies, but has a specific “discursive” identity, i.e. an identity oriented and determined at various levels by social discourses that take it as their object. In order to now explore this identity more thoroughly, I will consider in particular two commercials for Amazon’s Alexa broadcast during Super Bowls LII (2018) and LIII (2019).

3.2 A Momentary Lapse of Alexa

Broadcast for the first time during Super Bowl LII, “Alexa Loses Her Voice”, a video advertisement lasting 1′30″, was the most viewed commercial on Youtube in 2018, with 50.1 million views [39]. The commercial recounts in a comedic tone the catastrophic event in which the voice assistant for Amazon loses her voice because of a sudden cold. All that remains is to find many stand-ins, including a star-studded cast (from the chef Gordon Ramsay to the singer Cardi B, from the actress Rebel Wilson to Sir Anthony Hopkins) that tries, and obviously fails, to replace Alexa. Finally, we hear Alexa saying “Thanks, guys, but I will take it from here”; at the same time, the graphic element of the smile that is part of the Amazon logo is stretched out on the image.

The narrative structure of the commercial is extremely traditional: an initial loss is balanced by a final recovery of the missing element [40], in this case the voice of Alexa. Moreover, as usually happens in the advertising narrative, the missing element becomes an object of desire and therefore of valorization as a consequence of the dysphoric perception of its absence. In other words, the commercial works as a real and effective test of commutation [41], aimed at assessing how and in what measure Alexa’s voice is significant in everyday life practices. The result of this procedure is twofold. On the one hand, Alexa and the devices that convey her presence (the various models of Echo Dot), appear flawlessly and fluidly inserted into the physical environments and the life practices of the subjects who use them: the technological component of the dispositive is thoroughly naturalized within the unreflective actions of everyday life. On the other hand, Alexa appears to be a competent, relevant, and non-intrusive presence: for example, her plain and natural “grain of the voice” [42] positively contrasts with the aggressive (Ramsey), shrill and mocking (Cardi B), sensual (Wilson), or subtly threatening (Hopkins) ones of her substitutes. To sum up, Alexa’s absence ends up highlighting the qualities of her presence, and above all the fact that she constitutes a perfect form of presence for her user.

Moreover, this narrative structure binds to a specific enunciation regime [43]: the enunciator (identifiable with Amazon) proposes to the enunciatee (the spectator) a communicative contract [44] founded on irony, according to a typical strategy of post-modern advertising [45]. Thus, we find a gap between the reality status of the enunciated story and the relationships at the level of enunciation and narrative discourse [46]: Amazon as enunciator invites the spectator as enunciatee not to take seriously the literal meaning of the story, but only its moral; the presence at the end of the commercial (coinciding with the return of the “true” Alexa) of the Amazon “smile” underlines this enunciative choice. This strategy has an important implication: some sensitive points pertaining to the reputation of Amazon and Alexa can be at the same time declared at the level of story and denied (or, in psychoanalytical terms, “disavowed” [47]) at the level of narrative discourse. I refer in particular to two aspects: the charge against Amazon of promoting a gig economy, for example through the platform The Mechanical Turk (seen in the ad concept, in which big stars are rented to do gigs), and the fear that through the Echo Dots Amazon can watch what happens in the intimacy of the houses (the face of Anthony Hopkins threateningly appearing to the woman who tries to call her husband).

3.3 Alexa Unbound

“We’re putting [Alexa] in a lot of stuff now”; unfortunately, however, “there are [still] a lot of fails.” The leaks of an Amazon Alexa developer to a stunned colleague in the company’s cafeteria are at the center of “Not Everything Makes the Cut,” the 1′30″ Amazon advertisement released during the 2019 Super Bowl. The commercial recounts a series of disastrous performances by funny and unlikely devices controlled by Alexa: from Forest Whitaker’s toothbrush to Abbi Jacobson and Ilana Glazer’s Alexa hot tub (actually, an oversized version of the Echo dot); from an Hal 9000 style interface used by twin astronauts Kelly and Kelly in a spaceship, to Harrison Ford’s dog’s Alexa collar.

On the one hand, the story presents a situation opposite to that of the previous commercial: while the 2018 advertisement focused on Alexa’s absence, the 2019 one plays on her “excessive” presence. Moreover, while in 2018 Alexa was mainly represented in her interactional aspects, the new commercial insists on her operational aspects: the “new” Alexa is not simply a presence able to inform, entertain, play music, or make phone calls through Amazon devices, but has been “embedded” within various kinds of appliances. In this way, the commercial reflects (in paradoxical terms) the new actual Amazon policy, which is tightening various agreements with domestic appliance manufacturers to connect Alexa to the Internet of Things.

On the other hand, the enunciation strategy is similar to that of the previous commercial: the Amazon employee turns out to be an “unreliable narrator” [48], as her story is implicitly denied by the enunciator, relying on a relationship of complicity with the enunciatee who is presumed to be able to correctly interpret what is being shown. The conclusion of the commercial is once again revealing. Harrison Ford looks downcast at the arrival of the huge quantity of gravy and sausages bought by his dog thanks to the Alexa collar, and growls at the animal “I’m not talking to you.” In this way, Amazon as an enunciator points out its own “disengagement” (débrayage) [44] from the narrator’s discourse (the phrase can be interpreted as “it’s no [longer] me, Amazon, who is talking to you, the spectator, by referring the discourse of the (unreliable) narrator”), and then a new direct engagement (embrayage) of the enunciator expressed by the euphoric tones of the Queen song in the background (“Do not stop me now, I’m having a good time”) and the smile of the Amazon logo, which is simultaneously an Amazon signature and an interpretive indication for the viewer.

Furthermore, the enunciation strategy based on irony once more allows the ad to acknowledge and at the same time deny a series of anxieties related to the use of Alexa. It is interesting to note that among the different aspects, the fear that Alexa encourages automatic purchasing practices occupies a prominent place: it is no coincidence that the storyline that stars Harrison Ford and his dog, represented as a kind of compulsive buyer, returns several times in the commercial, and represents the closing gag of the story.

3.4 Alexa’s Discursive Identity

The analysis of these two Amazon commercials allows us to draw some conclusions about the discursive identity of Alexa as a post-advertising dispositive. First, the discursive construction of Alexa’s identity is based on a semantic and axiological universe in which different areas are blended and hybridized. For example, the recreational-aesthetic values and the practical-utilitarian ones are hardly distinguishable in Alexa’s presentations and valorizations: indeed, making a phone call, listening to music, making a purchase, or operating an appliance are similar activities in which cognition, emotion, movement and action are equally co-present.

Second, and consequently, Alexa is presented and tends to be experienced as a dispositive that is perfectly integrated into the network of everyday practices and operations: it is not an object, but rather a form of living presence with which subjects can interact, reflexively assessing at the same time the quality of their presence in living environments. Amazon does not sell technology, but a way of “being in the world” and interacting with the world, while it seeks with the irony of its commercials to defuse the possible anxieties linked to such a condition.

4 Towards a Post-advertising Semiotics

In these conclusions, I take a broader perspective, and ask more generally about the possibility of using the semiotic tools developed for the analysis of traditional advertising in the analysis of post-advertising phenomena. In doing so, I will bear in mind the analysis just carried out of Alexa’s discursive identity and its construction.

It is worth recalling the two founding essays at the origin of advertising semiotics. The first one is the well-known work of Roland Barthes, initially published in 1964 and dedicated to the advertisement of Pasta Panzani [49]. Barthes’ essay intends to “deconstruct” the advertising image to identify the system of cultural connotations underlying it (for example, its “Italianicity”). At the same time, his semiotic analysis reveals the cultural value of the medium used: in particular, photography is “a message without a code,” at the service of an ideology of transparency and immediacy. Barthes’ essay thus initiates a tradition of studies that analyzes advertising messages as “signification systems,” i.e. “Set[s] of meanings that are generated for a systematic association of various signifiers (brand name, logo, ad texts, etc.) with implicit signifieds relating to personality, lifestyle, desires, etc.” [50: p. 23]. Scholars often identified the logic governing these universes in the opposition between simple terms linked by relationships of contrariety, contradiction, or complementarity, as happens in Greimas’s semiotic square [44, 51], applied to advertising messages by Floch [52]. Such an approach has proved to be very productive for the reconstruction of the large, mutually articulated, and culturally-based semantic universes determining advertising strategy and in turn determined by them [53, 54]. This approach can be traced back to the strand of structural semiotics and more specifically to the current that would take the name of socio-semiotics.

The second founding essay of advertising semiotics in the analysis of printed advertisements was presented by Umberto Eco in La struttura assente in 1977 [55]. Despite the explicit reference to Barthes’s essay, Eco’s goal is different: he tries to understand which rhetorical and semiotic mechanisms the texts use to design and govern an interpretative path for their reader, and how these mechanisms can function persuasively. Eco’s hypothesis is that the more an advertisement re-proposes a series of consolidated topoi in apparently new forms, the more it appears reliable to the reader and therefore achieves persuasion. In other terms, the mapping of the semantic fields proper to socio-semiotics is integrated with the analysis of the strategic uses of the same semantic fields within the interpretative paths of the discourse: this approach can be labeled semio-pragmatic [56, 57], and is typical of interpretive semiotics.

Here we come to the question that has guided this intervention: is it possible to apply this system of thought and analysis developed by the semiotics of advertising to post-advertising phenomena, such as the Alexa dispositive I analyzed above? My answer is positive, but on the condition that some modifications and developments be added to traditional semiotic instruments. I will limit myself to some brief indications.

First, the socio-semiotic approach should revise the mappings of signification systems intended as the detection of binary oppositions; indeed, we have seen that the domains of signifiers, signified, and values typical of the post-advertising condition are characterized by extreme fragmentation, hybridization and fluidity. For example, the analysis of Alexa’s discursive identity showed how the opposition between human subjects and technological objects loses its strength, as does the opposition between playful and practical areas of everyday life. It would probably be more useful rethink signification systems in terms of different “modes of existence” [58, 59] and “modes of presence” [60] through which subjects think and engage with different types of human or artificial agents.

Second, the semio-pragmatic approach should overcome a conception of user activity as purely cognitive and linked to the interpretation of “texts.” It should instead deal with the design of the subjects’ experiences in their different and related dimensions (sensibility, cognition, emotion, movement and action, and so on), even drawing inspiration from the new models of experience emerging from neurocognitive sciences [61, 62]. At the same time, semio-pragmatics should realize that the experiences thus planned are deployed (and can therefore be identified and analyzed) within complex dispositives and environments that extensively mix texts, objects, spaces, actions, and interactions.