In the social sphere of the early twenty-first century, intersemiotic translation is enjoying something of a moment. Our digital age has spawned a revolution in social communications, providing new materials to help us fashion them. As first the arts, and now academia, begin to explore these new materialities, it is perhaps pertinent not only to revisit how we define ‘intersemiotic translation’, but to consider whether, where, and what its boundaries might be.

Defining Intersemiotic Translation

A shoal of terms swims loosely around any discussion of ‘intersemiotic translation’. Lars Elleström (2010) has offered a framework for finer definition of some of these terms. His taxonomy will serve to guide more considered use of terminology in this article.Footnote 1 ‘Media’, Elleström, argues “are both similar and different and one cannot compare media without clarifying which aspects are relevant to the comparison and exactly how these aspects are related to each other” (ibid.: 15). To translate intermedially must be to operate “a bridge between medial differences that is founded on medial similarities” (12). To understand these operations more closely, Elleström advances modalities as finer categories of media. These are “the essential cornerstones of all media without which mediality cannot be comprehended” (ibid.). He offers four categories; “material modality, the sensorial modality, the spatiotemporal modality and the semiotic modality”, “found on a scale ranging from the tangible to the perceptual and the conceptual” (ibid.: 15). To translate multimodally is to work across these categories, and multimodal work must necessarily be intermedial.

Elleström’s definition of the semiotic is founded on the work of pioneer Charles Sanders Peirce, and proposes “that convention (symbolic signs), resemblance (iconic signs) and contiguity (indexical signs) should be seen as the three main modes of the semioticmodality” (ibid.: 22). Intersemiotic translation, then, must surely involve recruiting the material, the sensorial, the spatiotemporal and the semiotic to effect transfers of meaning through new combinations of symbolic, iconic and indexical signs. The definition of intersemiotic translation offered by Campbell and Vidal in the introduction to this volume advances this definition in an interesting direction by suggesting that the translator “effectively plays the role of mediator in an experiential process that allows the recipient (viewer, listener, reader or participant) to re-create the sense (or semios) of the source artefact” for him or herself. As they write, “[t]his holistic approach recognizes that there are multiple possible versions of both source and target texts and this can help mitigate the biases and preconceptions a static, intralingual translation can sometimes introduce” (see Entangled Journeys in the present volume).

The Intersemiotic in Translating Sign Languages

Translators and interpreters working with sign languages rarely work intra-semiotically, and consequently have long been exploring multimodal practices. This is the very nature of their task. To work between written/spoken language and sign language is to work across material, sensorial, spatiotemporal and semioticmodalities, and to deconstruct and reconstruct meaning by receiving and producing combinations of very different symbolic, iconic and indexical signs. Yet rarely do sign language interpreters (at either theoretical or practical level) claim to be working intersemiotically, as mediators in a “holistic”, “experiential process” open to interactive re-construction by the recipient. My work seeks to problematize this discrepancy.

For this reason, this chapter offers a perspective that begins at the point of encounter between sign languages and alphabetized languages and draws examples from the more particular challenge of translating sign language poetry, or ‘Signart’ (Pollitt 2014). Here, the densely multimodal nature of the form increases the demand on translators to expand their semiotic range in order to achieve successful translation. A number of case studies will examine the different properties that can be brought to the fore by consciously harnessing the resources of various modalities and materials to intersemiotic translational practices, considering what new meanings become available, and exploring whether the translated products that emerge engage new audiences in different ways—thereby developing new social and cultural forms of communication.

To begin, it is necessary to gain some perspective on the historical and residual dominance of the word in our social communications and, in turn, how this dominance has shaped the field of Translation Studies to which sign language translation theory often makes reference.

Mapping Grammatology

Jacques Derrida’s (1967) descriptive work Of Grammatology remains the twentieth century’s seminal mirror on modern enslavement to words. His critique of logocentrism—“the metaphysics of phonetic writing (for example, of the alphabet)”—surveys a period in which spoken language is taken as the starting point of all studies of language and communication (1967: 3). It is an assumed orientation which much of the western academy has yet to consciously recognize. It is played out in descriptions of our individual biological development, and in our theologies. For those whose societies are informed by Abrahamic religions, “In the beginning was the Word” and that Word was spoken. According to the logocentric map, Derrida argues, the spoken word must be held as both interior and anterior to writing, and writing must be held as exterior and supplemental. Upholding these interior, anterior/exterior, supplemental co-ordinates creates “the old grid to which is given the task of outlining the domain of a science” (1967: 33). Derrida’s original grid might readily be populated with further dichotomies, such as representation versus presence, or image versus reality. In this way, we can begin to appreciate both how the grid serves to identify logocentrism as an ideological practice, embedded in the way we articulate our Weltanschauung, and how grid-like thinking underpins our epistemology.

Questioning the science of logocentrism, Derrida argued “the ‘original’, ‘natural’ etc. language had never existed, never been intact and untouched by writing, that it had itself always been a writing” (1967: 56). In place of the logocentric grid, Derrida posits a much more nebulous formation, which he calls Writing (sic) or arche-writing (1967: 55, 56). This expanded concept allows recognition of much broader compositional communicative practices that do not necessarily adhere to strict linguistic codes.

Derrida claimed logocentrism was “nothing but the most original and powerful ethnocentrism” (1967: 3). Yet the academy has been slow to embrace arche-writing. In a globalized age of commerce and communication, with its necessary awareness of diversity and its concomitant appetite for homogenization, such argumentation may seem extreme. But for half a century after Derrida’s publication, shifting away from the logocentric purview remained radical, requiring a revision of perspective as simple, profound and controversial as the replacement of a Mercator map with a Gall-Peter’s projection. In the late twentieth century, William J.T. Mitchell (1986: 157) described the “presentation of imagistic elements in texts, textual elements in images” as “a transgression, an act of (sometimes ritual) violence”. It was only with the turn of the millennium that scholars (Kress2003; Crow2006) began to take a more open approach to charting the cultural shift away from the logos of logocentrism. Perhaps the utmost irony lies in the fact that, in large part, it is the scientific linearity of binary code, with its strings, tables and sets, that has been the major impetus behind the radical shift away from logocentrism we have begun to witness in our digital age. For only with the widespread availability of smart technology, with the interconnectedness of the web, have we seen the rapid rise of popular arche-writing. From icons to emojis to the visual communications of Snapchat and Instagram, old literary and textual traditions are everywhere augmented by new communicative forms. By 2014, celebrated artist Xu Bing was publishing Book from the Ground, the result of a seven-year project bearing witness to the global return of the pictogram, which the author claims “expresses the ideal of a single, universally understood language, and my sense of the direction of contemporary communication” (Xu2014, artist statement, back cover). At last, it seems, we are all consciously Writing.

Sign Languages as Arche-Writing

For the deaf communities of the world, digital arche-writing may also represent communicative liberation, but for somewhat different reasons. Since the late 1970s, linguists of alterity (William Stokoe, Scott Liddell, Mary Brennan, Lars Wallin, Ursula Bellugi, Inger Ahlgren, Carol Padden, to name but a few pioneers) have revealed the natural sign languagesFootnote 2 of deaf communities as complex, often ancient languages. Although many sign languages borrow from the dominant languages that surround them—by incorporating alphabetic letter forms through “fingerspelling”, for example—they do so only to a limited and often highly inflected degree (Sutton-Spenceand Woll1999). Moreover, whilst natural sign languages boast the constituent parts familiar to human languages—syntax, semantics, pragmatics, phonetics, phonology, morphologyFootnote 3—they often achieve these in ways which differ radically from alphabetized languages.

Yet just as sign languages are not manual forms of spoken-written languages—not “words on the hand”—neither are they simply constructed of images. Sign languages tend towards iconicity because image is a material available to them, but that is not to say that they are crudely “visual”, and constructed of immediately accessible pictures. It is not surprising that a great deal of research has concentrated on the nature of iconicity in sign languages (for a small sample, see Brennan1992; Taub2001; Liddell2003; Emmorey2014). This article is not the place for examination of these complex understandings. Here we might just as readily adopt Elleström’s (2010: 16) definition of iconicity as “semiosis based on similarity (that only sometimes can be seen)”, providing we are able to accept that everything can be seen in a sign language but there are forms of visible iconicity which are less transparent than others.

However, perhaps the real radicalism of sign languages lies in their use of three-dimensional space. Here, Elleström (2010: 20) falls in line with the field of Sign Language Studies when he posits “[t]hinking in terms of spatiality is a fundamental trait of the human mind that has a significant effect on the way we perceive and describe media.” It is this spatial dimension of sign languages, not just their reliance on the human body as medium of expression, and the complexity of the relationship between image and lexeme, that distinguishes them amongst human languages.

With the emergence of the digital age and its turn from the logocentric, new possibilities and new threats have emerged. The ready availability of video recording equipment, the incorporation of image and live chat functions on social media platforms, and the presence of the internet as a universal conduit have effected a communication revolution for sign language communities that has been likened to that of the printing press (Rose1994; Krentz2006). It is now possible to send, receive and broadcast messages recorded in a sign language, and the presence of sign languages on internet fora such as YouTube has generated a wave of popular interest in and respect for these languages and those who use them.

There is real danger, however, that such interest also exoticizes sign language communities. We know that centuries of neglect and oppression have left sign languages vulnerable (Turner2006), and there is a risk that sheer volume of interest might overwhelm these fragile, minoritized, still threatened languages and their communities.Footnote 4As Cronin (2003: 141) suggests:

Minority languages that are under pressure from powerful major languages can succumb at lexical and syntactic levels so that over time they become mirror images of the dominant language. Through imitation, they lack the specificity that invites imitation. As a result of continuous translation, they can no longer be translated. There is nothing left to translate.

Offered the digital facilitation of increased exchange between national sign languages, and a degree of commerce with wider digital practices of arche-writing, the impetus to fortify the least readily translatable, iconic properties of sign languages may prove irresistible to sign language communities. This movement is already evident in the sphere of poetic activity, notably in the rise and increasing popularity of the so called “visual vernacular” poetic form (Pollitt 2011, 2017). This form utilises the grammatical spatiotemporality of sign languages but, unlike quotidian sign language, prioritises the visual over the linguistic, producing a form that robustly contradicts the intellectual taxonomies of scholars. Elleström (2010), for example, claims “there is no doubt about the basic semiotic differences between, for instance, a written text and a moving image” (23), and derides as misconception the notion “that media are always fundamentally blended in a hermaphroditical way” (12). Signart, however, is precisely a blend of Writing-through-image (Ulmer1985: 229). It is fascinating because it is always hermaphroditical and casts doubt on our ability to make such bold statements.

Gathering Resources from Intra-semiotic Translation

The historic relation of sign languages to alphabetized languages has been mirrored in the relationship between sign language interpreters and translators and their mainstream counterparts. For decades, the physical presence of the sign language interpreter was regarded as a vulgarity by those accustomed to the discretion of the interpreters’ booth (personal observation). The physicality of their practice, the lack of an orthographized literary canon, their reliance on digital technologies, and the sheer otherness of their language communities has marked sign language translators as outsiders within the wider profession. Yet this otherness has helped to raise, reinforce and expand questions of translational theory and practice (Turner and Pollitt 2002; Wurm2014). For example, the very nature of sign language translation denied its collusion in the myth of purely literal translation—a beguiling falsehood all too readily extrapolated from the promise of logocentrism. Once debunked (see, for example, Ortega y Gasset1937), translators of alphabetized languages were left with more questions than answers. If a translator is not a clear and impartial conduit, then what is the extent of their presence, and what effect does it have on the text? What can they reasonably claim as their purpose? What checks and balances might work to effectively regulate their power? Over the years, a number of theories have sprung up in answer to these existential dilemmas. They have served to “boundary” the profession by drawing out debate and sketching ethical and moral responses. Some have proved significant in the further development of understanding of sign language translation practices. Three works, in particular, may be pertinent here.

Hans J. Vermeer (1989) began developing his Skopostheorie in 1978. Essentially Vermeer seeks to recognize the freedom the translator requires to induce the response in receivers of the target text that has been experienced by audiences of the source text. Vermeer’s radical approach, then, recognizes the importance of culture in translation. For example, if target culture x has no tradition of limerick form, in a linguistically skilful translation of a language y limerick might remain culturally redundant. Instead, Vermeer urges translators to focus on the purpose (Skopos) of the source text—whether it is designed to make its audience laugh, cry, wonder—and recruit whatever cultural form is appropriate to achieving the same purpose in the target culture.

To Lawrence Venuti (2000), however, such an approach risks homogenizing the two texts. Receivers of the translated text will learn nothing of its original culture if it has already been modified to fit their own cultural expectations, he argues, and this may impact on their ability to understand the original resonance of the text. Venuti’s proposition offers an altogether different principle for acts of translation, arguing that they should seek to demonstrate rather than reconcile cultural difference. He calls this approach foreignizing.

Both Vermeerand Venuti recognize the importance of culture, but ascribe radically different functions to the act of translation. The work of Cecilia Wadensjö (1998) provides a qualitatively different counterpoint to this theoretical divide by highlighting the presence of the interpreter or translator in any translated exchange. Whilst her work was developed through close observation and analysis of interpreted exchanges, her conclusions are also applicable here. Wadensjö’s key contribution is to demonstrate the interpreter (translator) as an active third presence in the exchange, and to consider the interpreted text as a distinct third text, lying between both source and target text. This shifts ownership of translated texts considerably; reframing the translated text as a product of collaboration, and the translator visible and accountable for their acts.

Each of these theories finds relevance in the work of sign language translators. Vermeer’sSkopostheorie helps to reconcile the burden of working across two such distinct modalities. Wadensjö’s work helps reconcile the physical presence of the sign language translator, and the contrivances required of any text that bridges such linguistic and cultural distance. The application of Venuti’s proposition is less straightforward. Between deaf and hearing cultures there exists not only a minority-majority power divide, but the prejudice of disability. Here a foreignized translation risks being received not as a communication from an interestingly exotic other culture, but dismissed and patronized as the inadequate communication of an impaired version of the same culture. Subsumed into everyday multimodal sign language translation practices, particularly in the workplace or other mundane settings (Dickinson2010), foreignizing can tread a fine line between undermining and reinforcing existing prejudice. I hope, however, to demonstrate its value in intersemiotic translation practice.

Of course, neither Vermeer, Venuti, nor Wadensjö consciously seek to describe intersemiotic practice. In many respects, extending traditional theories to account for the arche-writing of intersemiotic translation leaves scholars facing the same challenges as those once faced by sign language interpreters and translators as they stood before the wider profession. As sign language translators have filtered and refined existing theories to more accurately reflect the realities of their practices, so drawing on a combination of Vermeer’s, Venuti’sand Wadensjö’s three theoretical paradigms may create a penumbra in which we can locate the general co-ordinates for intersemiotic translation practices. As Cronin (2003: 146) observes, translation theory should not be treated as “an esoteric luxury indulged in by the mandarins of major languages but as a crucial means to understanding the position of minority language speakers in relationships of language and power”.

Signart, Arche-Writing and Intersemiotic Translation

Surveying the mainstream field of poetry in France over the last twenty years, Nina Parish and Emma Wagstaff (2017) observe:

the definition of poetry is expanding to take account of its porous boundaries with other art forms, practices and genres, whose own boundaries and definitions are also challenged and expanded…the relationship between poetry and other creative practices produces new artistic forms rather than dialogue, and poetic objects emerge that are more than written texts inspired by other media. Poetry leaves behind the format of the written collection to extend into other domains…

The kind of “poetic objects” Parishand Wagstaff describe are precisely the kind that sign language poems have always been. Indeed, I prefer to describe sign language poetry as Signart because of its inherently multimodal nature (Pollitt 2014). My doctoral research (ibid.) identified six key constituents of successful Signart: linguistic flair; illumination; gesture-dance; the cinematic; compositional rhythm; and social sculpture. Lexically, sign languages are highly productive, allowing great freedom for morphemic elements such as handshape, movement, spatial location and orientation to combine, recombine and compound freely (Brennan1990, 1992). The creation of new combinations, and even of new lexemes is customarily included in the remit of the Signartist (Sutton-Spence2005: 7). My research also demonstrated that Signart magnifies and exploits the visual capacities of sign languages, often producing novel acts of drawing in space that are impossible to parse from the language they are serving to illustrate. The nearest equivalents in alphabetized languages offer weak points of comparison; the illumination of Medieval manuscripts, or perhaps creative play with the kanji, hieroglyphs or Chinese characters (Fenellosa1920).

A Signartist may produce innovative linguistic and illustrative work of the highest quality, yet remain unpopular if the physical movement of their body is jarring to the eye. This is also true if the spatial fluidity of the performance is poor. Renowned Signartist Peter Cook’s assertion (2002: 215) that “[a] good poem is like a good movie” adds yet a further layer of aesthetic expectation, echoed in testimony collected in my interviews with British Signartists and their audiences (Pollitt 2014). Meanwhile, analysis of four randomly selected pieces of British Signart revealed astonishingly rhythmic compositional patterning in the way images were repeated, positioned in the space around the Signartist’s body, and inflected.Footnote 5 Finally, all the Signartists I interviewed, and most audience members I asked, noted the importance of Signart as a vehicle for describing the community to itself, as a tool for liberation and political change, as a form created by a deaf sensibility for a deaf sensibility, as an expression of “deafhood” (Ladd2003). Using art to engage the creative potential of all individuals and effect incremental change in a society is what the artist Joseph Beuys termed “social sculpture” (1997).

As soon as one considers these six qualities, the challenge of translating Signart becomes clear. As a ready example, how might Signart’s use of cinematic technique and vocabulary—the pan, the freeze frame, slow motion, pulling focus, the reveal, play with scale, and so on—be rendered in terms familiar to consumers of poetry in spoken or written form? Yet satisfactory translation of Signart is surely possible. It is also necessary. When translation is the best means of communicating a politically marginalized minority’s culturally valued product, it behoves us to examine the impact of the translator’s various semiotic choices and decisions. An over-reliance on Skopostheorie, for example, would surely lead to an unacceptable reduction of the original text. Indeed, an exploration of this supposition formed the basis of my first experimental translation of Signart—a concrete translation of “Ocean” by Johanna Mesch, described in greater detail below. Wadensjö’s (1998) liberating focus on the act of translation creating a third text extends the scope of translational action, whilst Venuti’s (2000) offer of foreignizing might here be recruited to unashamedly promote Signart’s distinct form and robust political content.

If the combination of these frameworks can form a basis for intersemiotic practice, how far can such practice be taken without the translational act becoming so permissive as to eclipse the source text, to collapse the domestic into the foreign or the foreign into the domestic, or to be rendered mere resource for an artistic practice? Such questions could be annulled by the additional consideration of affordance.

Using Affordance to Explore Intersemiotic Translations of Signart

Gunther Kress (2003: 1) first proposed “modal affordance” as a variation on the basic concept made applicable to a broad, multimodal set of communicative acts. He defines modal affordance as “what it is possible to express and represent or communicate easily with the resources of a mode and what is less straightforward or even impossible.” His definition permits consideration of “the materially, culturally, socially and historically developed ways in which meaning is made with particular semiotic resources” (ibid.). Elleström (2010: 14) cites Kress’s work as the foundation upon which his own taxonomy seeks to build.

I suggest Kress’ concept may provide a useful frame within which to justify the activity of intersemiotic translation. Awareness of modal affordance can encourage the intersemiotic translator to more carefully consider the materialities of both source and target texts; to reflect on the sets of semiotic connotations pertaining to each particular modality; to discriminate according to the communicative properties they seek to amplify, substitute, or edit; and to trim their arche-writing practice according to the limitations that become apparent through these considerations.

To test this hypothesis I have, over the past five years, undertaken a number of small case studies in using modal affordance as a parameter in intersemiotic translations of British Signart. In each case, I have focused on a different affordance of Signart, and attempted to translate it through the modality and media most readily culturally, socially or historically associated with that affordance.

“Ocean” by Johanna Mesch

In 2011, the universities of the South West of England collectively established the Inside Arts Poetry Translation Prize, open to translations in “all the modern languages” taught in any of the universities.Footnote 6 The competition represented an opportunity to secure a higher profile for BSL, but also presented a challenge.

With page poetry as the target text, the act of transposing a visual, gestural, embodied performance into a linear, detached, two-dimensional orthography posed a risk to the integrity of the original; the divergence of modes and affordances of source and target texts compromising both Skopos and degree zero.Footnote 7

Johanna Mesch’s “Ocean”, unfolds from the perspective of the ocean personified, with the viewer positioned as witness. Ocean’s attitudes, moods, and responses to stimuli are given performatively, without reportage or commentary. The piece paints a portrait of a happy healthy environment, in which Ocean co-exists with the creatures that surround her, tolerant of the vessels that come into and out of her world, until a particular vessel drops a suspicious canister into her depths. As mysterious clouds swirl from the canister, Ocean becomes more and more unwell, her movement slowing until, eventually, it stops. The piece can be viewed using this search link: https://youtu.be/y7I3Fp8g-G4.

My translation (Fig. 9.1) sought to maintain the immediacy of Mesch’s performance through the strict use of the present tense, to reflect the embodiment of attributes through the choice of verbs, and to convey the rhythmic structure of the original through some complexity of ordering and repetition. The selection of verbs deliberately created ambiguity between active and passive, action and object, action and agent (“gloating”, “crusts”, “slick”, “bloating”, “sails”, “lap”, “rippling”, “shivers”, “still”) in an attempt to recreate a sense of the blurring of space and perspective between performer and observer. Lexical choices also invited the reader to imagine colour, texture and sensation (“slick”, “skirts”, “ric rac”, “petticoats”), in much the same way that an observer of Signart expects to populate the images created before them with their own sensations. Yet to capture the movement of the poem, or its directionality in space, a successful translation needed to co-opt broader semiological practices. Concretism, championed by poets such as e e cummings and Ian Hamilton Finlay, stretches the representational (grammatological) potential of written poetic form, and so provided a helpful semiotic resource with which to realize a fuller translation of Mesch’s piece. In the finished translation, continuity of movement, and the shapes and images created in space are re-imagined in the shaping of the stanzas, and the graduation of the font size. The rhythmic interspersion of diacritics is a further attempt to create an image on paper of waves radiating out from the centre (representing the location of the artist in signing space) to the reader (viewer).

Fig. 9.1
A translation diagram of the ocean. Text reads ocean, bloating gloating, shivers, and schools. Rippling brooding, watching, the opportunistic carrier. Brooding fretting and watching, etcetera.

A concrete translation of Johanna Mesch’s “Ocean” by Kyra Pollitt

Full size image

Whilst this translation met with some degree of success,Footnote 8 it provided only a partial key to the source text, and none of its kinaesthetic interactivity. For example, the scale, nature and detail of the images Mesch creates for each of the elements of the work (such as the relative size of the ship to the cylinder, the cylinder to the ocean) are not made equally explicit in written English. Such remainderingFootnote 9 is significant. At the very least, it suggests successful translation of Signart should engage with more intersemiotic practices.

The People of the Eye Collective

To test a broader range of intermedial possibilities, I was curious to discover what professional artists might find “possible to express and represent or communicate easily” of Signart within the resources of their particular artistic medium, and what they might find “less straightforward or even impossible” (Kress2003: 1): In 2012, I established a collective of twenty individuals, named “The People of the Eye”.Footnote 10 The collective included myself as researcher, four Signartists, and fifteen artists working with: performance; stone sculpture; small object sculpture; ceramics; jewellery; paint; ink, graphite, charcoal; textiles and screen printing; sound art; photography; and landscape-based digital media. All the artists were hearing, but their number included one fluent signer, and one person who had had basic exposure to British Sign Language. Each was given a recorded piece of Signart, with the option to access the Signartist’s introduction to the work (translated into written English). In all, eight members of the collective made responses; five non-signers and the fluent signer returned art works, whilst two further non-signers submitted diaries documenting their thoughts.Footnote 11

All the submissions confirmed aspects of translatability (Benjamin1923/2000) in Signart discernible to non-signers through intersemiotic practice. Sophia Lyndsay Burns’ work on Paul Scott’s “Three Queens”Footnote 12 demonstrated the modal compatibility between Signart and drawing (an example is given at Fig. 9.2); Tom White’s work on Donna Williams’ “That Day”Footnote 13 demonstrated its compatibility with film and the shared modality of gesture-dance (available to view here https://vimeo.com/62699375); whilst fluent signer Howard Hardiman’s work on the same sign poem (Fig. 9.3), and non-signer Fliss Watts’ work on John Wilson’s “Home”Footnote 14 (Fig. 9.4). illustrated the accessibility of Signart’s social sculpture.

Fig. 9.2
A drawing describes three people of the modal compatibility of Signart.

Non-signer Sophia Lyndsay Burns draws Paul Scott’s “Three Queens”

Full size image
Fig. 9.3
A drawing of a person and a set of eyes in different sizes. It describes whilst fluent signer on the same sign poem.

Note the gesture of Howard Hardiman’s deaf character as she defies the hearing gaze

Full size image
Fig. 9.4
A diagram describes the accessibility of Sign art's social sculpture.

Non-signer Fliss Watts perceives the deafhood in John Wilson’s “Home”

Full size image

Action/Assemblage: Drawing Together (a Happening)

This installation took place over two days in May 2013, as part of the Royal West of England Academy’s “Drawn” season. Situating Signart performance in the gallery harnessed the institution’s cultural, social and historical semios to foreground Signart’s overlooked acts of drawing (illumination). The public, joined by members of the research-through-drawing collective HATCH (hatch-drawing.org), were recruited as intersemiotic translators, responding graphically to live performances of Signart by Richard Carter and Paul Scott (Fig. 9.5).Footnote 15

Fig. 9.5
A photograph of live performances of Signart. There are people standing and sitting in different places.

The gallery of the RWA during Action/Assemblage: Drawing Together

Full size image

Within the limits of the materials provided (paper, plain and coloured pencils, erasers, pens, scissors) participants were given complete freedom to respond as they saw fit, and were invited to leave their work and any comments in a large cardboard box. No English introductions to, or translations of the Signart were given, but as the Signartists performed, I simultaneously wrote quotations on a large wall-mounted chalkboard. Examples included: “Art does not reproduce the visible; rather, it makes visible” (Klee2013: 7); “the line discloses an inner sound of artistic significance” (Kandinsky1919/1902: 425); and “to see is to have at a distance” (Merleau Ponty1969: 259).

Even though Signart is more visually motivated than everyday sign language, non-signers could not be expected to grasp its full meaning and they did not. However, many of the drawings went beyond representations of the Signartist in performance, operating as intersemiotic translations of segments of content, and readily demonstrating the shared affordances of Signart and graphic drawing (Figs. 9.6, 9.7 and 9.8). In this case, then, the shared modality of drawing confirmed an overlooked translatability in the sign language source text that could only be accessed and shared through intersemiotic translation.

Fig. 9.6
A graphic drawing of a woman with a significant artist in performance. It describes the different sizes and shapes of hands and birds.

The representational gave way to intersemiotic translation in many contributions to Action/Assemblage: Drawing Together

Full size image
Fig. 9.7
A graphic drawing of a man with translations of segments of content.

A member of the public chooses an appropriate semiotic form at Action/Assemblage: Drawing Together

Full size image
Fig. 9.8
A graphic drawing of a person with a crown.

Drawing is recruited to translate movement as well as image at Action/Assemblage: Drawing Together

Full size image

The Stars Are the Map

“The Stars Are the Map” is an intersemiotic translation using film-poetry to foreground the temporal-spatial modality of Signart. The piece was commissioned through a 2014 residency at the Scottish Poetry Library. It sought to bring Signart within reach of the Library’s poetry consumers, and to reinforce British Sign Language as one of Scotland’s indigenous languages. This latter was particularly important at that time, given growing momentum towards the eventual passage of the British Sign Language (Scotland) Act of September 2015.

The properties of film permitted simultaneous translation into two written languages, through the use of subtitling. Thus, Gary Quinn’s Signart is accompanied by parallel texts provided by Edinburgh Makar Christine De Luca. De Lucahas no fluency in BSL, and careful control of her access to content and nuances constrained her ability to re-present it. This forced the translation beyond the multimodal, towards the intersemiotic. In this way, we established “a relationship of the [three] texts in terms for which our western critical vocabulary does not offer an adequate equivalent” (Clüverand Watson1989: 56).

De Luca’s Shetlandic text occupied the traditional subtitling space at the foot of the screen (itself a recruitment of semiotic resource to political purpose), whilst the English text employed dynamic captioning to animate and illuminate the space around the Signartist on screen. This extended the capacity of alphabetized language to make apparent and to normalize the temporal-spatial affordance of Signart.

The film proved of great interest to its target audienceFootnote 16 and, unexpectedly, also to traditional audiences of Signart. It is available to view using this search link: youtu.be/wFWbnjylyAY. An audio recording of Christine De Luca’s Shetlandic text is also available at: bit.ly/saw44s-thestars.

movement.language.line.sign

One inescapable characteristic of Signart is that the poet is always in the poem. In 2016, an invitation to contribute to Tamarin Norwood’s residency at Bristol’s Spike Island provided the opportunity for intersemiotic play with this quality. In Norwood’s darkened, empty studio, a video recording of Paul Scott performing “Three Queens” was projected large-scale. The film was set on a loop on three projectors, each slightly out of synchronicity with the next. Since the background to the video was black, it merged and was lost on the shadowy walls, leaving three giant figures of Scott moving silently in the space. Without translation, visitors to the space were left to contemplate Scott’s ephemeralized physicality; the quality of movement, the gesture-dance, the poem in the poet.

This degree of abstraction could only be achieved with an audience of non-signers, for whom Scott’s movements were linguistically opaque, whilst obviously structured and meaningful. For this audience, it proved a successful translational strategy. A question and answer session at the end of the three days drew a large, curious and engaged audience, eager to discuss their experiences of the installation and, crucially, to explore Signart further (Fig. 9.9).

Fig. 9.9
A photograph of a woman conducting a question-and-answer session with the audience.

A sizeable audience is drawn to the Q and A session of movement.language.line.sign

Full size image

Airpoems in the Key of Voice

My doctoral thesis uncovered a hitherto unrecognized layer of poetic complexity in Signart—rhythmic composition. This is the pattern of visual motifs as they are repeated across the piece, carefully distributed in the space of the Signartist’s performance, and inflected (where key parameters of articulation of the image may be substantially altered yet the essence of the image is retained). Not immediately obvious, even to native receivers of these texts, this layer of composition nonetheless makes a significant contribution to the overall success of any piece. Experiments in mapping this phenomenon produced a series of charts that resembled medieval music transcripts (Fig. 9.10).

Fig. 9.10
A graph contains four rows and three columns. In the second row, the first column is 2, 3. The second column is 25, 1, and 3. The third column is 2, 5. The first and third columns flow with the second column.

An early example of analysis of the rhythmic composition of image in a Signart sample

Full size image

Given the rhythmic nature of this layer of composition, it seemed reasonable to translate this to a hearing audience through sound. In this way, the visual Skopos of the original could be reproduced in a form more familiar to the target audience. To set an English translation of the text to music, however, risked too great an homogenization, with the translation likely to both overpower and misrepresent the original. Instead, I began a collaboration with vocal artist Victoria Punch who developed a series of “vocal gestures” (Punch, personal communication, 2016) which she mapped to the transcripts I provided. In this way the intersemiotic translation maintained some ambition of foreignizing the target form. Learning these equivalences by rote, Punch was able to perform the vocalizations live, adapting in situ to the vagaries of Paul Scott’s live Signart performance.

Traditional translation of the content of Scott’s Signart was replaced by bespoke film-poetry projected on and over Paul’s body during performance. Neither of the film creators, Helen Dewbery and Chaucer Cameron, are familiar with sign language, so their work was informed by my carefully skeletal guide notes. Controlling their access to the content and nuances of the source text restricted their ability to re-present it, pushing the translation further towards the intersemiotic. The resulting film-poetry was grounded in Scott’s work, but more of a parallel filmic text than an attempt at rendition; “an equivalence of essence, the evocation of a reality not directly represented by the denotata of words or visual marks” (Clüverand Watson1989: 56).

The complete tripled-layered performance premièred at Ledbury Poetry Festival, 2017. The intersemiotically translated pieces were further framed and contextualized by a lecture on the nature of sign languages, and Signart as a form, interspersed between the performances. In a question and answer session, many audience members agreed on the efficacy of using music as a translation tool, particularly non-lyrical music made “of the body”. A short documentary of the work, and audience reactions to it, can be viewed at: https://vimeo.com/232071846.

Affordance as Intersemiotic Boundary Marker

On the evidence of these forays into intersemiotic translation of Signart, Kress’s (2003) notion of modal affordance is useful in directing and boundarying the translator’s activities. It is theoretically plausible, of course, that boundaries per se are grammatological constructs. Yet there exists a natural affordance in all modalities and media, and it seems reasonable to claim an equally natural tendency in intersemiotic acts of translation predicated on this. Considering “the materially, culturally, socially and historically developed ways in which meaning is made with particular semiotic resources” helps the translator refine their choice of resource, whilst understanding “what it is possible to express and represent or communicate easily with the resources of a mode and what is less straightforward or even impossible” directs the activity and success of each intersemiotic translation (ibid.: 1).

Broader compositional communicative practices, however, do not necessarily adhere to strict codes, as languages do. What we know of the ethical considerations of intra-semiotic translation begs the question of whether we are drawn to intersemiotic practice to escape moral obligations as much as restrictions of form: Does intersemiotic translation conjure potential democratization of translational practice, or a dangerously unregulated free-for-all? It is incumbent on the field to discuss and debate such considerations, and to try to develop transparencies of practice.

Depending on the cultural property being translated, consideration of affordance alone may be insufficient to safeguard the interests of those affected by any arche re-Writing. Certainly in working with sign language products, using affordance as the sole parameter would risk compounding the cultural harm historically visited on sign language communities. Yet where affordance as a parameter is scaffolded with applications of Skopostheorie, foreignizing, and explicit awareness of the translator as contributor to the creation of this third text, the resulting intersemiotic translation can achieve much more than traditional translation. The case studies presented here could only have been realized intersemiotically; this practice not only proving the multimodal nature of Signart, but opening hitherto unrecognized aspects of its translatability in new ways to new audiences. To reflect the new realities and superdiversities of our digital age: “Translation must be understood from a more flexible, heterogeneous and less static perspective, one that encompasses a broad set of empirical realities and acknowledges the ever-changing nature of practice” (Díaz-Cintasand Remael2007: 10).