Keywords

1 Introduction

Books and associated individuals’ practices have been widely studied over the past centuries in diverse research communities. There is thorough research on what people read and the effects it has on them (e.g., [40]), how they read (e.g., [30]), how they learn to read (e.g., [44]), how books are created and crafted (e.g., [35]), etc. In the realm of Human-Computer Interaction (HCI), it is of specific interest how people interact with interactive instantiations of digital books (e-books), and how they are accessed. E-book readers and screen reading have been investigated to understand what practices are associated with these forms of reading (e.g., [18]), and to create artifacts and interactions that facilitate such practices (e.g., [52]).

In this paper, we pursue the question of what “reading a book” may mean for individuals when different (im-)material instantiations of books are available to access them. We position this research within the scientific discourse around materiality in HCI and Interaction Design. We do so to emphareize the experiential consequences of different material instantiations of the same content. We complement existing work that focuses on the differences of digital and physical books (mainly e-books as opposed to printed books, e.g., [19]) by including a further instantiation, namely audiobooks, and by investigating people’s debates (on Twitter) about these different formats. These debates were triggered by asking whether or not listening to audiobooks or reading electronically count as reading; though we do not aim to find an answer of what counts as reading, the respective discussions are highly informative to better understand how the materiality of books influences individuals’ experiences, perceptions, and opinions.

Although HCI might primarily be interested in the future of digital books, we also include non-digital books into our research for three reasons. First, reading physical books or listening to audiobooks allows us to extrapolate individuals’ preferences and practices around reading that may be of value for future interactive books as well. Second, all formats of books are interactive in their specific ways; researching how individuals interact with them (whether they are physical or digital, material or immaterial) may reveal issues and potentials for future interactions. And finally, there is a reciprocal interaction between the evolution of different formats, e.g., not only do physical books inspire the design of ebook readers – ebook readers also influence the advancement of physical books. For instance, Dresang explored non-linear reading [13] to create books with characteristics of digital media; the publisher Dutton recently released mini books, which are cell phone-sized books with pages “as thin as onion skin”. They can be read with one hand, the text flows horizontally and one can flip pages upward, “like swiping a smartphone”, “hoping that younger readers from a generation that grew up with the internet and smartphones might be receptive to the concept of a miniature flipbook.” [1] Furthermore, individuals’ skills and preferences may change, showing the increasing entanglement between physical and digital reading and reading practices that provide challenges for HCI and Interaction Design:

This is going to sound incredibly lazy, like someone who gets in their car to drive a few blocks rather than walk, but the physicality of the book, having to hold it open then lift and turn each page, was a lot more exhausting than I remembered. All of that holding and lifting and turning distracted me from the act of reading, took me out of the story if you will. A few pages into it I gave up, logged in to Amazon, and bought the Kindle book. [46]

In order to inspire discussions around individuals’ (materialized) reading experiences, we engage with qualitative material by analyzing two independent Twitter threads that address this topic, being interested in the range of things people would associate with “reading”, i.e., different experiences, meanings, opinions, or concerns. The respective research question for the study at hand was, consequently, what “reading a book” may mean for individuals when different (im-)material instantiations of books are available. After summarizing the related work, the study and its finding in the following, we provide a discussion and depict implications for (research on) the design of future books.

2 Related Work

The work at hand is positioned within HCI’s and Interaction Design’s scientific discourse around materiality in the design and usage of interactive objects, and, in particular, it is focusing on reading as an interactive practice. Consequently, what follows is a summary of related work on materiality and reading. Of specific interest is related work that addresses the intersection of (interactions with) physical and digital books, which is mainly to be found in discourses around tangible interactions and interaction design for children.

2.1 Materiality

Design materials and materiality have received increasing attention in HCI and Interaction Design in the previous years (e.g., [5, 16, 47]). Materiality may be considered “a holistic paradigm of feeling and sensing the material, which allows also subtle peculiarities of materials to be recognized and used in design” [16, p. 2858]. Thus, the way how interactions are materialized is decisive for designing and experiencing them.

In regards to the design of interactive artifacts, design materials can be understood in various ways; material things, such as leather [43] or concrete [45], immaterial things, such as light [50] or radio [5], as well as digital materials, such as code [20], play a crucial role, as they only together result in interactive things. Consequently, Interaction Design may be understood as the practice of imagining and designing interaction through material manifestations [48]. Doing so requires material sensitivity, “i.e. an ability to carefully consider how different materials could be brought together in the design of an interactive system.” [48, p. 203] An emphasis on materials, considered as critical in design processes is not new. In his 1992 article about “designing as a reflective conversation with the materials of a design situation”, Donald Schön accentuated the influential role of the material [38].

When it comes to experiencing different materialities in interactive objects, it is increasingly recognized that the social world is heavily characterized by material aspects. “Sociomateriality”, the term used to emphasize this, refers to social practices that shape the materiality of a technology as well as its effect; materiality is present in all phenomena that are considered social [27]. In the realm of HCI, the important role of materials in humans’ experience with (interactive) systems has been evidenced often times. For instance, in an earlier work, we investigated workers’ experiences in an industrial setting through an ethnographic study [15]. Thereby, we found that depending on whether electronic or physical materials were interacted with, experiences weere different in such a context. This results in consequences for Interaction Design in this domain, such as taking into account subtle qualities that (invisibly) influence workers’ routines. Overall, new materials and combinations of analogue and digital materials are promising to “enable new experiences of computational power” [47].

In order to further the understanding and articulation of situated experiences of materials that eventually supports design, Giaccardi and Karana [17] defined a framework of materials experience. This framework is of particular relevance for the work at hand, since it is based on Interaction Design and HCI’s emerging practice-orientation and, consequently, proposes a dynamic relationship between people, materials, and practices. Materials are characterized by properties (e.g., computational properties) and embodiments through which properties can be experienced and performed (e.g., physical or temporal form). People are characterized by different competences and values; practices are situated ways of doing. Furthermore, they suggest to distinguish four experiential levels that materials are experienced on: The sensorial level defines how materials’ properties impact human senses; the interpretive level denotes how individuals interpret and judge materials; the affective level is characterized by emotions that result in affective dispositions towards the embodiment of a material; the performative level indicates humans’ performances that they establish around material objects, which are influenced by perceptions, meanings, and affects.

The unfolding of performances into unique and peculiar ways of doing, and their assimilation into practices, are both mediated and affected by the material character of such performances. [17, p. 2451]

In this understanding, materials may not just have a functional or aesthetic role (i.e., sensorial, interpretive, affective level), but also be active in unfolding or transforming practice (i.e., performative level), “when performances are rooted in the unique properties of the material” [17, p. 2454].

Given this notion that materials are important, if not decisive, for how practices develop and transform, it can be assumed that the same is true for how books are materialized – humans might develop practices for reading that are inherently shaped by the format of the book, the interaction that it enables, and the constraints it involves. In the following, we provide an overview of existing research on reading from an HCI and Interaction Design viewpoint and how it already incorporates questions about the book’s materiality and individuals’ material experiences, and we depict the gaps that are still to be filled within the state of the art.

2.2 Reading Experience

In HCI and Interaction design, different aspects of reading have been investigated in the past years, many focusing on the intersections between the physical and the digital world. For instance, Wu et al. [51] suggested a system that would allow readers to access digital content through fingertip interactions on physical books. Similarly, Bailey et al. [4] designed a way to turn digital pages through actually turning physical pages. Burstyn [8] presented the gbook, a two-page e-book reader, which recognizes page-flipping motions for page navigation. Yoshino et al. [53] created a UI system for public spaces that mimics look, feel, and usability of traditional books; Pillias et al. go even further back in history and revamp the antique scroll by exploring digital rolls as reading surfaces [31]. Kim et al. implemented a paper-book metaphor for flipping pages on a tablet [23].

Explorations of how physical materials can coincide with computation in order “to construct devices that look, feel, and function very differently from the boxes [the usual form of computational devices] we have become accustomed to” [32, p. 127] are also increasingly described in relation to books, such as Qi and Buechley’s exploration of computationally-enhanced pop-up books [32], or Zhao et al.’s design of a “paper-book based interactive reading system” that supports information revisiting processes through a combination of page flipping as in a “real” book complemented with electronic functions such as keyword searching [54]. Dunser and colleagues investigated a combination of paper and on-screen elements as instructional media for children, finding that the tangible input devices triggered physical interaction “too well”, when children transferred physical world affordances to 3D elements [14].

Others focused on improving reading on screens, such as how to optimize skim reading [26], how students would make use of e-book readers [41], whether dual-display e-book readers would be beneficial, for instance, in case of multi-document interactions [9], or how to best interact with materially flexible e-book readers [49]. Yoo et al. [52] experimented with humans’ interactions with “living” books by capturing the readers’ head shake as a sign of not understanding the words and providing vocabulary hints, or capturing when the reader comes closer, interpreted as curiosity, leading to display additional text for the reader. Raffle and colleagues created an e-book as a shared interaction for grandparents and their grandchildren in long-distance relationships, which resulted in remarkably longer sessions in comparison to typical video chats [33]. Nonetheless, there are also concerns about the effects of different material manifestations of books. For instance, Revelle and Bowman found that when children interacted with e-books, even if these had limited interactive features, there was less parent-child conversation about the book content than if they had interacted with paper books [34].

Another strand of research focuses on how to provide reading experiences for people with impairments, such as visual impairments. For instance, Thomas and Rufus investigated parameters of laterotactile Braille reading [42]; Stangl et al. explored how to improve the technical and human processes required for 3D printed textile picture books as a means to support parents and teachers of children with visual impairments, who have emergent literacy needs [39]. However, haptics supplementing narratives are not only favourable for people with impairments, but also recommended, for instance, for children’s e-books [10].

Though rarely in form of audiobooks, sound in regards to books has also been discussed in HCI and Interaction Design. For instance, Colombo and Landoni investigated how children would experience e-books and found that read-aloud narrations in enhanced e-books had a particularly positive effect on children’s experience [11]. The engaging effect of sound was also evidenced by Knoche et al., whose findings indicated that such interactive elements would trigger longer spontaneous utterances in children [24]. Another way to consider sound in books has been suggested by Back et al. in form a “personal interactive reading experience that combines the look and feel of a real book – a beautiful cover, paper pages and printed images and text – with the rich, evocative quality of a movie soundtrack” [3, p. 202], which was designed “with the idea that form affects meaning, and in fact is inextricable from it” [2, p. 28].

As this overview of related works shows, findings are available on various levels, from reading practices with different forms of books to experiments with novel forms of materialized reading. However, there are several research gaps that are yet to be filled. While there is HCI and Interaction Design research on children and their perception of (e-)books, we lack a substantive discussion of reading experiences in other readers beyond experiments with novel interaction formats. Recent research starts to contribute to this discussion, such as studies on the affordances of physical and digital books and how individuals value and use them over time [19]. It focuses on how people used and felt towards them as “objects”: people display and re-use books differently than e-books, e.g., the visibility of physical books reinforces ownership with physical books in contrast to e-books [19].

Furthermore, Hupfeld et al. have, by means of diary studies, intensively investigated people’s practices with e-books, finding, among others, that people would choose printed books if the books mattered to them while digital books were rather seen as transitory. Further, e-books were not considered to replace physical books [22]. Additionally, Hupfeld et al. interviewed people about their temporal, spatial, material, and social practices with books, demonstrating that books are more than reading devices, such as placing books around them for comfort or augmenting them to create traces. While e-readers seem to be valued primarily as reading devices, individuals’ relationships with physical books seemed intensely emotional – a finding that may impact the design of e-reading technologies [21].

While our work is inspired by the aforementioned related work, it is driven by a question that extends beyond the opposition of physical books and e-books in that it asks how individuals experience, have opinions and concerns, or create meaning with different forms of materialized reading, including physical books, e-books and e-book readers, audiobooks, etc. It aims to reveal reading qualities rather than to focus on specific aspects by trying to understand the experienced commonalities and differences of materialized reading.

The timeliness of this research is furthermore evidenced by the Stavanger Declaration on the Future of Reading [12], which has been released in January 2019. This document, an outcome of the European research initiative E-READFootnote 1, was published by over a hundred scientists who had researched the impact of digitization on reading practices (understood as how well individuals’ comprehend or remember written text when using print versus digital materials) over a period of four years. Among others, they found that

Digital text offers excellent opportunities to tailor text presentation to an individual’s preferences and needs. Benefits for comprehension and motivation have been demonstrated where the digital reading environment was carefully designed with the reader in mind; [...] Our embodied cognition (i.e. that how and what we learn, know, and can do depends on features of the entire physical body) may contribute to differences between reading on paper and on screen in terms of comprehension and retention. This factor is underestimated by readers, educators and even researchers. [12, p. 1f]

One of their concluding recommendations is directed to educators, reading experts, psychologists, and technologists, who “should partner to develop digital tools (and related software) that incorporate insights from research about the processing of digital and printed formats, including the role of embodied cognition, for reading practices” [12, p. 2]. The research at hand aims to contribute through inquiring into materialized reading practices that translate into design recommendations.

3 Method

The research goal of the study was to shed light on individuals’ experiences with and reflections on different forms of accessing books in order to inspire discussions around the design of future interactive books. The respective research question for the study at hand was, consequently, what “reading a book” may mean for individuals when different (im-)material instantiations of books are available.

One of the researchers involved came across a Twitter thread (see thread 1 below), a thread that perfectly matched the aforementioned research interest as it asked people to respond to a poll about whether or not listening to audiobooks would count as reading. This thread contained a reference to another similar one, with a similar number of statements, but with an additional focus on e-book readers (see thread 2 below). The material was considered relevant, as it was assumed that individuals would describe their perspectives, without us as researchers intervening.

3.1 Data

The first Twitter thread was following a poll of an account of a radio show (a show focusing on arts, literature, film, etc.), asking “If you’ve listened to an audiobook, can you say you’ve read the book?” The poll resulted in 69% saying “Yes, absolutely”, while the other 31 % said “No, it’s not the same”. However, it was not this quantitative data that we were interested in, but the reactions from individuals to this poll. The thread consisted of 331 individual tweets, whereof nine were excluded from the analysis due to their irrelevant content (being an argument of two individuals on their respective attitudes).

The second Twitter thread was referred to in the first one by a book blogger. This thread was started by the blogger’s tweet, stating “Audiobooks count as reading and ereaders are as good as paperbacks”. This thread consisted of 329 individual tweets, whereof five were excluded from the analysis since they were either incomprehensible or without any content (such as references to people without any statement).

One limitation with this material is based on the formulation of the starting statements of both threads. Certainly, they impact how people answer, how they debate and contend, and what they focus on. While the first thread opposes books to audiobooks, the second one equals e-book readers and physical books, and both are phrased as a provocation in order to (successfully) trigger reactions by playing with the word “reading”. This needs to be kept in mind when interpreting the subsequent findings, especially when it comes to discussions of semantics.

Another limitation is that we do not know much about the socio-cultural backgrounds of the people contributing to the two Twitter threads. From their Twitter profiles, we know that most of them indicated Europe or Northern America, and a few Australia, Africa, or Asia as their location. About half of the contributors indicated their profession, with a majority of professions related to books (writers, book bloggers, editors, etc.) and arts (artists, musicians, etc.). Further professions ranged from retired nurses and soldiers to scientists, students, journalists, etc. Many profiles revealed interests in books or reading (along coffee, dogs, or politics), which is not surprising given the topic of the Twitter threads. We also know that they are English speaking. However, the availability and inferability of this socio-demographic data is limited. Consequently, we refrain from relating them to specific findings. We also need to acknowledge that the contributors seem to share many characteristics (such as a passion for reading and books), meaning that the findings are specific to this group of individuals and cannot be generalized beyond the sample.

3.2 Ethical Considerations

Certainly, there are specific conditions associated with research that relies on data extracted from Twitter in regards to ethical considerations. Therefore, we consulted the “Ethics Guidelines for Internet-mediated Research” [7], wherein a main aspect of whether to use such data freely is the difference between public and private data. Public maybe “perhaps best thought of as ‘readily accessible by anyone’ ” [7, p. 7], which we consider applicable to the two aforementioned Twitter threads because (a) they can be considered public, since they were collected from a public radio show’s thread that was the basis for a subsequent public radio show, and a book blogger’s thread that was referred to in the radio show’s thread; (b) it can be assumed that individuals posting to these threads were aware that their contributions are public. In cases of sensitive information (e.g., about disabilities) we do not use any quotes that would draw attention to a particular person.

We also refrain from backtracking contributors (e.g., regarding socio demographic characteristics), since we do not want to invade someone’s privacy. While this limits the interpretability of the findings, as they cannot be socio-culturally contextualized, they are nonetheless relevant for our research as we are interested in the qualitative range of opinions and experiences, rather than in quantitative, generalizable correlations or predictions. We also decided not to publish the names of and links to the two threads to retain anonymity as far as possible.

3.3 Data Collection and Analysis

We collected the research material by copying and pasting it from Twitter to an Microsoft Excel spreadsheet instead of collecting it via an API given the small number of tweets and the possibility to prepare the data at the same time (e.g., examine appropriateness). We (two HCI researchers) then familiarized ourselves with the material and excluded what was irrelevant. We engaged with the qualitative, subjective material in order to inform and inspire discussions around the materiality of reading. Methodologically, we followed thematic analysis [6] by going through the material several times to identify themes. After merging some identified themes as they occurred to be highly related, five themes remained that we eventually labelled as “meanings”, “norms and values”, “inter-individual needs and preferences”, “the reader and the book”, and “situatedness”. In the following, we describe the themes in detail and add quotes to complement the abstracted material with individual thoughts. We refrain from adding any (individual or socio-cultural) information to these quotes for the aforementioned reasons. As mentioned earlier, we did not quantify the results as we are aiming to unfold qualitative issues around reading experiences; furthermore, given the non-representative nature of our data, any quantification would potentially be misleading.

4 Results

4.1 Meanings of Materialized reading

Given the trigger questions of both threads, it is not surprising that much of the discussion centered around terminology. Whether “reading” is to be understood literally or not was one major debate, with some people arguing that the meaning of the word may change or evolve alongside developments of different materializations of books, while others persisted on a narrow meaning that would include written representation only, i.e., visual perception would be essential for reading a book. However, there were further notions in between those contrasting two that included or excluded different sensory perceptions, such as reading via Braille being fine, but listening not. Furthermore, the boundaries were discussed if listening would count as reading, such as whether plays or films would be “valid” forms of books as well – with many people agreeing that, if text is adapted or cut, it is not reading any more.

People suggested a variety of alternative terms to overcome the opposition of “reading” and “listening”. Many alternative terms concern the act of reading or listening, such as “getting the stories”, “going through a book”, “consuming an author’s ideas”, “accessing a story”, “completing a book”, “consuming words”, “absorbing” or “processing information”, etc. Some of them already entail experiential allusions, such as “encountering stories”, “engaging with a narrative”, “learning information”, etc. Others discussed what it means to “have read a book”, i.e., after having completed reading or listening: knowing a book, appreciating it, comprehending it: I know how well I listened. So I know if I read* or not. (thread 1)

Similarly: If you ask me what book I read last, you want to know if I consumed the content not how I did so, so I’m going to say I finished Anthony Bourdain’s “Kitchen Confidential” yesterday. If you ask me the format of the book though, I’ll say audiobook and Bourdain was the narrator. (thread 2)

While there were several mentions that the activity of “consuming” would need to be characterized literally (listen as opposed to read), it seemed to be fine to talk about having read the book after completion, independent from the material manifestation. Overall, audiobooks and e-books were mentioned to nicely complement reading physical books; sometimes, even consuming the same book in different formats was considered to have an added value.

4.2 Norms and Values of Materialized reading

The question of whether or not audiobooks and e-readers would count as reading resulted in many normative statements (concerning mainly what is better, what is proper reading) as well as to discussions about the benefits and downsides of each format. Among others, people were considered “snobbery” if they disregarded alternative manifestations to physical books as reading. Some referred to audiobooks and e-books as “real”, “legitimate”, or “valid” ways of reading, and audiobooks or story tapes were considered to be an (performance) art form on their own: Audiobooks are however very valuable for their own unique aspects: listening to well crafted words. (thread 2)

Celebrating reading independent from the formats was regarded as important. Furthermore, the impact of literature was valued over the format. Another subthread discussed the bounding box of listening – would listening prevent from learning grammar and orthography, or even prevent from acquiring proper reading skills? Would it need to support skill development anyways? What about learning pronunciation? Would reading and listening at the same time be cheating? One mentioned to feel guilt when having listened to an audiobook, but said to have read the book, another one indicated to feel unfaithful: All reading is good BUT..I just can’t get into e-books..I’d feel like I’m being unfaithful to printed matter..“you’ve been around so long...” (thread 2)

Would listening to audiobooks while doing something else (see details on what activities audiobooks accompany below) allow to immerse oneself, to create meaning? Overall, meaningfulness was discussed intensively, which seemed to be a major value in reading, similar to bed time stories that were mentioned to contribute to developing reading skills and interest in reading in children.

In terms of efforts of reading, audiobooks and physical books/e-books were assessed similarly, since all of them require time and attention. Some were arguing that listening would require less effort – leading to a discussion whether or not it would be less valuable then: I wonder if it’s a repressed protestant work ethic which makes us imagine that consuming the audiobook rates lower than reading. (thread 1)

It seemed to be very important though, that the audiobooks are not abridged, which is easier today than it was in the past, since cassettes required to cut text to fit their length.

4.3 Inter-individual Material reading Needs and Preferences

People seemed to acknowledge that the ways and formats of reading are highly influenced by individual needs and preferences. There is no ‘proper’ reading, but a variety of approaches (see section above). They argued over whether or not the experience of books in different material manifestations would be akin to each other or not; children’s books were often referred to: would reading out the book to children mean that the children had read the book? Yes! Our youngest son learned a love [sic] of stories by having professionals read him books at bed time (audio books.) So he started school with excellent listening skills and the ability to follow a story. He learned the best stories are in books, and that motivated him to read.(thread 1)

Furthermore, the imagery that was created through reading or listening was discussed; some considered physical books to facilitate the creation of an own world better than audiobooks, while others reported that audiobooks would result in rich imaginative worlds. There was also disagreement of whether books or audiobooks would provide more detail or lead to increased involvement, or whether recollection was facilitated. However, there seemed to be agreement that those experiences would differ for individuals.

The triggering questions as whether or not audiobooks and e-books count as reading led to intensive discussions about individual accessibility needs as a result from impairments. If individuals are not able to handle a physical book for any reason, would they never have “read” a book? Many conditions were mentioned that would prevent people from reading a physical book, ranging from visual impairments (low sight, double vision, glaucoma, blindness) to dyslexia, fatigue, attention difficulties, aphantasia, autism, arthritis, chronic pain, multiple sklerosis, travel sickness, and illiteracy. Audiobooks were considered to be helpful: Some ppl can not access books in written form whether due to visual impairment or learning difficulty. Audiobook versions are their way of reading a book and a lifeline to many. (thread 1)

For instance, fidgeting in case of attention deficit disorders, or providing the correct tone and thereby supporting understanding the content, emotions, or sarcastic tones were mentioned to be particularly beneficial (expect for people with aphantasia, for whom this would not be helpful). E-book readers were specifically valued for their low weight, since they would allow holding them, as opposed to physical books, which would be painful to hold in case of arthritis, for instance.

4.4 The reader Engaging with the (im-)material Book

One of the major things that seemed to matter to the readers was their relationship with the author of the book. Physical books were considered to be experienced as intended by the author, resulting in a “direct” conversation between the reader and the author, whereby the reader creates his own interpretation All I’ll say is that when you read a book, you create the voices in your had [sic]. You add the context, the inferences, the tone etc. You become part of it. In audio books you’re hearing someone else’s interpretation. Maybe that’s an arguement [sic] over enjoyment rather than better/worse. (thread 1)

Own interpretations would be less relevant in audiobooks because the third person, the narrator, might alter the experience through her/his performance, substituting one’s own interpretation with someone else’s. It, thus, would remain unclear whether the author reaches the audience as intended, whether the book is authentic. I think there’s a difference between reading as practise and consuming an author’s ideas in audio. I love audiobook, the tone is set for you as author intended, but reading allows more active participation in the creative process; more resistance, debate, tension, subversion. (thread 1)

Others considered it differently: [...] I’ve connected w/ the words, the characters, the story as closely when reading an audiobook as I have reading [sic] a physical book. Reading a physical book is not somehow magically “more authentic” than reading audio. (thread 1)

Some considered books to be more authentic than audiobooks, while others indicated that listening would be very authentic given that stories were passed on verbally in the past. The narrator might even improve the experience, e.g., the narrator’s voice sounding better than one’s own, or audiobooks might provide different readers or voices, similar to a play. Audiobooks, furthermore, would have advantages since they allow to speed up or help with pronunciation. They may also slow down, since they preset the pace. However, revisiting specific places in a book differs (rewinding, finding the right spot, etc.), and footnotes and references are missing. Physical books were valued due to several advantages over audiobooks, for instance, they provide a specific smell, they are tangible, they allow to physically turn pages, and they can be arranged in form of a library. They allow a dynamic reading experience, and the experience after finishing it seems to feel more complete than with other formats. E-books, however, may be tailored to personal needs and situations (fonts and font size, contrast, lights), and they provide immediate access to books: Never wanted an ereader until I got one. Finish a book? You have 10 to choose from right away. Read a review of a book that sounds interesting, 2 clicks and it’s waiting for you whenever you need it. Love it! (thread 2)

However, they often imitate the feel of a book, though they lack qualities that physical books have, such as the possibility to have them signed by the author. Also, some have low screen resolutions and bad scrolling modes, which affect the experience.

4.5 Situated Materialized reading

In particular in regards to audiobooks, the possibility of doing things in parallel was mentioned several times. Audiobooks were described as particularly beneficial for time consuming activities, with low (mental) workload. They may accompany driving (e.g., commuting, bringing kids), doing sports, housework (e.g., cleaning, cooking, doing the dishes), crafting (e.g., knitting), watching and walking the baby, and they are valuable in case of sleeplessness. A lack of time (e.g., due to work, due to parental obligations) was indicated as one reason for turning to audiobooks, others were boredom, tiredness, work conditions (e.g., being a lorry driver, or working all day with a screen and aiming for a break for the eyes), aiming for a larger “output” (finishing more books) that audiobooks would allow. The possibility to listen together was considered specifically valuable, resulting in a shared, social activity. However, audiobooks would not meet requirements for textbook reading, such as skipping and re-reading. E-book readers were preferred for their mobility advantages (e.g., on holidays, on planes, while camping): Since becoming a student mine [e-book reader] lives in my handbag, if I’m waiting for anything I can get it out and do some revision even if it’s just 10 mins. It’s also great for camping. (thread 2)

E-book readers were also considered to be particularly useful at night due to included light. Physical books were not discussed too much in regards to their situated potentials and problems, except for mentioning that they do not need to be charged.

5 (Inter-)disciplinary Positioning of Findings

The difficulty associated with the aforementioned findings is that they affect various (intersections of) disciplines. For instance, findings related to how imagery is created while reading or listening to books may be positioned within linguistics, developmental and cognitive psychology, cultural studies, etc. While a discussion that connects the different disciplines would be desirable, it is beyond the possibilities of this paper. Thus, we focus our discussion on those findings that may have an immediate implication for HCI and Interaction Design research, but integrate related work from other disciplines where possible. In particular, we aim to strengthen the relation between audiobooks and HCI, which seems to be somehow deprived in our field. Thus, we take a glimpse at “audionarratology”, “an umbrella term for narrative approaches that take into view forms and functions of sound and their relation to narrative structure.” [29, p. 8] Therein, text experience [25] is a relevant notion, i.e., how individuals experience narratives; among others, we address this in the following discussion.

6 Discussions and Implications

In regards to the specific notion of materiality of reading artifacts, which this paper started from, we now revisit the earlier described framework of materials experience [17]. Considering the four experiential levels that materials are experienced on, there had been many statements in the findings that affect the sensory level. The diverse statements regarding sensory qualities that books, e-book readers, and audiobooks have, underline that the very question of whether audio is a valid form of reading matters to individuals; based on the findings we might assume that the form of sensory input does make a difference for individuals. On an interpretative level, especially norms and values associated with reading were vividly discussed, including the notion of elitism or inclusiveness of reading. Further, there were also statements belonging to the affective level that denoted affective dispositions towards the material embodiment of books, such as different experiences in regards to immersion, likings, or aversions.

Finally, it seems that reading is a highly performative activity, i.e., happing on the level of humans’ performances, which they establish around material objects that are influenced by perceptions, meanings, and affects. Performative reading activities ranged from spatially, temporally, or socially situated reading that are influenced by (and, at the same time influence) the material manifestations of the book.

Giaccardi and Karana furthermore talk about materials experience patterns (different material experiences for different individuals, i.e., a “situational whole”). These patterns can be active or broken; the latter only fulfil a functional or aesthetic role, in the former, materials play “an active role in the unfolding and transformation of practice” [17, p. 2451]. In our findings, we find both active and broken patterns. For instance, the notion that physical books can be displayed in form of a library may be considered an active pattern, since the very material configuration of the physical book allows to do so: the paper in the specific format, book binding and cover is self-standing, it can lean against a bookshelf or another book, etc. In contrast, in regards to displaying books in form of a library, patterns in e-book readers may be considered broken, since the material reality is not resulting in a specific “performance” (here: organizing one’s library). In contrast, practices of searching for something within the book may form an active pattern in e-books, but not in physical books.

Implication #1: In order to take the materials serious in the design of future books may mean to deliberately explore active material experience patterns to create novel and rich reading experiences. Therefore, radical material explorations are needed that go beyond the known text-material relationships in physical books, e-book readers, and audiobooks.

On one hand, our findings support existing research on reading in that we also found that physical books have values that e-books or audiobooks don’t have, such as providing the possibility to arrange them in libraries (e.g., [19]), or that the form factor of e-book readers (e.g., lightweight, mobile) influences how and when they are used (e.g., [22]). On the other hand, our findings also add to the existing body of research. For instance, while there is a huge body of knowledge around reading practices and experiences with physical books and e-book readers, there is surprisingly little engagement with audiobooks in HCI and Interaction Design research. Though one could argue that research on audio players of different kinds would include audiobooks, it might be important to consider audiobooks as a material manifestation of reading and, as such, include it into research on reading and (interactive) reading experiences.

Apparently, e-book readers are hardly researched independent from their content, i.e., fiction or non-fiction texts (e.g., [4]) or pictures (e.g., [36]), while HCI research on audio players hardly seems to focus on texts. This, however, would be highly important; our findings unveil that people highly value audiobooks next to physical books or e-book readers. Furthermore, the content defines how people interact with the particular material manifestation. For instance, in her work on text experience, Kuzmičová discusses differences in people’s desire to go back in reading or listening to grasp details; in contrast to academic reading, narrative reception is characterized by a fluency of experience, meaning “that recipients prefer trying to catch up with a narrative before taking the radical step of rereading or re-listening” [25, p. 229]. This may have serious implications for the design of audio players that may not focus on optimizing rewind or forward features, but on immersion, e.g., through providing additional details via additional sensory information. Consequently, if we consider audio as a design material, we might be able to overcome the current use of audio as an add-on to “enhanced books”, which seems to be the prevalent form of auditive components (e.g., [24]).

Implication #2: Audio might be considered a design material as much as visual text or computation, to enrich text experiences for individuals. More specifically, audionarratology, (i.e., forms and functions of sound and their relation to narrative structure [29]) may be the material to start from.

Much of the experiments in HCI and Interaction Design regarding computational books are rooted in the idea that mimicking physical books would support ease of interaction, such as allowing to turn pages in e-book readers just as in physical books (e.g., [4, 8, 23]). However, our inquiry also showed that people value physical books, e-book readers, and audiobooks for different reasons and in different situations. Thus, creating different options that do not aim to replace, but complement each other, would be beneficial. Qi and Buechley’s suggestion to design artifacts that function differently than what we are accustomed to [32] might provide guidance here, though not only affecting the function, but also the form and interaction gestalt (i.e., the composition of interaction qualities [28]) of future books. When considering that the readers (or listeners) develop a distinguishable relationship to the author of the book depending on how the text is materialized, it is imperative to reflect these substantial differences in the design of future artifacts.

Implication #3: Research and design need to envision different alternatives for reading that go beyond translations from one into another material. Novel materialized reading options should be created instead of computational artifacts replacing physical books through mimicking them as much as possible.

In particular, the situatedness of reading seems to be highly related to the experience of different (material) manifestations. Our inquiry revealed that the qualities of the manifestations facilitate, constrain or afford specific use practices, which supports the aforementioned notion of socio-materiality. However, it may be considered the other way round as well, when the environment that one aims to read or listen in affords a specific material manifestation:

[A]dults with developed reading skills and relatively solid reading habits rarely ask themselves whether they want to read a narrative or listen to one instead. Rather, they choose between the two media based on their instantaneous situation and the type of reception it affords. [25, p. 220]

Based on our findings it may be assumed that there are many instances were the material manifestation is chosen to fit the environment, rather than creating an environment that fits the material manifestation of reading (i.e., rather using an e-book reader at night instead of turning on the light in the room to be able to read a physical book).

Implication #4: In order to facilitate individuals’ reading, HCI and Interaction Design research may deliberately focus on both the affordances of the technology and the affordances of the reading environment, and, in particular, on their intersections and dynamic nature; designing for reading may also mean to design the (interactive) reading environment and situations rather than just the interactive book.

Finally, the many examples in the research material, which relate to accessibility, remind us to rethink the relation between reading and the reader’s body and mind. Though the differently materialized forms of books already provide various possibilities for different bodily or mental needs and preferences, we have not yet developed sufficient, satisfying options. For instance, Schilhab et al. [37] explain how

We all outsource otherwise fragile and costly mental processes to the environment as we integrate the materiality of the text in our memory. The materiality of the printed book makes it a stable environment in the same sense as the familiar rooms. [37]

In contrast, when individuals read digitally, solid external crutches that are formed by different sensory processes are almost absent. Thus, “[t]he unfolding meaning attribution to the text occurs without much material anchoring.” [37]. And the same might apply to audiobooks as well. As mentioned earlier, the Stavanger Declaration on the Future of Reading also emphasized the necessity to focus on embodied cognition in (novel forms of) reading [12]. Our inquiry supports this notion in that it points to issues of individuals’ bodily experiences (e.g., the pace of listening or reading, bodily capabilities and impairments, or the tangibility of books); it becomes apparent that the body, the mind, the material, and the environment play together. We have argued earlier that we need additional options (rather than replacements) for reading, but this is actually not enough.

Implication #5: The body-book relationship is a matter of accessibility, which not only means that there should be some access, but that alternatives for rich experiential accesses become avaiable. Furthermore, there are much more complex relationships (body-mind-material, body-mind-material-experience-text, body-mind-material-experience-text-author, etc.) that need exploration and, eventually, fitting designs. In order to reach this, research and design may need to focus on multi-sensory, materially-rich devices and environments.

7 Conclusion

This work was motivated by the observation that there are various material manifestations of reading (what we call materialized reading) that evolve, influence, and inspire each other – independent from whether they are physical, computational, or auditive. Positioned within the scientific discourse in HCI and Interaction Design around materiality, i.e., the material qualities of interactive artifacts that affect how individuals experience them, we addressed the question of how the materiality of books influences individuals’ experiences, perceptions, and opinions, and what we can learn from respective findings for the design of future books. We therefore engaged with qualitative, subjective material in form of two discussions on Twitter in order to inform and inspire discussions around the materiality of reading. We found five themes to be central: meanings of reading, norms and values of reading, inter-individual reading needs and preferences, the reader engaging with the [audio/physical/e-]book, and situated reading. Our findings expand the existing body of HCI research, among others, by focusing not only on physical books in contrast to e-book readers, but by including audiobooks as another material manifestation of reading. For instance, we found that the intermediary role of narrators in audiobooks is a powerful one, in that it can change (improve or diminish) the individuals’ experience or interpretation by interfering “the conversation between readers and authors”. We also saw people engaging in discussions around accessibility, a topic that is of high relevance in regards to the materiality of books.

Overall, the findings urge us to continue to research and design for rich reading experiences that are inherently influenced by the (interactive) material manifestations not only of the texts, but also of the reading environments. As our study was a qualitative inquiry with limited insights on people’s socio-demographic, socio-cultural, or motivational background, we refrained from establishing any inferences. These will be necessary and worthwhile research directions to take in the future.

Through our work, we contribute recommendations for research on and design of future books: Our (most evident) recommendation for HCI and Interaction Design research and practice is to take the materials serious in the design of future (interactive) books; hereby, we specifically aim to draw attention to audio (and, in particular, audionarratology) as a neglected design material, going beyond adding sound, but considering it an inherent sensorial quality in the experience of texts. Additionally, we suggest to put efforts into envisioning different – novel – alternatives for reading that not just mimic the qualities of a physical book in an e-book reader. Eventually, we hope to see alternatives for rich experiential reading evolving that address the complexity of the relationships between body, mind, material, and text.