Keywords

1 Introduction

Free choice learning environments like museums and exhibitions have struggled to become more interactive and collaborative. They have also been challenged to balance the focus of the interactive displays with the needs of the exhibition [10]. For example, adding interactive multimedia displays can monopolize the attention of visitors and the focus becomes on the technology, rather than the exhibition. As technology progresses, more options become available to create engaging digital solutions for visitors [11]. These solutions replace the standard, static displays that are common in traditional museum settings. For example, Planar, a company that provides digital technologies for museums, notes that large video displays and interactive flat screens have become a fixture in many modern-day museums [15]. Planar also notes that large video displays introduce an exhibition and set visitor expectations, encourage engagement, and create visually rich experiences. The ultimate goal of these displays is to encourage return visits, attract visitors, and create memorable experiences within an exhibition.

Although large wall screens may make learning experiences more appealing to museum visitors, without quality content, a large wall screen is an expensive piece of wall art. Contrast Creative, a company that designs for large interactive wall displays, describes them as, “the perfect vessel for imagination. Imagine content that comes alive when someone approaches and where no users have the exact same experience twice” [9]. Currently, there are limited studies that focus on the storytelling potential on large wall displays. Rather, they reference stories as the content on large wall displays and then focus on the design and interactivity associated with software and hardware. Multiple studies exist to understand how large displays function and to assess usability. As large wall screens grow in popularity, some studies have been conducted to determine possible heuristics for designing large displays. Usability experts from the Nielsen Norman Group have conducted studies to understand the user experience of large touch screens. Additionally, studies have explored how multiple users interact with one display [13]. However, fewer studies seek to understand the relationship between interaction design patterns for large touch screens, information architecture for a complex collection of artifacts, and content for a multimedia integrated exhibition. In museums, digital displays are sometimes gamified in order to tell their stories, referred to as “edutainment” [2]. However, studies that focus on edutainment often focus on understanding the gamification of learning rather than the interplay of HCI and storytelling.

Large wall displays are increasingly incorporated in museums and exhibitions, as well as public places to support social activities [4]. Also, [4] discuss different ways to design interactive displays that invite interaction and engagement. Their study indicates that social embarrassment is the leading factor preventing people from participating in public interactions. Therefore, for public interaction to become acceptable, the large display must have strong physical and social affordances, so visitors can easily become more familiar with the display. In order to promote public participation in large wall displays, the user experience must be simple and intuitive. Additionally, the interface needs to be clear so that users are reassured that the experience will be quick, enjoyable, and a low commitment activity. It is important to acknowledge the behaviors of museum visitors and other users in order to build the right experience for the space.

This paper chronicles the results of research conducted in the context of an interdisciplinary project to explore rich narrative content with cutting edge technology, specifically, a large touch screen display. To better understand the interplay of human-computer interaction and storytelling in the designs for this display, a series of design concept and usability tests were conducted. Results indicate that large displays with complex narratives should consider the height and display of icons, affordances are crucial for users to understand interactions such as scrolling, complex storytelling should be concise, and the design should encourage exploration.

2 Background

This literature review draws from three main areas of study: (1) the evolution of storytelling in museums and exhibitions, (2) interactive exhibitions in museums, and (3) usability research on large, interactive displays.

2.1 Storytelling in Museums and Exhibitions

At the heart of a great museums or exhibitions is robust storytelling that provides visitors with both depth of information and richness of experience. The best storytelling allows visitors to make personal connections with engaging exhibits because the displays support interpretation and multiple perspectives [3]. A case study on narrative in exhibitions [3] argues that “narrative allows people to imagine themselves in an unfamiliar world” (p. 31). In the first half of the 20th century, exhibit stories were usually directly related to the historical context of an artifact. However, in the latter half of the 20th century, museums began to evolve exhibition storytelling by offering thematic arrangements and presenting multiple points of view [10]. Likewise, as technology continues to improve, museums have developed new ways to communicate stories associated with exhibitions. In fact, experts have asserted that as museums work to make exhibit spaces more interactive, developing a cohesive narrative is increasingly important [11].

Edutainment experiences in museum exhibitions have increasingly made use of interactive technologies to attract and engage audiences. [7] note that “digital technologies allow more sophisticated nonlinear stories; allowing visitors to interact with the story at different points in time” (p. 106). This research evaluated the use of narrative in digital interactive storytelling and found that most audiences appreciated interactivity as a central part of the story design. They also found that users will stay engaged with an interactive exhibit as long as they perceive that the content is of high quality. In addition to quality of content, perceived ease of interaction and users’ level of enjoyment also determine users’ level of engagement.

However, most museums are challenged to effectively balance narrative content with technology. [10] argued that “richness and complexity of information, in combination with the availability of a variety of media (audio, visual, haptic) should not take the focus away of the initial goal which is to tell a story from a specific point of view that gains the visitor’s interest” (p. 422). To achieve this balance, the authors argue that visitors should feel a sense of personal connection with stories. When museums work to personalize stories, users become less overwhelmed by the interplay between content and interactivity. Museums are still adopting more technology to encourage interactive storytelling, however, there are few best practices to guide the creation of a narrative on a large wall display [10].

2.2 Interactive Exhibitions in Museums

As interactive exhibits become popular in art, history, and cultural museums, technologies are also becoming increasingly valuable for visitor engagement and involvement. Not only does interactivity promote immersion, but also helps understand and recall exhibits and their content [1]. According to [1], interactive exhibits can be memorable, as visitors were able to describe the feelings they had at the exhibits months after their visit. Additionally, rather than interactive exhibits offering a different experience, visitors can use these as a tool to enhance their visit and appreciation of the museum [12].

There are key factors in the implementation of interactive exhibits or kiosks in museums e.g. they should be strategically positioned within the space, they should have a well-designed welcome screen, and a clear indication of interactivity as oppose to signage for non-interactive screens. In a study conducted by [14], four characteristics were examined to test the effectiveness in holding the attention of visitors in a science museum: technological novelty, user-centeredness, sensory stimulation, and open-endedness. Technological novelty and open-endedness showed positive correlations with the amount of time spent at the exhibits. Aside from these characteristics, the physical context where the digital media is experienced influences visitor interaction; as visitors consider the space and time they spend at each exhibit. If users have space to comfortably interact with kiosk or similar experiences, they will more likely engage with them.

Museums have begun to integrate multimedia storytelling exhibits beyond physical or medium-screen touch screen kiosks. [16] explore digital representations of large-scale artifacts in non-instrumental, location-based, multi-user interactions in order to enhance engaging experiences for visitors. Their computing system displays artifacts that are difficult for the public to access. Its features allow for visual and textual annotations of the artifact, as well as additional background information about the object or creators. Visitors can interact with these digital representations by walking around a location-based and motion-sensored space. For example, a wall painting from the museum was used as the digital artifact, and users had the opportunity to experience it by triggering designated hotspots located at different locations on the floor. Systems like this increase visitor engagement and provide innovative ways to access difficult museum artifacts.

2.3 Usability of Very Large Displays

One-way museums are incorporating multimedia storytelling and technology into exhibitions, is through the use of large touch screen displays. Dorothy Shamonsky, Lead UX Designer for ViewPoint and a large touch screen kiosk designer, reports that the freedom and novelty of a large screen are the two most appealing qualities. She notes that humans find it incredibly satisfying to have information presented in a large view and the option to interact with the information through the freedom to “play.” Additionally, Shamonsky asserts that very large displays allow users to become an entertainer. “Obviously, others can observe what you are doing, so you become a performer of sorts with the application, which can be fun” [13]. When it comes to the usability and design of a very large touchscreen, [13] states that there are still several basic guidelines that apply to any screen size, including to allow natural gestures; minimize the interaction cost of tapping, typing, and moving between screens; offer user feedback via simple animations; make it easy to decipher which elements are tap-able; make targets easy to tap; and offer legible text and graphics. Despite this, there are important differences when designing for large touch screens compared to mobile phones. [8] used an eye tracking device and interviews with both expert and non-expert users to study usability on very large touch displays. [8] found that users adapted easily to new gestures within the large multi-touch display system based of their previous habits, users were satisfied using the large multi touch display system, eye movements of all users were around the center of the screen, the most common used gesture is tap, and users preferred to use both hands for multi touch gestures like zoom in, zoom out, and rotation.

3 Methodology

To understand the relationships among interaction design patterns for large touch screens, information architecture for a complex collection of artifacts, and content for a multimedia integrated exhibition, this study employed a mixed-methods approach that included two concept tests and two task-based usability tests.

3.1 Concept Tests

Concept tests were conducted to understand the interactions, designs, and concepts that users would most enjoy. Two sessions were conducted with five users who had some prior experience with interaction design either in school work or professionally. During the sessions, researchers walked participants through the design concepts for the large wall screen. Participants could ask questions, interact with the screen, and provide ideas. At the end of the session, participants were given a post-test questionnaire to solicit users’ feedback about the design, storytelling content, navigation and overall experience. The post-test questionnaire included open-ended questions that allowed participants to provide suggestions, thoughts, and insights about the design. A medium-fidelity, interactive prototype was then developed to be used in a subsequent usability test (Fig. 2).

3.2 Usability Tests

Two task-based usability tests were conducted, the first of which occurred after the design was revised based on feedback from the two concept tests. This first test included five participants with little-to-no prior experience with large wall displays. The second usability test was conducted after the designs were again revised based on feedback from the first usability test. The second usability test included five expert users, with professional experience or academic instruction on interaction design and five novice users, with no prior experience in interaction design. The goal of testing users with varying skill levels was to determine how well the large wall design was understood by both novice and expert users. This is important as the large wall screen is prototype of a public display intended for an exhibition created for all types of users.

Task-based usability sessions lasted approximately 15 to 30 min each. Five tasks were designed to explore the key interaction patterns in the design and navigation, i.e. swiping to browse through the layered narratives, identifying key icons and affordance elements, recognizing multimedia content, and understanding the interface structure. Participants were asked to think-aloud during their testing sessions to inform researchers of their thoughts while attempting to complete a task. Each test was audio recorded.

At the end of each task, participants completed a task difficulty questionnaire, and researchers completed success rating scales intended to gauge how successful each participant was in completing a task. The rating was based on the following scale: successful–users were able to fully complete the task; partial success–users completed part of the task; failure–users were unable to successfully complete the task; and quit–users abandoned the task. Each rating was assigned a value as follows: successful (3), partial (2), failure (1), and quit (0). At the end of the testing session, participants were asked their thoughts on the design and the experience. Finally, all participants completed a System Usability Scale (SUS) [5] questionnaire to assess the overall usability of the system. Finally, an open-ended post-test questionnaire was used to gauge how users felt about the narrative content and if they were able to understand it.

4 Results

4.1 Concept Tests

The concept tests yielded important qualitative data. Open-ended questions provided insights into how to further develop the wall screen design. Six key themes emerged from the users’ feedback: (1) users are only be willing to go three to six clicks into a story before losing interest; (2) tap-able elements in the initial design were not easily identifiable; (3) users were open to sharing the same touch-screen space with one another; (4) sound would add to the experience, but headphones may be needed; (5) more movement and animation are desired for the design; and 6) a user’s height may determine accessibility to all content.

4.2 Usability Test One

The overall ratings of the large wall screen during the first task-based usability test indicate that the experience was easy for participants to use. A perfect score for all tasks completed was 75 (Table 1). The total score from the test was 48, an overall task success a 64%. This data suggests that users completed each task with some issues. For example, task one required users to scroll and select a narrative (Fig. 1). All users selected a narrative, but three out of five did not know to scroll in order to browse. The most difficult rated tasks were task three and four. Task three tested whether users could locate the artifacts from the navigation by locating the carousel icon. Task four tested whether users could switch between themes.

Table 1. Usability test 1 - Tasks and overall task success rates.
Full size table
Fig. 1.
figure 1

Initial concept consisted of a carousel displaying different artifacts.

Full size image
Fig. 2.
figure 2

Second concept displayed each artifact and narrative in a grid form.

Full size image

Icon recognition was a problem many users suggested should be addressed in order to avoid confusion. For example, the icons on the side navigation were difficult to find, causing confusion as how to further explore the experience (Fig. 3). For example, users had the most trouble completing task 3, which asked them to identify certain items in the collection (Fig. 3). The artifact carousel icon did not register significance, which kept users from successfully completing the task (Fig. 3). Additionally, each narrative began with a story title and a down arrow to indicate each grid element was could be tapped (Fig. 3), but participants had difficulty identifying this feature for each narrative grid. Users were unable to identify different narratives within the grid, as they did not understand there were different narratives to explore. Another problem was the swiping affordance to differentiate where the narratives began and ended (Fig. 3). For example, users did not know how to interact with the design in order to find additional narratives. Participant 2 stated, “Oh, I didn’t know that scrolled” when completing this task.

Fig. 3.
figure 3

(1) Icon recognition: Users struggled to find this specific icon to explore the carousel feature. (2) Users consistently tapped on this static arrow. (3) No swiping or scrolling affordance to direct users to complete action. (4) Main navigation bar difficult to interact with and not user friendly for height.

Full size image

The layering system was also a problem many users encountered and should be addressed to improve navigation (Fig. 4). For example, all users were able to find the correct story within the design, but they were unaware that they could dive deeper into each story and bring up additional content through the layering system (Fig. 4). Additionally, users believed stories in the layers should be concise. User 2 stated, “I think it’s really interesting and a cool way to give an overview of a topic or a person’s life, but I’m not sure how often I’d come back to the same story.” Other users indicated that they would be interested in using this system but would prefer shorter stories so they would not become fatigued while exploring multiple different narratives.

Fig. 4.
figure 4

(1) Each narrative grid consists of individual explanations for the selected media content. (2) Complex and longer narratives have additional ways to explore content as “pop ups.” The additional content layers are identified through highlighted or underlined keywords or phrases.

Full size image

4.3 Usability Test Two

Overall ratings for the large wall screen during the second test indicate the experience was easier for users than the first design (Table 2). The system usability score for the first test was a 72, and the final test was a 75.25. For the second test there was an overall 85% success rate across all tasks. Users still struggled to recognize the scrolling interaction, despite the implementation of a down arrow at the bottom of the screen (Fig. 5). The addition of the menu icon at the top of the screen made it easier for users to navigate to the most difficult rated tasks (Fig. 5), like tasks three and five. Task three tested whether users could locate the artifacts from the navigation by locating menu icon and selecting artifacts from the navigation. Task five tested whether users could switch between themes. The implementation of key words like “switch theme” or “artifacts in the collection” paired with corresponding icons helped users identify these features. Although users had an easier time doing this with the new menu icon, it was rated as a more difficult task. Users suggested moving the menu icon to the bottom of the screen for easier accessibility and repositioning the top menu from the middle to the top-right corner. Expert 5 stated, “I thought that was decoration” when referring to the location of the menu icon (Fig. 5).

Table 2. Usability test 2 - Tasks and overall task success rates.
Full size table
Fig. 5.
figure 5

(1) A Google Design icon was implemented for the navigation bar. This icon hides the navigation options and shows them when tapped. (2) The narrative grid was redefined as “stories.” (3) Specific language was used to identify the artifacts in the collection. (4) An option to “switch” to a different theme was added to signify the difference of both themes. (5) The arrow was removed and replaced with key phrasing. (6) Down arrow was added to encourage swiping through the narrative grid but had no effect on the usability.

Full size image

Another recurring issue was the lack of understanding between the meaning of an artifact versus a story, which left users unable to navigate between the stories section and the featured artifacts. While there was a 30% success rate improvement on this task, users experienced difficulty identifying an artifact in the carousel feature (Fig. 5), as they searched in the wrong layer of the experience. Users felt that in order to understand the design and appreciate the narrative, exploration of the design was required. Novice 4 stated, “I feel like if I was just left to do it, without tasks, I would have figured it all out, just exploring and messing with it.” Lastly, users were unaware that they could swipe up and down to scroll through the stories. Task 1 tested that users would scroll and choose a story, while all users were able to select a story, only two users scrolled to see additional content.

5 Discussion

This study chronicles the usability and interaction patterns of a large touch screen display. There are three key interaction design issues that were illuminated by this study, which can be characterized as follows: (1) the location and height of key icons should be within reaching distance for all users, (2) if scrolling is included in the design, it should use more visible affordances, and (3) complex storytelling should be concise and encourage exploration.

Key icons should be placed at reasonable heights and locations, because not all users are the same. When the menu icon was placed at the top in the center of the screen, users had difficulty identifying it because it was too far out of their immediate field of vision. Although the icon was moveable and could be re-positioned by the user to account for height differences, its design did not indicate that specific functionality. As a result, users were not aware the icon could be dragged to move it. Icons that are essential to navigation should be easily recognizable and be accompanied by keywords if the concept is too complex. Second, if scrolling is a key interaction for navigation, the design of scrolling content should include more visible affordances. This feature resulted in the least successful task, as users did not immediately know they could swipe the narrative grid up and down. Despite adding explicit and implicit affordances, such as arrows and cut-off content, users did not immediately understand the scrolling interaction. They needed time to explore before discovering the option to scroll. Key design indicators should be present to suggest swiping or scrolling. Using a carousel feature could work, giving the user visual cues of where they are present in the design or storyline. Finally, scrolling on a large display will also allow users to view the content at their desired height for comfortable viewing.

Complex storytelling should be concise and encourage exploration. Information must be displayed in short “chunks” instead of long-form text displays. The initial design presented challenges with in-depth storytelling and limited user attention. To solve this problem, we introduced a layering method intended to break up long narratives into shorter “chunks.” The layering method easily organized the use of extensive stories and multimedia elements into smaller, easier-to-digest text blocks. However, through usability testing we discovered the layering method was not an effective way of displaying content. We found that all five users from the first usability test were not likely to go deeper and read more than the first layer of information. Rather, the users were more intrigued and entertained by multimedia elements. Additionally, due to the large amount of information, users become overwhelmed by the narrative elements. Therefore, long text paragraphs should be avoided. User 4 from the first usability test stated, “It [the large wall screen] was a lot of fun. My only thing is, if you’re trying to find stuff on the spot, you get really nervous.” The user went on to say she liked the interaction, found it entertaining, but that she would need lots of time to explore. While users are less likely to read long paragraphs of text, they do engage with media content. Jumping from layer to layer, or screen to screen should be smooth, intuitive, and clear to the user. If the design encourages exploration, users are more likely to engage with the content provided on the display. Additionally, if the design encourages exploration users are more likely to see how different stories are connected. Encouraging exploration on a large display will make users more comfortable with the experience. With the expansive amount of content, exploration allows users to quickly learn how to interact with the system as well as efficiently navigate between different layers of the design.

5.1 Limitations and Future Work

The software used to design and build this large wall experience was Intuiface Inspector and Intuiface Player, a multi-touch and sensor-driven tool used to build web-based platforms or kiosk experiences without coding. Although it supports different types of media such as images, videos, documents, websites, 3D models, and more, it limits the number of experiences that can occur at a single given time and location. Although many users can interact with it at once – i.e., they can touch and drag objects – multiple users do not have the ability to freely have a one-on-one experience in the space. Additionally, Intuiface also limits user interface design options, such as typography and iconography. Designers can import original designs, but these are placed as raster images, often reducing the quality of work. Furthermore, Intuiface does not support basic animations like “sliding,” “appearing,” “fading in and out,” and “pulsing.” These simple animations are crucial affordances for user understanding and facilitating key interactions with the design. Finally, a big limitation and weakness of this experience was the lagging and slow response from the large wall screen. Because the software contained multiple content layers, due to its limitation in holding two experiences at once, the experience reacted slowly, leading users to believe some interactions were not functional.

Future research opportunities include exploring how gesture-based interaction patterns affect user experience. Gesture-based technology would allow users to stand farther back and have a greater view of the large display. This would prevent the need for users to strain themselves by looking up or down, thus providing a greater opportunity for them interact with more of the content and more quickly see necessary icons. Additionally, future research could also include usability testing with multiple users interacting with the display to better understand how large touchscreen designs affect multi-user environments.