Advertisement

Journal of Computers in Education

, Volume 6, Issue 1, pp 143–166 | Cite as

Interactive digital storytelling: bringing cultural heritage in a classroom

  • Selma Rizvic
  • Dusanka BoskovicEmail author
  • Vensada Okanovic
  • Sanda Sljivo
  • Merima Zukic
Article

Abstract

Interactive digital storytelling is becoming a popular choice for information presentation in many fields. Its application spans from the media industry and business information visualization, through digital cultural heritage, serious games, education, to contemporary theater and visual arts. The benefits of this form of multimedia presentation in education are generally recognized, and several studies which explore and support the opinion have been conducted. In addition to discussing the benefits, we wanted to address the challenges of introducing interactive digital storytelling and serious games in the classroom. The challenge of the inherent ambiguity of edutainment, due to opposing features of education and entertainment, is augmented with different viewpoints of multidisciplinary team members. We specifically address the opposing views on artistic liberty, at one side, and technical constraints and historical facts, on the other. In this paper, we present the first findings related to these questions and hope to initiate further discussions in this area.

Keywords

Interactive digital storytelling Edutainment Cultural heritage 

Introduction

Technology development is introducing new ways of interactive communication in a domain of multimedia cultural heritage applications. The quality of user experience is assessed in terms of ease of use and effectiveness. This can be directly linked to entertainment and education, respectively.

Therefore, presentations methods are also changing. The content is divided into hyperlinked structures, giving the users on the first sight an overview of what they can find inside. Digital storytelling is following this methodology. Nowadays, online applications are presenting the content as a serial of short stories, enabling the viewer to choose how deeply he/she wishes to dive into the information. Still, there is no universal solution for all possible presentation areas.

Handler Miller defines digital storytelling as the use of digital media platforms and interactivity for narrative purposes, either for fictional or for non-fiction stories (Miller 2004). Interactive digital storytelling (IDS) enables the user to influence the flow and sometimes even the content of the story. This new form of conveying information involves professionals from multiple disciplines.

The area of use for interactive digital storytelling is expanding fast in several application domains. In our research, we focus on virtual cultural heritage presentation and serious games. Our experience in this area has led us to develop guidelines for IDS presentations of cultural heritage. The objective of these IDS is to present documentary historical information about heritage objects or sites in an engaging and immersive way. The applications target a general audience in museums and online.

In our attempt to develop a new method for interactive digital storytelling, hyper-storytelling, we engaged experts from computer science, visual arts, film directing, literature, psychology, communicology, and human–computer interaction. They have analyzed a sample interactive digital storytelling application and offered their insights and recommendations to be embedded in the new methodology.

In order to bring the cultural heritage in the classrooms and employ multimedia applications, especially IDS and serious games, we need to extend our team and include pedagogy expertise. The primary objective for introducing these modalities of content presentation is motivation for learners and accommodating the preferences and attitude of millennials. Importance of entertainment in educational applications is to create a motivating and successful environment for learning (Wiberg and Jegers 2003). Combination of entertainment and education introduces a new feature: edutainment.

Edutainment is not a novel attribute in designing educational applications, but the importance of edutainment is recognized in the research field of serious games (Ma et al. 2011). Edutainment was not widely used term at that time, but since then, edutainment has become familiar and recognized (Ma and Oikonomou 2017), and also important when introducing serious games in digital cultural heritage presentations (Philpin-Briscoe et al. 2017). It is notable that serious games can create more engaging educational practices leading to “the emergence of serious games as a new form for education and training” (De Freitas and Liarokapis 2011; Liarokapis et al. 2017).

Difficulties in discussing and measuring edutainment arise from the opposition between the important pedagogical aspects and aspects of importance for the entertainment part of the content presentation as identified in Wiberg and Jegers (2003), identifying necessity of tradeoffs in balancing both the entertainment and the educational part of the application. In assessing the edutainment influence on successful learning it is important to include measurements which take into account cognitive load theory (Sweller et al. 2011; Leppink et al. 2013).

In this paper, based on our experience, we present the guidelines upon which the hyper-storytelling method will be founded, but also, in addition, we address the challenges of the successful introduction of the IDS into classrooms. In “Related work,” we offer a brief overview of related work in IDS field with particular emphasis on virtual cultural heritage applications and serious games. “Lessons learned from the evaluation of IDS application” presents lessons learned from evaluations of the sample IDS application. The part of the evaluation process was conducted in order to introduce the interdisciplinary team members with the topic and goals of the research. “Guidelines for interactive digital storytelling” summarizes the recommendations of interdisciplinary experts in the form of guidelines for future IDS applications. “Exploring the challenges” presents results of the user evaluation survey designed to explore the challenges of balancing the education and entertainment, and also opposing views on artistic liberty, at one side, and technical constraints and historical facts, on the other side. We conclude with the future work directions.

Related work

California State University, Chico, class generated a five-part definition of digital stories, according to which, for assessment purposes, they should include a compelling narration of a story; provide a meaningful context for understanding the story being told; use images to capture and/or expand upon emotions found in the narrative; employ music and other sound effects to reinforce ideas; invite thoughtful reflection from their audience(s) (Alexander 2011). This definition introduces some keywords for our research: narration, images, music, emotions. It shows that only multidisciplinary teams can combine all these notions into an interactive application.

All authors in the literature agree that the foundation for successful IDS applications is the skillful use of general storytelling principles defined through history in all kinds of media. Aristotle’s seven golden rules, plot, character, theme, dialog, music, decor, and spectacle, are easily recognized in engaging and immersive interactive digital stories.

Hero’s journey is another storytelling structure which is or could be used in IDS. It is a pattern of narrative identified by the American scholar Joseph Campbell that appears in drama, storytelling, myth, religious ritual, and psychological development (1949). It describes the typical adventure of the archetype known as The Hero, the person who goes out and achieves great deeds on behalf of the group, tribe, or civilization. The proposed structure consists of 12 stages, starting with the introduction of Hero’s world, describing the call for adventure and following him through different obstacles until the desired goal is fulfilled. This structure is better suited for adventure movies and novels than for documentary narrative, but some elements could be applied.

Interesting example of users’ involvement is the Portland Art Museum offering visitors to record their own narratives about personal objects. This innovative initiative called Object Stories is fully engaging participants and “aims to demystify the Museum, making it more accessible, welcoming, and meaningful to a greater diversity of communities” (Burns and Murawski 2013).

The other example of innovative user engagement, also situated in museums’ environment, is the Modular Audio Story Platform for Museums (Salo et al. 2017). The platform allows visitors to record their stories and attach them to artifacts, and to share stories with other visitors. It is a portable open source-based audio digital asset management system (ADAM). The system was successfully implemented and evaluated it in the Museum of Technology in Helsinki, Finland (Salo et al. 2016).

PoliCultura is a large-scale educational digital storytelling initiative, having more than 40,000 users, both teachers and students collaboratively engaged in creating a multimedia story (Di Blas and Paolini 2013). The specific feature of the project is collaboration, since students and teachers are interacting and working together in order to create educational digital stories.

As the focus of our research is IDS in virtual cultural heritage applications and serious games, we present here some examples of such projects and discuss their advantages and drawbacks.

Etruscanning 3D project (Pietroni et al. 2013) is an IDS application created to present findings from Etruscan Regolini-Galassi tomb through an interesting combination of storytelling with the 3D environment of the tomb and interactive models of artifacts found there. The user stands in front of the screen where the virtual environment of the tomb is projected and interacts with the application using gesture-based interaction interface with Kinect motion sensor. This project is a great example of how to use storytelling to convey to the user the history and importance of archaeological findings. However, gesture-based interface limits its usability to a museum setup.

Admotum application created within the Keys to Rome exhibition on Roman culture during the rule of Emperor Augustus is a serious game engaging the user in the quest for objects from 4 involved museums (Pagano et al. 2015). The exhibition was held in 2014 at the same time in Rome, Alexandria, Amsterdam, and Sarajevo, with the goal to present different parts of Roman Empire at that time through a combination of museum collections and digital content. Admotum application was designed as a treasure hunt, where the users explore first the virtual environments of Roman objects from their location, and, upon finding all objects, they can unlock virtual environments from remaining three locations and look for their objects. Storytelling plays a significant role in this application, as every virtual reconstruction and particular objects are described through narrations of virtual characters. However, the users are so engaged with not simple gesture-based navigation that most of them do not pay much attention to stories they hear.

The mentioned projects open new research questions which will be addressed within our research for a new IDS methodology. The first question is how to obtain maximum user immersion in the virtual presentation of cultural heritage. Secondly, it is important to build applications with high level of edutainment, e.g., to convey enough information to educate the user in the historical context of heritage object or site, while making him/her amused and engaged by the presentation. Virtual environments with interactive storytelling enable the user to watch stories on demand. This advantage could turn into a drawback if users do not watch all the offered content. In literature, this is called solving the narrative paradox. It is the conflict between pre-authored narrative structures, especially plot, and the freedom a virtual environment (VE) offers a user in physical movement and interaction, integral to a feeling of physical presence and immersion (Schoenau-Fog 2015). The new IDS method should introduce a motivation factor for users to view the whole offered content.

Lessons learned from the evaluation of IDS application

In order to facilitate the work of interdisciplinary experts on the development of a new IDS method hyper-storytelling, there was a need to familiarize them with typical demands of IDS applications for cultural heritage presentations and serious games. Such applications usually consist of stories, interactive 3D models of cultural heritage (CH) artifacts, and interactive virtual environments (IVE) presenting reconstructions of cultural monuments’ original appearance. The users can virtually explore the IVEs, watch or listen to the stories, and learn about the purpose and historical context of selected objects. The applications are usually online or accessible for mobile download, but they can as well be set up in a museum. Some of them introduce augmented reality elements for combining the digital content with landmarks and on-site elements.

User experience evaluation studies of existing IDS applications have shown the following major drawbacks:
  • Stories are too long to keep the attention of users.

  • Users have problems with navigation in IVEs and do not find triggers for all stories.

  • The content is missing the motivational factor which would keep the user engaged until all of it is explored.

  • There is too much information which is not well structured and makes users bored.

  • The application does not give satisfactory user experience to all audience target groups.

  • Serious games for cultural heritage are too easy or too difficult for playing.

The sample IDS application: White Bastion

White Bastion is a fortress overlooking the city of Sarajevo. It has been guarding the access to the city since the medieval period. Through its history, it has changed appearance several times.

4D virtual presentation project (Rizvic et al. 2016) introduces Internet users with the history of the fortress, its appearance from medieval, Ottoman, and Austrian-Hungarian period till present times, through interactive digital stories. This application has been selected as a sample IDS application for introducing the interdisciplinary team of experts, and their experience developed into the guidelines for interactive digital storytelling presentations of cultural heritage (Rizvic et al. 2016). The following paragraphs summarize the guidelines.

The White Bastion application structure is shown in Fig. 1. The user can watch on demand ten digital stories and explore six interactive virtual models of the fortress. The intro story presents the overview of the content implemented inside the application. Stories about medieval, Ottoman, and Austrian-Hungarian period offer more details on events and characters related to the fortress in those times. Some of the interactive virtual environments also contain stories about particular parts of the fortress and its inhabitants. They also contain models of digitized archaeological findings from the site with their virtual reconstructions. This application can easily be set up in the museum, possibly next to the artifacts found on the site, which will add to the visitors’ experience and not influence their perception of the application itself, nor the proposed guidelines.
Fig. 1

White Bastion project structure

Initial user evaluation study has shown some drawbacks in the concept of White Bastion IDS application. Most of the users have not seen all digital stories. Some of them could not orientate in interactive virtual environments and find stories inside them. Although the majority of participants have appreciated presented IDS concept, particularly the narrator character, the eternal soldier of the fortress, they drew our attention to possible improvements. As the initial user evaluation has been conducted with a variety of user target groups (Bosnian, non-Bosnian, different computer literacy, different ages, and professional backgrounds), we considered as useful to evaluate the same application with interdisciplinary experts. This evaluation will identify the flaws of the concept and contribute to new IDS method development.

The interdisciplinary team

A common issue in new online media research is that it is constrained to just one scientific field. Interactive digital storytelling, as a novel form of media communication, needs to be considered by experts from all relevant fields in order to develop a new methodology that can be appreciated by all user target groups. Here, we present arguments for engaging particular multidisciplinary experts in our research team. Our experience in developing multimedia cultural heritage IDS has led us to identify specific expertise and knowledge areas to address diverse challenges in this field. Following sections describe specific expertise areas required for the multidisciplinary IDS development team. The structure of the multidisciplinary team is inherently flexible as one person can have more than one expertise, and more than one people can share the same expertise. The specific team structure depends on production demands and available resources.

Computer science

Computer science is a point of synthesis for all team members’ contributions. The computer scientists will implement the new IDS method on Internet media. In the method development phase, they gather information and recommendations from remaining team members and decide with them which of their recommendations should be incorporated into the final research product.

Pedagogy

One of the main goals of the new IDS method is to transfer knowledge to the user. It is not feasible to discuss the role of IDS or technology in general in the classroom without an understanding of teaching theories and principles. Introducing IDS in the teaching process should be done in such a manner that it enhances learning, not hinders it. The role of educators is to provide guidelines for creating IDS appropriate for the specific teaching situation and students and also to offer modalities of introducing IDS in the whole teaching process.

Psychology

Cognitive psychology offers a large set of tools to be implemented in this research to obtain the highest level of learning the presented information. Modality principle is a subject of multiple cognitive theories. It is introduced to the design of multimedia to be in accordance with the cognitive architecture (Mayer 2009). The hyper-storytelling method will be developed and evaluated following suggestions of this expert who will identify its advantages over existing IDS methods.

Communicology

Internet media has its rules and laws for presenting information. The communicologist contributes to this research by suggesting the most appropriate ways of adjusting the IDS to this media, with the goal of obtaining maximum user immersion into the story/application. The new IDS method should produce educational, fun, and easy to use IDS applications. The communicologist will offer patterns for achieving this goal in a novel way.

Film arts: directing

Interactive digital stories created using the new method contain a large number of film forms. In fact, they are a combination of movies and interactive virtual environments such as computer games. The director has a leading role in the creation of these movies, as well as directing the interaction in the VEs, to obtain the most user-friendly combination and enable the user to learn the presented information.

Literature: storytelling

The writer of interactive digital stories scenarios is adjusting his storytelling methodology to the Internet media. Now he has on his disposal an enhanced set of tools, instead of pure text, to tell the story. In collaboration with the director and other team members, he decides on the narrative method, characters, and user interaction, to raise the attractiveness for the users.

Visual arts: graphics design

Graphics design expert is in charge of the visual appearance of the IDS application, starting with the user interface until the picture element distribution and shot composition. He needs to define a set of rules to be applied to the IDS application to make it visually appealing for the user, who should interact with it in an easy and natural way.

Human–computer interaction (HCI)

For a long time, the success of products which involve interaction has been measured by the quality of the user experience. There are different existing methods and approaches for user experience evaluation, some of them general, some linked to a specific application domain. The expert from this field will develop and apply a new method for user experience evaluation dedicated to the IDS. Through a set of user studies, we will assess the advantages of the proposed methods. The evaluation will encompass the quality of interaction and information perception, and focus on the following heuristics important for digital storytelling: user immersion and edutainment value.

The user experience evaluation study

We conducted several user experience evaluation studies of the White Bastion application. The initial user experience evaluation results (Rizvic et al. 2016) have shown that users appreciated interactive storytelling more than a linear one, empathized with narrator character, and learned information about this cultural monument in an attractive and immersive way. The main drawback of the application was, according to them, the difficult navigation in IVEs which prevented them from finding triggers to all digital stories.

The psychologist has evaluated the White Bastion application from the aspect of cognitive load theory (Sweller et al. 2011), integration of cognitive load theory and concepts of human–computer interaction (Hollender et al. 2010), and process model of hypertext reading (DeStefano and LeFevre 2007) applied to hypermedia. He divided his observations into macro and micro levels. Appreciating the home page as an “organizer” which would decrease the cognitive load as also recommended in previous evaluations (Chalmers 2003), he noted the possibility of esthetic and functional improvements to make the page more dynamic and attractive. The possibility for users to choose their way through the presented information was recognized as respecting the segmentation principle which contributes to the optimization of the intrinsic load. Rather long loading time of interactive virtual environments he qualifies as a decrease of the presentation’s usability. On the level of digital stories, he appreciated the possibility for users to control the presentation. Finally, he noted redundancy in information presentation on occasions when the same information is presented visually and also in the narration simultaneously. These concurrent presentations compete and increase cognitive load.

The communicologist stated that in the modern American journalism there are two kinds of genres: news and story. According to Quintillian (2006), in order to be communicable, the presentation of information needs to contain three elements: to have a logical foundation, to be attractive, and to be ethical. She found that White Bastion (as a novel way of presenting information) invokes curiosity founded on known denotations and motivates users to stay with it until the end. The presentation could be enriched by adding legends or personal experiences of the narrator-soldier character, such as love, prayer, or feelings. She recommends the addition of sign language interface for persons with special needs.

The graphics designer has noted the lack of unique visual style for the user interface of the website and design of digital stories’ elements. The whole presentation should have had a logo, a coherent font selection, and navigation and screen elements. He appreciated the esthetics of video production. An intro sentence or short sequence should have been added to introduce users what they could expect from the presented content. Virtual environments could have been more realistic with better illumination and rendered quality.

The story writer states that the main quality of interactive information presentation in White Bastion application is entering the creative field of possibilities, which enables us to follow the already established reflexes of the recipient who is an experienced consumer of cyber contents and interactive communication and offer him/her to gain knowledge in such familiar way. Drawbacks are in not implemented possibility to further use creative tools for a pageant of different situations from the story through film or visual arts.

HCI expert has performed the heuristic evaluation of White Bastion application (Boskovic et al. 2017) using an instrument based on well-established ten principles of interaction design (Nielsen 1995). The resulting recommendations concern improvements in navigation and user interaction with the clickable objects in VEs. It has been observed that the users did not recognize that they can click on 3D models of objects and the picture of the soldier, because the objects were not suitable emphasized.

Pedagogy expert was instrumental in accommodating pedagogy principles as heuristics in the usability evaluation (Crozat et al. 1999). In order to attain learning outcomes, it is necessary to synchronize educational content with narrative and user navigation adequately. For example, flaws in interaction will prevent a user to see a page with a detailed reconstruction of the artifact providing intended educational content. Similarly, if it is not obvious for a user to click on a picture of a soldier, he will fail to see relevant video story.

The film director considers that being the White Bastion implemented with the minimal budget is not noticeable on the implementation of the project. The content is interesting, coherent, and systematic. He appreciates the digital stories as poetically stylistic and the excellent performance of the actor-narrator who was interpreting historical facts in location in the real world. All contributing elements such as text, costume, and rhythm should be appreciated. The application succeeded to recreate the atmosphere of the past times in a way attractive to the modern spectator. The general remark is the absence of stylistic unity of the “gestalt,” particularly a visual one, being it a basis of the media we are dealing with. The visual aspect of the level of information theory is the first step in keeping the consumer, as it creates the final first impression.

Guidelines for interactive digital storytelling

As Denard established the London charter (2012), internationally recognized principles for the use of computer-based visualization by researchers, educators, and cultural heritage organizations, we present a set of guidelines for interactive digital storytelling presentations of cultural heritage. These guidelines are provided by interdisciplinary experts from computer science, visual arts, literature, film directing, psychology, social sciences, and human–computer interaction after their evaluation of a sample IDS application White Bastion.

The psychologist concludes that for IDS to fulfill the usability criteria, we need to decrease unnecessary cognitive load, adjust the intrinsic and increase the relevant load. According to learning principles derived from cognitive load theory, a digital story will be more educationally efficient if:
  • Both text and pictures are used, instead just text (multimedia principle).

  • Text and pictures are temporarily integrated (principle of divided attention).

  • Graphics content is used with narration instead of written text (modality principle).

  • One information is not presented in two or more forms (redundancy principle).

  • Key information is emphasized within information organization structure (signalization principle).

  • The unnecessary material is excluded (coherence principle).

  • The material is presented in segments, and the user has control over them, instead of a linear structure (segmentation principle).

  • For beginner users, all kinds of organizers are provided.

  • The application is easy to use.

The communicologist has a different opinion regarding redundancy. She states that qualitative redundancy should be introduced in IDS. She recommends the hourglass narration structure for storytelling. The information should be segmented into short structural units, connected similarly to 1001 night stories, constantly keeping the user’s attention, even attracting him to come back to the story. Keywords for the new IDS concept should be attractiveness and edutainment value. Apart from the narrative, very important are the richness of picture (colors) and sound, significantly contributing to the attractiveness of the application.

The graphics designer emphasizes the importance of user experience quality. The user interface should contain movement and animation to enrich the overall satisfaction. Design trends and technologies should be followed and applied. Lately, parallax scrolling has shown great results for interactive storytelling applications. The users appreciate to be lead through the presentation without investing much effort, but we need to leave a detail or two for those who like exploring. Historical storytelling should be enriched with a human element (for example hand drawing) to create some warmness and soften the communication. Unique visual style of all used elements (website, individual stories, live footage, text, and illustrations) enhances user satisfaction.

The story writer has described interactive perspectives of digital storytelling through the following elements:

Storytelling Art of storytelling follows socio-cultural and techno-esthetic changes. The digital revolution has opened new possibilities for storytelling techniques. Digital storytelling is becoming a creative combination of accessible technologies enhancing the traditional approach through the unity of picture, sound, and movement with the narrative, in order to transfer the story message.

Multimedia narrative Multimedia is a result of a combination of text, picture, sound, photograph, animation, movie, and other types of new media, in cyberspace also the hypertext. It is extremely important to arrange the narrative aspect with visual, interactive, and compositional elements in a functional entirety. The content should not have a unique flow of reading/watching/listening, but the user should be able to determine it dynamically.

Process model Narrative content is open for interaction and connecting various media elements in one entirety. The most important task in developing this new media and storytelling expression is the definition of an efficient process model.

Narration visualization Design process is never the idea of only one of its structural elements, but a process, a set, a coherent thought about external and internal, detail as well as a whole. Digital idea design means conception of visual style content components, derived from the demands of narrative. The visual component is nowadays a foundation of human communication, the same and even more than the text.

Interactive narrative The goal of interactive narrative is not to author the story, but to offer the contextual building blocks and environment where narration could be discovered or built by the user. The key concept of interactive narrative is the ability of users to make decisions on the narrative.

Goals of interactive narrative The successful interactive narrative is designed according to the user needs to provide him/her a pleasant and inspiring experience. This includes all experience aspects (physical, sensual, cognitive, emotional, and esthetic). The first step in creating such a narrative is a clear definition of the main message and emotion of the story, as an often neglected category of narrative aspect.

Structure of interactive narrative Interactive project starts with developing a concept which connects the user experience and intellectual involvement with the user interface and contents. The story is experienced through its content elements instead of as a linear narrative. This includes the creation of a structure in which the content components will be arranged to form the complete narrative user experience.

Motivation in interactive narrative The curiosity of the user is a moving force through an interactive story. It motivates him/her to pass all steps of interaction.

Interaction design User experience could be divided in 3 aspects: the form—dealt with by graphics design through the creation of a visual language for communicating the content; behavior—shaping user’s behavior towards the story, its form and content; content—created by animators, sound artists, and information architects.

Pedagogy expert is responsible for selecting an appropriate pedagogy strategy depending on the context, and more specifically on learning situation and educational content. Educational IDS application development demands close co-operation between story writer, interaction designer, and pedagogy expert. Pedagogy and psychology experts together ensure accordance with the cognitive architecture, mainly the modality principle. Modality principle is instructional principle contributing to increasing the capacity of working memory space. According to the theory of cognitive load, the primary reason for failure in learning is a limitation of working memory to process information simultaneously. The effect of modality is the most studied in the field of multimedia design.

When presented with multiple sources of information users must split their attention, due to a heavy cognitive load. Modality principle offers a solution in combining animation with the speech narrative involving both visual and audio information channels. Working memory capacity is increased by using more than one modality of content presentation (Mayer 2009).

The HCI expert states that, based on ten Nielsen’s usability heuristics (1995), IDS applications should provide users with a sense of control and consequently contain information on navigation in VEs, emphasize trigger objects for certain actions, and enhance integration of interactive 3D geometry and narrative content. For successful user evaluation of IDS applications, Nielsen’s heuristics should be extended with evaluation of experience related to content (content itself, personalization, strategy, presentation modes interconnection), as well as experience related with navigation through the story and within interactive virtual environments.

The film director

Since the discovery of the first camera, film art has been founded on a tradition of novels, drama, and performing arts. After that, the sound was introduced, followed by music. Various elements joined through the development of film technology, but film art still remained the same as at the beginning of telling a story. Interactive digital storytelling needs to inherit all elements taken over from the film, to follow the film language syntax and grammar, to appreciate scenario as a movie on paper, to respect film’s internal logic and convey its message. If a movie director were asked the difference between film and theater directing, he would say it is the same except the tools are different, according to the media. This applies to IDS as well. Furthermore, such a novel media brings us an opportunity and a challenge to become pioneers of its poetics.

Particular recommendations for the new IDS method would be the following: stories have to be short, informative, and dynamic; structure should be defined by a content editor(s) instead of a director; actors should be used to enrich documentary information and add emotion to it.

Computer science experts emphasized the importance of measuring immersion and edutainment value of IDS applications, as a quantitative parameter for their evaluation. They state that a crucial element for IDS applications, particularly those which contain interactive virtual environments, is to solve the narrative paradox and motivate users to explore all offered content completing the story. Interactive virtual environments introduce an additional immersive aspect to interactive digital stories, as users have the possibility to browse recreated cultural monuments and watch related stories inside them. IDS applications need to be portable to all platforms, desktop, and mobile devices.

Adding up all presented considerations, we can summarize them in the following guidelines for IDS presentations of cultural heritage:
  • Engage professionals for all content creation fields.

  • All content has to have a unique visual identity.

  • Use multimedia and virtual reality.

  • Divide content in sub-stories which can be watched independently.

  • Stories should be short, dynamic, and informative.

  • Use characters to communicate emotion and raise edutainment value.

  • Introduce motivation factor to solve the narrative paradox.

  • Create IDS application to be platform independent.

According to these guidelines, the hyper-storytelling IDS application development process can be described by a workflow diagram in Fig. 2.
Fig. 2

IDS workflow diagram

The diagram shows that the development of hyper-storytelling application consists of preproduction, production, and post-production stages. This is a common workflow for digital production process (Kerlow 2009), but we extended it with some elements particular for IDS, and all different expert roles and responsibilities described earlier. In preproduction, the producer, director, and visual artist should agree on the scenario and visual styling of the application. Involvement of pedagogy and HCI experts in this phase is vital for the successful outcome. All production planning activities (actors casting, location scouting, team members selection, budget planning) are performed in this stage. The production will be performed according to the scenario and storyboard, the main results of this stage. Production stage includes all assets creation (music, illustrations, footage, computer animations, 3D models), web design, and design of interactive virtual environments. In the post-production stage, all results of the previous stage are put together through editing and implemented on the website. Evaluation is performed during the whole process, starting with the prototype and continuing even after deployment of the final product. Evaluations are designed to improve the user experience and also to assess achievement learning outcomes.

Exploring the challenges

Working in a multidisciplinary team highlighted different opinions on benefits of the multimedia cultural heritage applications and also priorities in IDS presentations. All team members agreed on evident significance, but opinion on objectives and how to accomplish them were quite diverse. These differences were highlighted in the educational context: either within a museum or standard classroom.

Studies evaluating benefits of the introduction of cultural heritage IDS in the classrooms researchers usually address efficiency and influence on learners and the learning process. The benefits for both aspects have been confirmed in the past, summarizing the results of several studies showing the ability of serious games to engage both young and older learners, both experienced gamers and non-gamers, and showing the efficacy of the game (De Freitas and Liarokapis 2011).

Our primary objective was to explore opinion on benefits and hindering of using multimedia cultural heritage applications in the educational process and compare different user groups with respect to their involvement in the educational process.

The user evaluation is addressing the standard benefits of multimedia as motivation, increased understanding and learning, and also disadvantages such as technical problems, learner motivation, and readiness (Coomey and Stephenson 2001). We decided on the self-selection of users, with emphasis on their reliability, and openness to convey negative opinions. In order to ensure that the responses represent a diverse cross section of respondents and to establish the validity of survey responses, we included questions for relevant demographic data: age, professional background, and link to education (Lazar et al. 2010).

Experience of different attitudes within the multidisciplinary team motivated us to explore what users think about balancing the liberty of artistic expressions, on one side, with the fidelity towards scientific facts, on the other side. The level of fidelity is not linked only to artistic freedom; it could also be affected to provide adaptation of the content to different educational levels.

Procedure

The evaluation involved 49 participants. Presentations of multimedia digital heritage are intended for a broad audience, and we aimed to balanced different user types regarding their professional background (art, engineering, and humanities). We were interested in the difference in opinions of participants who have a role in education: students and teachers, and also participants without a role in the educational process (Table 1).
Table 1

Participants’ demographic data

 

Number

Percent (%)

Students

12

25

Teachers

18

37

Others

19

38

Professional background

  

 Engineering, math and science

19

39

 Humanities

11

23

 Arts

9

19

 Other

9

19

Age

  

 18–29

17

35

 30–50

20

40

 > 50

12

25

Our target group was participants who had experience as users of IDS cultural heritage multimedia applications developed by Sarajevo Graphics group: the White Bastion presentation, or Kyrenia and Bridges of Sarajevo. After watching the IDS, the users needed to spend an additional 10–15 min answering the web-based post-interaction questionnaire.

Questionnaire

The questionnaire used is very simple. After the introductory part with data for user profiling, main part for contained comprises three sub-scales addressing: (A) importance of IDS for education, (B) features of IDS contributing to success, and (C) IDS features relevant for motivation of users, containing a total of 9, 8, and 4 Likert items, respectively. Likert scale items were defined as straightforward statements with positive logic; repetitions were avoided.

Results

Results are analyzed separately for different user profiles regarding their role in the education process (Teachers, Students) and “Others” for participants not involved in the education process. There were 18 teachers, 12 students, and 19 others.

Table 2 shows frequency distribution for questions A1 to A9, where participants are marked T—teachers, S—students, and O—others. The level of importance could be from 1—not important at all to 5—very important. As it can be seen from the Table 2, the frequency distribution is asymmetric (with negative skewness value) in all three groups.
Table 2

Frequency distribution of answers for the first section of the questionnaire (Section A)

Question

Not important at all

Not important

Neutral

Important

Very important

T

S

O

T

S

O

T

S

O

T

S

O

T

S

O

A1

0

1

1

0

0

0

0

1

0

5

7

5

13

3

13

A2

0

0

1

0

1

0

0

1

0

6

4

6

12

6

12

A3

0

0

1

0

1

0

2

2

2

11

5

5

5

4

11

A4

0

0

1

0

0

0

0

1

1

7

5

5

11

6

12

A5

0

0

1

0

0

0

0

2

1

9

4

2

9

6

15

A6

0

0

1

0

0

0

0

2

1

10

3

3

8

7

14

A7

0

0

1

0

0

0

2

3

1

5

3

6

11

6

11

A8

0

0

1

0

2

0

1

1

0

9

3

4

8

6

14

A9

0

1

1

1

0

0

0

3

0

8

1

2

9

7

16

The “Teachers” evaluated the following aspects as important and very important: “Usage of IDS in teaching of history,” “Usage of IDS in teaching of art,” “Access to cultural heritage from remote locations,” “Virtual reconstruction of cultural heritage,” and “Students’ motivation to independently explore history.” The “Students” evaluated the following aspect as important: “Access to cultural heritage from remote locations.” The “Others” evaluated following aspects as important and very important: “Usage of IDS in the teaching of history,” “Usage of IDS in the teaching of art,” “Synergy of multidisciplinary knowledge,” and “Promotion of cultural heritage in tourism.”

Table 3 shows frequency distribution for questions B1 to B8, related to the features of IDS contributing to success. The level of importance is reported in the same manner as in Section A (1—not important at all to 5—very important). The frequency distribution is also asymmetric (with negative skewness value) for all three groups in Section B. The participants in this section were asked to evaluate the level of importance of features of IDS contributing to success.
Table 3

Frequency distribution of answers for the second section of the questionnaire (Section B)

Question

Not important at all

Not important

Neutral

Important

Very important

T

S

O

T

S

O

T

S

O

T

S

O

T

S

O

B1

0

1

0

1

1

1

2

5

2

13

1

7

2

4

9

B2

0

0

0

3

2

4

3

6

2

10

1

7

2

3

6

B3

0

0

0

0

1

0

0

2

1

12

6

5

6

3

13

B4

0

0

0

0

1

0

1

1

1

10

6

8

7

4

10

B5

0

0

0

0

0

0

0

2

0

7

3

4

11

7

15

B6

0

0

0

0

0

0

1

1

0

10

5

6

7

6

13

B7

0

0

0

0

0

0

1

1

0

7

6

4

10

5

15

B8

0

0

0

0

4

0

3

0

1

7

3

4

8

5

14

The Teachers evaluated the following aspects as important and very important: “Help with navigation in the application” and “Technical quality.” The Students evaluated the following aspects as important and very important: “Visual implementation” and “Fidelity.” The others evaluated the following aspects as important and very important: “Technical quality,” ”Visual implementation,” and “Fidelity.”

Table 4 shows frequency distribution for questions C1 to C4, related to user motivation.
Table 4

Frequency distribution of answers for the third section of the questionnaire (Section C)

Question

Not important at all

Not important

Neutral

Important

Very important

T

S

O

T

S

O

T

S

O

T

S

O

T

S

O

C1

0

1

0

2

0

3

1

3

1

15

6

7

0

2

8

C2

0

0

0

1

2

0

1

1

1

12

6

12

4

3

6

C3

0

0

0

0

0

0

2

1

4

9

8

8

7

3

7

C4

1

1

1

9

4

8

6

2

3

1

2

2

1

3

5

The Teachers identified the following aspects as important and very important: “Time needed to explore the application” and “The importance of the application for learning.” The Students assessed that “The importance of the application for learning” influences the motivation the most. The others also evaluated the time as very important for the motivation.

In order to test statistical significance in the difference in frequencies, Chi-square test was performed.

As it can be seen in Table 5, there is the significant statistical difference in frequencies of answers between all three groups. It is obvious that the Teachers have the most positive attitude towards usage of the IDS in the classroom. The Students express less confidence in introducing the IDS in teaching, and the participants not involved in the teaching process are the most skeptical.
Table 5

The result of Chi-square tests

Chi-square value

Sub-scale A

Sub-scale B

Sub-scale C

Teachers–students

5.38

6.97

1.65

Teachers–others

23.26

19.47

5.89

Students–others

9.76

8.07

2.67

All tests satisfy p < 0.05

Introduction of IDS in teaching is additional tool for teacher helping them to present educational content in more realistic manner and with more details. This is especially important for abstract and complex content. Therefore, IDS is instrumental in supporting didactic principle of visualization versus abstraction, and didactic principle of activity. Visualization is natural to multimedia, and activity is achieved by students being able to interact, to move, and to influence.

The difference in evaluation results can be easily linked to difference in knowledge and experience about the teaching process. Teachers are in better position than Students and participant not involved in education (Others), to appreciate benefits of IDS. Students are more focused on content that is directly related to process of assessment.

The results can be easily linked to reports by Di Blas and Paolini (2013), where teachers were very positive in evaluating different benefits of using IDS in the educational process, including student motivation, improvement in communication, etc. We have observed the difference between students’ and teachers’ opinions. This can be explained as teachers see IDS as assistance in knowledge delivery since teachers are focused on learning outcomes. The students’ attitude towards the introduction of IDS in the classroom should be assessed in more details. It would be useful to explore how this technology enhancement can be involved not only for knowledge delivery but also for assessment of knowledge. The flipped classroom strategy proved to be successfully supported by the use of multimedia (Bechter and Swierczek 2017), what is in line with our results.

Visualization provides insight in the assessment of answers for each specific item in the main part of the questionnaire, and answers are grouped according to the sub-scales A, B, and C, as presented in Figs. 3, 4, and 5. Distribution of responses for digital stories and interactive digital model are presented in accordance with Heiberger and Robbins (2014).
Fig. 3

IDS evaluation—teachers

Fig. 4

IDS evaluation—students

Fig. 5

IDS evaluation—participants not involved in education

The questionnaire included an additional set of four items addressing the opposition between the artistic liberties and fidelity to technical constraints and historical facts. We included only four questions and put them in the sequence reflecting the level of non-fidelity:
  1. D1.

    “The costumes are not genuine as in the age presented in the story.”

     
  2. D2.

    “The food or tools used in the story were not known in the age presented in the story.”

     
  3. D3.

    “Animations do not comply with technical restrictions.”

     
  4. D4.

    “Some facts presented are not backed by the historical evidence.”

     
The participants’ evaluation of the statements in the group D is presented in Fig. 6.
Fig. 6

IDS challenge: artistic liberty versus fidelity

The sample of users involved in this pilot evaluation displayed foreseen patterns, specifically that historians (Humanities) and engineers highlight the importance of fidelity over artistic liberty. Not foreseen is the strict adherence to facts within the artist population.

Conclusion

In this paper, we presented research undertaken by an interdisciplinary team of experts from computer science, visual arts, film directing, literature, psychology, pedagogy, communicology, and human–computer interaction, aimed to develop a new method of interactive digital storytelling for cultural heritage presentation: hyper-storytelling. The experts offered their insights into concept, structure, and elements which an IDS presentation needs to contain in order to achieve the maximum level of user immersion and offer the most satisfactory edutainment experience. This research shows the need to view IDS as editorial content assembled according to rules of multiple professions, each contributing in its own way to the common product.

In order to include the IDS content as a part of the standard educational process, and bring it into the classroom, some additional exploration of user expectations were measured and analyzed. The participants in the evaluation are grouped according their role in the education process, as teacher, students, and other participants not involved in education. It is significant how teachers are positive in evaluating importance of IDS for education. Students’ answers indicate their motivation to be engaged in digital narratives, but in future applications it would be beneficial to embed knowledge assessment within IDS. Future work of this team will be performed according to the established guidelines. IDS application intended for educational purposes will be further evaluated by measuring both edutainment levels and effectiveness. According to results of these more detailed studies, the new IDS concept will be adjusted and finalized.

Notes

Acknowledgement

Funding was provided by Horizon 2020 (Grant No. 727153).

References

  1. Alexander, B. (2011). The new digital storytelling: creating narratives with new media. Westport, CT: Praeger Publishers.Google Scholar
  2. Bechter, C., & Swierczek, F. W. (2017). Digital storytelling in a flipped classroom for effective learning. Education Sciences, 7(2), 61.CrossRefGoogle Scholar
  3. Boskovic, D., et al. (2017). Measuring immersion and edutainment in multimedia cultural heritage applications. 2017 XXVI International Conference on Information, Communication and Automation Technologies (ICAT), Sarajevo, pp. 1–6.Google Scholar
  4. Burns, K., & Murawski, M. (2013). Object stories: Storytelling and community collaboration. Portland, OR: Demo at Museums and the Web.Google Scholar
  5. Campbell, J. (1949). The hero with a thousand faces (1st ed.). Novato, CA: New World Library.Google Scholar
  6. Chalmers, P. A. (2003). The role of cognitive theory in human–computer interface. Computers in Human Behavior, 19(5), 593–607.CrossRefGoogle Scholar
  7. Coomey, M., & Stephenson, J. (2001). Online learning: It is all about dialogue, involvement, support, and control: According to the research. In J. Stephenson (Ed.), Teaching and learning online: Pedagogies for new technologies (pp. 37–52). London: Kogan Page Limited.Google Scholar
  8. Crozat, S., Hu, O., & Trigano, P. (1999) A method for evaluating multimedia learning software. Proceedings IEEE International Conference on Multimedia Computing and Systems, Florence, Vol. 1, 1999, pp. 714–719.Google Scholar
  9. De Freitas, S., & Liarokapis, F. (2011). Serious games: A new paradigm for education? In Serious games and edutainment applications (pp. 9–23). London: Springer.Google Scholar
  10. Denard, H. (2012). A new introduction to the London Charter. In A. Bentkowska-Kafel, D. Baker, & H. Denard (Eds.), Paradata and transparency in virtual heritage digital research in the arts and humanities series (pp. 57–71). Farnham: Ashgate.Google Scholar
  11. DeStefano, D., & LeFevre, J.-A. (2007). Cognitive load in hypertext reading: A review. Computers in Human Behavior, 23(3), 1616–1641.CrossRefGoogle Scholar
  12. Di Blas, N., & Paolini, P. (2013). Beyond the school’s boundaries: PoliCultura, a large-scale digital storytelling initiative. In T. Leo, L. Spalazzi, P. Ghislandi, & M. G. Ierardi (Eds.), Journal of Educational Technology & Society Special Issue on “Innovative Technologies for the Seamless Integration of Formal and Informal Learning” (Vol. 16, Issue 1, pp. 15–27).Google Scholar
  13. Heiberger, R. M., & Robbins, N. B. (2014). Design of diverging stacked bar charts for Likert scales and other applications. Journal of Statistical Software, 57(5), 1–32.CrossRefGoogle Scholar
  14. Hollender, N., et al. (2010). Review: Integrating cognitive load theory and concepts of human-computer interaction. Computers in Human Behavior, 26(6), 1278–1288.CrossRefGoogle Scholar
  15. Kerlow, I. W. (2009). The art of 3-d computer animation and effects. Hoboken: Wiley.Google Scholar
  16. Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research methods in human–computer interaction. Hochheiser: Wiley.Google Scholar
  17. Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072.CrossRefGoogle Scholar
  18. Liarokapis, F., et al. (2017). Multimodal serious games technologies for cultural heritage. Mixed reality and gamification for cultural heritage (pp. 371–392). Cham: Springer.Google Scholar
  19. Ma, M., & Oikonomou, A. (2017). Serious games and edutainment applications (Vol. II). New York: Springer.CrossRefGoogle Scholar
  20. Ma, M., Oikonomou, A., & Lakhmi, C. J. (2011). Serious games and edutainment applications. New York: Springer.CrossRefGoogle Scholar
  21. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.CrossRefGoogle Scholar
  22. Miller, C. (2004). Digital storytelling: A creator’s guide to interactive entertainment. Waltham: Focal Press.Google Scholar
  23. Nielsen, J. (1995). 10 usability heuristics for user interface design. ‎Fremont, CA: Nielsen Norman Group.Google Scholar
  24. Pagano, A., Armone, G., & Sanctis, E. D. (2015). Virtual museums and audience studies: The case of ‘Keys to Rome’ exhibition. In 2015 Digital Heritage (Vol. 1, pp. 373–376).Google Scholar
  25. Philpin-Briscoe, O. et al. (2017) A serious game for understanding ancient seafaring in the Mediterranean Sea. Proceedings of 9th International Conference on Virtual Worlds and Games for Serious Applications VS-Games, Athens, pp. 1–7.Google Scholar
  26. Pietroni, E., Pagano, A., & Rufa, C. (2013) The Etruscanning project: Gesture-based interaction and user experience in the virtual reconstruction of the Regolini-Galassi tomb. In 2013 Digital Heritage International Congress (DigitalHeritage) (Vol. 2, pp. 653–660).Google Scholar
  27. Quintilian. (2006). In L. Honeycutt (Ed.) Institutes of Oratory.Google Scholar
  28. Rizvic, S., et al. (2016). 4D virtual reconstruction of white bastion fortress. In C. E. Catalano & L. D. Luca (Eds.), Eurographics workshop on graphics and cultural heritage. Lyon: The Eurographics Association.Google Scholar
  29. Salo K., Giova D., & Mikkonen, T. (2016). Backend infrastructure supporting audio augmented reality and storytelling. In S. Yamamoto (Ed) Human interface and the management of information: Information and knowledge in context, Part II, LNCS 9735 (pp. 1–11). Springer, Switzerland.Google Scholar
  30. Salo, K., Zinin, V., Bauters, M., & Mikkonen, T. (2017). Modular audio story platform for museums. In Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion (IUI ‘17 Companion). ACM, New York, pp. 113–116.Google Scholar
  31. Schoenau-Fog, H. (2015). Adaptive storyworlds (pp. 58–65). New York: Springer.Google Scholar
  32. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer.CrossRefGoogle Scholar
  33. Wiberg, C., & Jegers, K. (2003). Satisfaction and learnability in edutainment: A usability study of the knowledge game ‘Laser Challenge’ at the Nobel e-museum. Proceedings of HCI International—10th International Conference on Human–Computer Interaction, Crete, Greece, pp. 1096–1102.Google Scholar

Copyright information

© Beijing Normal University 2018

Authors and Affiliations

  1. 1.Faculty of Electrical EngineeringUniversity of SarajevoSarajevoBosnia and Herzegovina
  2. 2.Faculty of PhilosophyUniversity of SarajevoSarajevoBosnia and Herzegovina

Personalised recommendations