A TUI-Based Storytelling for Promoting Inclusion in the Preschool Classroom

Abstract

Technologies such as Tangible User Interfaces (TUIs) take advantage of the natural ability of children to tell stories, play and explain their personal and social behavior. TUI technologies can be designed to constitute scenarios of technology use for all and thus benefit inclusive schools. Challenges of designing such scenarios in the classroom include distraction of students, acceptance by teachers, and inclusion of students with disabilities. In this paper we focus on investigating the acceptance of a TUI environment, designed for the educational context of creating, sharing and telling stories collaboratively. We present a system as background for an evaluation of acceptance based on the Self Assessment Manikin model. Two groups of subjects participated in the evaluation: a group of HCI specialists, and a group of teachers working in an inclusive educational context. The pilot study with HCI specialists established a baseline showing that the system potentially has a high acceptance rate. The teachers reported in a subsequent study high levels of Pleasure and Arousal while we detected greater variance in the Dominance dimension. Although we do not see this variance as critical, it requires attention for the more complex modes of the system.

1 Introduction

Using Information and Communication Technology (ICT) in the classroom can be beneficial for education due to motivational factors [5]. Tangible User Interfaces (TUIs), which “augment the real physical world by coupling digital information to everyday physical objects and environments” [8], have shown an even stronger effect on engagement and motivation than traditional GUI-based systems, and thus have the potential to promote learning [11].

Challenges of using ICT in the classroom include distraction of students, acceptance by teachers, and inclusion of students with special needs. Regarding distraction, an advantage of a special-purpose TUI over general purpose ICT such as laptops, tablets or smartphones is that special-purpose TUIs seems to have a lower potential to distract students from classroom activities due to a lack of applications for browsing, chatting, etc.

Regarding teacher acceptance, when teachers feel that they do not dominate the technology or that their students are more proficient than themselves, teachers are often reluctant to incorporate technology into teaching activities [4]. As to inclusive education, students with certain special needs require assistive technology to be able to use many types of software and hardware.

In this paper we present and describe a storytelling application that uses the TUI paradigm and that can be used in different school subjects. Our application has been designed using a socially aware approach [1, 2], and addresses the challenges of distraction, acceptance and inclusion. Regarding distraction and acceptance, storytelling is seen as favorable to learning by theories such as constructionism [6]. Mediating storytelling with TUI has been described as “a powerful way to supply the storytelling process with affordances and intuition”, promoting engagement and minimizing preliminary training and learning [10]. Inclusive aspects are treated by the universal design paradigm we considered in the design solution.

In this work, results of the acceptance for the designed storytelling application are discussed based on pilot and actual case studies; acceptance is investigated using as instrument the Self-Assessment Manikin (SAM) form [3]. The paper is structured as follows: Sect. 2 describes the context and method used in the pilot and in the case study, Sect. 3 presents the results of the two studies, Sect. 4 discusses the results, Sect. 5 concludes.

2 The Pilot and Case Studies – Context and Method

The work reported in this paper is based on one pilot study and one case study. The pilot study was conducted with seven Human-computer Interaction (HCI) specialists from our research team called Human-Digital Artifact Interaction Group¹ – InterHAD, and the case study involved eighteen teachers and twenty students of the Children Living Center² – CECI, a day care center for children from six months to six years of age, responsible for children’s care and education during the workday or study day of their parents at the University of Campinas (UNICAMP) in São Paulo, Brazil.

2.1 The TUI Scenario

Figure 1 shows an outline of the system. The main input components are the RFID reader (Selection Controls) and the webcam with microphone (Creation Devices). Output components are the projector/screen and the speakers (Output Devices). The computer/laptop stays visually “hidden” in order to reduce the perceived complexity of the system. The use of RFID technology is related to accessibility considerations similarly to the way considered by Pastel [9].

Fig. 1.

Components of the system – Interaction mode 1: Presenter

This system configuration is called “Interaction mode 1: Presenter” and is one of four interaction modes proposed in the system. The four modes comprise “simple presentation with/without sound” (Interaction mode 1: Presenter), “multimedia presentation with animations and audio” (Interaction mode 2: Storytelling), “scripted multimedia presentation” (Interaction mode 3: Scriptwriter), and a mode that uses scripted multimedia presentation with audience-sensor feedback (Interaction mode 4: Scriptwriter Plus). The four interaction modes are intended to be used by people of different age groups, e.g. children and teachers. In this paper we focus on the presentation mode which has been designed especially for users with low technical skills such as young children from 4 years of age.

Regarding inclusion, the application uses an RFID reader and a webcam as input and a projector as output. Physical objects of interaction include RFID cards that can be customized and labeled by the students as well as any object (sketches, drawings, toys, etc.) that can be captured by the webcam. RFID cards might be beneficial to accessibility by enabling a more tangible and less abstract interface [9].

2.2 Participants

Figure 2 shows the pilot study with two master students and five doctoral students in computer science, all with thesis projects in the area of HCI and all with intermediate to advanced knowledge regarding accessibility, usability and related topics.

Fig. 2.

Research team from Human-Digital Artifact Interaction Group (InterHAD)

Figure 3 shows the pilot study conducted with teachers of the CECI center, who are working with children between four and five years of age, including a child with special needs. This child requires continuous assistance from an adult, for example regarding locomotion, fine motor movements, and especially regarding communication, because her speaking ability is very limited.

Fig. 3.

Teachers in activities at the Children Living Center – CECI of UNICAMP

The proposed system was used by some of these students (20 in total), including students with special needs sharing the same environment and activities. Although detailed analysis of the activity with the students is out of the scope of this paper, the activities conducted by the teachers presuppose their prospective use with the children; thus, the scope of this article is limited to the InterHAD group evaluation regarding the proposed system and the teachers activities in their preparation for the system use in the CECI center.

Just to illustrate what has happened with the students, we want to share two situations: the first is related to a little girl (almost 4 years) who, according to their teachers, is shy. However, she discussed the part of the story she wanted to create with her classmates and her teacher, designed the image that represent the part of the story using crayons, used the system to insert her drawing (as will be described later), and, most importantly, told her part of the story to the whole group. The second situation happened with a girl who has cerebral palsy which impedes speech and body movement. In order to tell her part of the story, she accepted help from her teacher for making her own drawing and using the system. When it came to telling her story, she made a great effort to do it. During the activity, she expressed happiness and desire for communication. According to her teachers, the final storytelling part was very difficult for her, especially in the presence of a group of strangers as we were at that moment.

2.3 Method

In both studies, the participants were divided into small groups of three or four and given the task to tell a story through a sequence of scenes (three or four) created by them using different physical resources (markers, colored pencils, clay, paper sheets…) (Fig. 4a), and capturing them through the system (Fig. 4b).

Fig. 4.

Example of a created scene and the system capture process

To complete the task, the participants were asked to execute four sequential activities (Fig. 5): define the general topic to be presented with their respective subtopics; create, with different physical resources, each subtopic as if it were a slide; transfer these slides to RFID cards through the system; and finally tell the story for the whole group.

Fig. 5.

Activities executed by the InterHAD group

Finally, after completing all activities, we asked the participants to fill in a form for measuring the pleasure, arousal, and dominance dimensions of their experience for each sequential activity. The form is a non-verbal pictorial assessment technique, called Self-Assessment Manikin (SAM; Bradley and Peter, 1994).

Figure 6 shows some participants completing the SAM form and the options that the form offers for measuring pleasure, arousal, and dominance.

Fig. 6.

Measuring the pleasure, arousal, and dominance dimensions of their experience

2.4 Materials and Procedure

The technology involved in the designed scenario occupies several different layouts, maintaining the principle of invisibility in terms of not being intrusive to the process of storytelling.

In some installations, the computer is physically close to the reader and therefore visible to users Fig. 7(a), however, the interaction is done through the RFID reader and the webcam, and not with the keyboard or the mouse, so the user is not distracted by the presence of the computer. In other installations, the computer is literally invisible, i.e. hidden for example under a table or in a grocery bag (Fig. 7b). In still other installations, we made an effort to hide other devices such as a monitor (Fig. 7c), or the camera within a lamp shade, (Figs. 7a and b). The intention behind these efforts is to make story and storytelling the focus, placing the technology in the backstage.

Fig. 7.

Different system installations

Figure 8 shows how to interact with the system in mode 1. The user has a set of RFID cards with associated commands, such as: create a new slide; assign a narration or sound effect to a slide; delete the contents of a card. An example of creating a new slide can be seen in Fig. 8. In the first step (Fig. 8a), the user brings the card with the command to create a slide close to the reader. This action causes the system to turn on the webcam. Then the user places previously created content (e.g. a drawing or small figurines) under the camera (Fig. 8b). Subsequently (Fig. 8c), the user brings a card without an associated content close to the reader, and the system takes a picture and assigns the image to the card. Finally (Fig. 8d), when the user brings this new card close to the reader, the system displays the picture taken and plays an associated sound, if it has one.

Fig. 8.

Interaction with the system (Interaction mode 1)

To tell a story in interaction mode 1, it is required to assign different RFID cards to different moments of the history (for example to scenes for the beginning, the middle, and the end of the story) and to use these cards to actually tell the story. The narration can be performed at the same moment of telling the story, or recorded and associated to each card of the scenes.

3 Results

In this section we present the results of the participants’ emotional self-assessment according to the SAM. Figure 9 shows the numerical values we assigned to the possible values of the SAM scales.

Fig. 9.

Self-Assessment Manikin (SAM)

For each activity (Define, Create, Transfer, Tell) we determined the frequencies and the mode for each SAM dimension ((P)leasure, (A)rousal, (D)ominance). In this step, we noticed that not all participants completed the SAM form. Tables 1 and 2 show the result of this step.

Table 1. Frequency table of the InterHAD group.

The question that we wanted to answer is or not here is a statistically significant difference among the four activities (Define, Create, Transfer, and Tell) regarding each of the three dimensions measured with SAM, that is, whether the (P)leasure, (A)rousal and (D)ominance remain similar or change during the four activities previously mentioned.

To answer this question, knowing that not all participants completed the form, we applied a variance analysis that supports different sample sizes and that does not assume restrictions on the data, e.g. normality. Specifically, we used the Kruskal–Wallis Test on the original data set for each dimension resulting in three tests.

If applying the Kruskal–Wallis Test on a certain dimension yields a p-value less than 5 %, then there is one or more activities that evoke different level of (P)leasure, (A)rousal or (D)ominance in the participants. Only in this case, we proceed to apply a multiple range test (Tukey’s Honestly Significant Difference HSD Test) to determine which activities elicit similar emotions among themselves and which not.

As mentioned in Sect. 2, this study is based on one pilot study and one case study. In the following, we will present the results obtained in the interHAD group.

It is possible to observe in Table 1 that smaller modes occur for (A)rousal and (P)leasure in the activities of Transfer and Tell, respectively. However, looking at Fig. 10, we can see that justly these two dimensions present greater variability (larger box).

Fig. 10.

InterHAD – Box-Whisker plot and p-value of Kruskal–Wallis test

The (D)ominance of the activity “Define” is the one with the least variability (smaller box), i.e., this is the point at which the members of the InterHAD group agreed more.

Based on the Kruskal–Wallis Test, it is possible to conclude that for the InterHAD group there was no statistically significant difference between the four activities for each of the three SAM dimensions (p-values are presented in the Fig. 10, all of them are greater than 0.05). That is, there is no significant difference in the level of emotion that each of the four activities of the SAM dimensions evoked.

The results obtained in the CECI center are presented following the following. In Table 2, it is possible to observe that smaller modes occur for (D)ominance in the activities Define and Create. Again, the two dimensions with smaller modes have the greatest variability (smaller box in Fig. 11).

Table 2. Frequency table of the CECI teachers

Fig. 11.

CECI – Box-Whisker plot and p-value of Kruskal–Wallis test

There are four combinations of dimension and activity where teachers had a higher level of agreement, namely regarding (P)leasure and (A)rousal to Create and Transfer content (smaller boxes in Fig. 11).

Finally, it is also possible to observe in Fig. 11 that the teachers did not experience the same (P)leasure and (A)rousal along the four activities. Specifically, we found two similar activities for the dimensions (P)leasure and (A)rousal. In the case of (P)leasure, the first group is formed by the Define activity and the second group is formed by Create, Tell and Transfer. Therefore, it is possible to conclude that the level of (P)leasure evoked during the Define activity is different than the level of (P)leasure evoked during the other activities. Similarly, in terms of (A)rousal, it is possible to observe two groups. The first is formed by the Create and Define activities and the second is formed by Create, Tell and Transfer. Therefore, there exists a difference in terms of (A)rousal when comparing elements of these two groups with exception of the Create activity that belongs to the two groups.

4 Summary and Discussion

The discussion is divided into two parts, the first dealing with InterHAD group and the second related to the CECI center.

For all four activities in the InterHAD group (Fig. 10), we see a greater variability in the dimension (P)leasure and (A)rousal, and a greater degree of agreement in relation to (D)ominance. When analyzing the activities separately, we can see that variability in the response increases with respect to each of the activities in the following order: Create, Define, Tell and Transfer.

One possible explanation for the (D)ominance dimension may be related to the fact that this group of participants are expert users of technology. Regarding the other two dimensions, the explanation may be related to the fact that we presented all possible interaction modes to this group before conducting the activities. This a priori knowledge could have affected (A)rousal and (P)leasure since interaction modes two to four are more advanced and interactive.

With the group of teachers in the CECI center (Fig. 11), we have a slightly different behavior. The greatest variability is in (D)ominance and (P)leasure, leaving (A)rousal as the dimension where the teachers have higher level of agreement. Analyzing the activities separately, we observed that the variability increased in the activities following the order Transfer, Create, Tell and Define.

A similar explanation can be given for the teachers, i.e. assuming more heterogeneous skill levels regarding technology use compared to InterHAD group, the dimension of (D)ominance has greater variability. The interaction mode 1 was a novelty for teachers, and they did not know the most advanced interaction modes of the system, which could explain the results obtained in the (P)leasure and (A)rousal dimensions. The Define activity might evoke different levels of (P)leasure and (A)rousal among teachers, due to the challenge of selecting a unique idea among several proposals. Some people might experience different degrees of pleasure and arousal when working on their own ideas than when working on the ideas of others.

Besides all this, we observed a real desire among the teachers to use the system permanently in the institution with their students. They explicitly expressed the potential of the tool for activities in the context of an inclusive school, and highlighted some elements such as: allow to combine different types of designs and storytelling by the students with varying degrees of skill and physical needs; the use of technology selected for reasons of inclusion, for example the RFID reader that does not require fine motor skills; the “invisibility” of technology, leaving the focus on the story and not the devices; and allow combining of images and sounds created by the students themselves, thus creating a greater degree of motivation among students, and freedom to work all kinds of themes in the stories.

5 Conclusion

In this paper we presented a storytelling system that is based on the TUI paradigm and that targets the promotion of inclusion of preschool children in the classroom. The system supports different storytelling interaction modes that enable various levels of complexity of the storytelling process. An important aspect of such a system is teacher’s acceptance.

As a preliminary approach to acceptance, we presented the results of a pilot study and a case study and analyzed the results of the study participants’ self-reported levels of pleasure, arousal and dominance, using the SAM instrument. The pilot study among graduate students with an HCI research focus established a baseline and showed that the system potentially has a high acceptance rate. Although not explicitly investigated, this study also showed the system’s potential for inclusion, since we received positive comments from the (universal) accessibility specialists among the participants.

The actual case study was conducted with 18 teachers of a day care center who teach children between four and five years, including children with special needs. Results provided evidence of a high acceptance rate among these teachers. The participants reported high levels of pleasure and arousal. We detected greater variance in the Dominance dimension. Although we do not see this variance as critical, the Dominance dimension will require more attention during future activities using the more complex interaction modes of the system. We detected differences in Pleasure and Arousal levels along the four activities within the system (Define, Create, Transfer and Tell). Future work includes establishing and testing hypotheses for these differences, as well as their quantification, relating them to the system design elements.

Other future work includes the use of the additional interaction modes and case studies involving preschool children in order to evaluate the acceptance of the more complex modes of storytelling, the acceptance by preschool children (detailed analysis of the activity with the students will be reported in a future article), and more aspects related to inclusion. We will furthermore perform an analysis using different data sources and instruments such as the principles of Design for Affectability [7]..