Keywords

1 Introduction

Writing and reading are important skills to obtain knowledge, to get these capabilities a different process is required for each of them. One of them is the development of phonological awareness [1], in the case of blind children, this one is a crucial one as one of the more developed senses is the audition [2]. In Spanish, a language in which the formation of words is more connoted for the vowels is a hard task for blind people to create phonological awareness, especially in kids. It is really hard for them to identify phonemes in every word [3].

The present project implements an interactive system, which is intended to improve the development of the phonological awareness of blind children. This system provides no traditional methods to display information, interaction, and integration of senses, with the objective that the children train the phonological awareness through four interactive activities with emphasis in vowels since this is the fundamental base to obtain the reading skills.

It’s fundamental to stand out that the needs of the children are extremely important in this project and the center of the development of the interactive system are them since finally they are the ones who use the system to strengthen their competence.

2 Problem

The process of language acquisition in blind children has a fundamental role in the development of their lives because it is one of the main sources of learning and communication. According to the Spanish National Organization for Blind People (ONCE), “reading and writing is a secondary process of speech, derived from the oral language” [4]. Also, it should be considered that reading and writing are intellectual processes that include aspects beyond the translation of graphemes or the interpretation of phonemes. In order to achieve these competencies, the process begins at different levels, where children acquire different activities that allow them to reach this goal later [4]. Among these levels is the obtaining of phonological awareness.

To develop the phonological awareness in a child must be a pass for the different subprocess, which according to Emilia Ferreiro, are [5]: the syllabic hypothesis and the syllabic-alphabetic hypothesis. Besides, is important to mention that for blind children’s it is a bigger challenge, since it is not easy for them to associate the graphemes with phonemes as a child would commonly do, when they use of their visual and auditory senses [6], because of this the blind children’s have difficulties in the stage syllabic-alphabetic. In this phase, the child must understand that the sound similarity implies similarity of letters and that the sonorous difference supposes different letters, and to associate in a clear way a phoneme for each letter. This is a hard process for a blind child, who can not associate visually the things that they hear. In that way, the blind children present complications in the differentiation between the consonants and vowels, because in the Spanish the vowels are more frequent. This causes children to become confused in their differentiation and to lengthen the learning processes.

3 Methodology

For the development of the interactive system, we follow the methodology Engineering Process Model for usability and accessibility (MPlu+a), proposed by the Griho research group of the University of Lleida (Spain) [7]. MPlu+a adopts user-centered design principles. This model offers a guide for the construction of usable and accessible interactive systems.

MPlu+a breaks down into three basic pillars: Software Engineering, prototyping, and evaluation, but it does not establish a specific method to follow for its development as such. Therefore, it is flexible to the quantity, the type of prototypes and evaluations to be used. It should be noted that the stages of prototyping and evaluation are iterative throughout the development of the interactive system.

4 Preliminary Results

4.1 Users

The students that participated in the project were in pre-garden and garden grades at the Institute of blind and deaf of Valle del Cauca, located in the city Cali, Colombia. Actually, the pre-garden grade is formed by ten children (four girls and six boys), among them, there are two blind girls, one inclusion child (child without disability) and seven children with low vision. In garden grade, there are eight children (four boys and four girls), three blind children, two of inclusion and three with low vision. Moreover, when we approached this last course, it was possible to identify that one of the children, in addition to suffering from blindness, has attention deficit and dyslaliaFootnote 1.

Furthermore, the project counted on the participation of the professors and the speech therapist of the institute, who assumes the role of the counselor and use the system to select the activities that the students will carry out according to their needs.

4.2 Study of the User’s Context

The investigation was made in the installations of “Instituto de Niños Ciegos y Sordos del Valle del Cauca”, specifically in the classroom of pre-garden and garden grades. The user’s context was studied through the technique of direct observation. This research was initiated by visiting the institute to observe the language class and interview people who were in charge and in constant contact with the children, in order to interact with those involved and identify the needs of the users.

Both pre-garden and garden grades, children begin to develop phonological awareness focused on vowels, working with rhymes, tongue twisters, songs, a combination of words and semantic categories. Also applies the invariant method, which has been adopted by the Institute for blind children, the method consists in through pieces (green and square: consonants, red and circle: vowels), the child is able to replace the phonemes of the word according to its classification. For some activities, physical representations of a word (objects) are used, in such a way that the child relates the concept and the word.

4.3 Users Needs

Once the information collected through the observation techniques and interviews carried out with those involved in the context study stage was analyzed, the following needs of the users were detected:

  • Information perception: The blind children’s need a mechanism that allows them to optimize the interpretation of the information obtained through the auditory and tactile sense, unlike the sighted people who perceive 80% of the information through the sense of sight, the blind children perceive it to through your other senses, the stimulus received through hearing, touching and smelling should be clear and frequent.

  • Skill development: The blind children face a process more complex at the moment to learn new concepts, as are the vowels in this particular case since they need to develop and strengthen phonological awareness. This is a fundamental factor in their education because it allows the child to access reading and writing skills, which are significant when thinking about inclusive education.

  • Generation of interest regarding the activity in development: For blind children participation in activities that motivate the use of different communication channels, plays a vital role to motivate their participation in the classes, that is necessary to perform activities like sing, rhymes, stories, and repetition, where children, apart from stimulating their other senses, approach language.

  • Reward during the activity: It is necessary to include the gamification in the classroom and the interactive system to propose, with the objective that blind children can have stimulation in their learning process.

Activities for the Development of Phonological Awareness

Once the analysis of the information obtained in the first phase was done, it was found that the child must acquire the ability to decompose, segment and differentiate the phonemes, skills that can be developed through activities that involve the auditory sense. Therefore it was chosen four activities that were implemented in the interactive system, which were selected taking into account activities that are currently carried out in the institute and other activities that were determined from the investigation of the context and the state of the art, considered influential for the development of phonological awareness.

The activities for the development of phonological awareness that were included in the system, are:

  1. 1.

    By which vowel the word begins [8]: this activity consists in that the students must identify which is the first phoneme of the word that Pistacho has said and introduce the vowel piece corresponding to the first phoneme in the mouth of Pistachio.

  2. 2.

    By which vowel the word finish [8]: this activity consists in that the students must identify which is the last phoneme of the word that Pistacho has said and introduce the vowel piece corresponding to the first phoneme in the mouth of Pistachio.

  3. 3.

    Classification of words according to the vowel [9]: Pistacho will say a set of words that belong to the group of a vowel, depending on their first phoneme, for example, if all the words begin with the A belongs to the group of the vowel A. The child after hearing all the words, they must decide which group it belongs, and put the corresponding vowel piece in the mouth of Pistachio.

  4. 4.

    The vowels sound [12]: Pistacho reproduce the sound of the vowel and the child has to identify the sound and insert the vowel piece correspondent.

The activities mentioned are both pre-garden and garden grade, nonetheless, the activities of pre-garden grade have a level of difficulty more than ones of pre-garden.

5 Design and Developing of the System

5.1 Description of the Interactive System

Based on the identified problem, as a solution, it was proposed the design of a multimodal interactive system called Pistacho, which has two components: software and hardware. The software consists of an application developed for the speech therapist and the teachers of the institute, in charge of the pre-garden and garden grades. The application performs three main functions: (a) control of the activities that the hardware component will execute, (b) registration of children’s information in the database, and finally, (c) allows teachers to know the children answer in each activity.

The component hardware allows interaction between the child and the system in the classroom. The hardware consists of a character from another planet called Pistacho (which gives the name to the interactive system) and suggests to the children the realization of a set of activities for improving topics seen in class. With Pistacho we seek to motivate the child to participate actively in the learning process. Pistacho presents attributes that try to capture the child’s attention, such as their voice, shape, size, and texture in their body. Besides the software and hardware, the child has pieces to interact with Pistacho, these pieces represent each vowel.

Pistacho, explains the children a set of activities to enhance the development of phonological awareness. The activities were identified after realized the analysis of the information obtained in the context study, with which, we discovered that the activities for the development of phonological awareness should implicate the auditory sense.

It is important to clarify that with the implementation of the interactive system in the classroom, it has a purpose to offer support in the development of phonological awareness specifically in the vowels, nevertheless, we are not trying to replace the role of the teacher.

Storyboard

The storyboard explains how is the interaction between the users and the system (see Figs. 1 and 2), the information in the images are in Spanish, but it is planned to include another language such as English. In first instance the teacher select the activity (A) it will send to Pistacho which will explain to the children’s the activity (B), to which the child must answer with one of the vowels that they have to their disposition (C) and Pistacho indicate if it is the right or not (D).

Fig. 1.
figure 1

Storyboard of the user experience part 1 (Color figure online)

Full size image
Fig. 2.
figure 2

Storyboard of the user experience part 2 (Color figure online)

Full size image

5.2 Design and Implementation of Hardware

The design of Pistacho was proposed in the way that has a friendly appearance and attractive for the children, its structure was made in wood “mdf” of 5.5 mm of thickness and it has one meter of height, due to this child doesn’t do a big physical effort and/or cognitive to look the mouth of Pistacho, and 70 cm of width to his easy perception in the place. His mouth is always open, for that the child can deposit the objects; its horns, its tongue, its texture, and its anatomy make Pistacho a tool that will call the attention of the children (see Fig. 3).

Fig. 3.
figure 3

Pistacho.

Full size image

In order to create a hardware system with different communication channels, Pistacho has a layer with furry plush fabric, providing a tactile experience, thus allowing to the children create a mental image of how Pistacho is.

Hardware Electronic Components

For the development of the component hardware, it was used the tool of open code Arduino, through this software we programmed two microcontrollers of the system: the nodeMcu and the Arduino Mega. The Arduino mega is in charge of the controller the decoder plaque of audio Shield Mp3, which is connected to a speaker. This connection is made with the objective of can play the audios that Pistacho reproduce.

The nodeMcu is the one who does the connection to the database which is hosted in the cloud, and it is consulting permanently the status of the same, through requests HTTP. This node has the function of sending to the Arduino the code of audio that is required for the activity that is in progress at the moment, at the same time this plaque must controller the RFID1 and the RFID2, the first one read the tag of the bracelet of the child and the second one read the tag of the vowel that the children deposited in the mouth of Pistacho. After all, the information collected is sent to the database, for later this was consulted for the teacher.

Figure 4 presents a general scheme of the connection and the location of every component inside of the structure. As you can see the RFID1 is ubicated strategically in the mouth of Pistacho, for reading the bracelet of a child at the moment that the children deposited the vowel, and the RFID2 as mentioned earlier is ubicated at the end of conduit with the purpose to read the vowel after that the child the deposited to Pistacho.

Fig. 4.
figure 4

Location of the components inside Pistacho.

Full size image

5.3 Vowel Pieces

The five vowel pieces that the child has to interact with Pistacho, represent each of the vowels (see Fig. 5), thinking about taking advantage of the prominent development of tactile sense that has the blind child, the pieces were made with different geometric figures and textures. Also, each piece count with two sides, one side face A has a special texture to differentiate itself from the others, and the face B, it is written the vowel in braille and in Spanish.

Fig. 5.
figure 5

Vowel pieces.

Full size image

5.4 Identification Bracelet

With the purpose of keeping a record of the answers of the users, each child must use a unique bracelet of recognition (see Fig. 6). Each bracelet has an identification tag, that be read by the RFID at the moment in that the child introduces the hand for deposited the vowel to the mouth of Pistacho. The RFID recognizes the unique code of the bracelet which is associated with the child in the moment of the register, and this way will save all the answers of the child in the database.

Fig. 6.
figure 6

Bracelet of identification.

Full size image

6 Tests

With the purpose of obtaining the ideal interactive system that fulfills with the objectives set. Along the process of development different evaluations were made to each component of the system. The developers of the interactive system made a test of the functions of the system, another test with a usability expert and different tests with the final users, with the feedback of these tests let make improvements to Pistachio in the visual appearance and functionality.

6.1 Functionalities Test

The objective of this test is to verify that the interactive system entire set working correctly, for this the test was carried out by the developers of the system. A total of Nine functionalities was evaluated. It should be noted that this test was made previous to the test with the final users. The result of this test was successful, the nine functionalities tested works correctly to the requirements of the evaluators.

6.2 Usability Test of Software Component

The method used to evaluate the software component was formals experiments, this consists in a controlled experiment and measurable with the user of the test, in this case, the ones who try the system were the teachers of the institute. This method of usability test was made with the objective to prove that the users could complete the task in the software application, in a reasonable time and check that the teachers understood how the system works. In this method, the users made the task in the system while the evaluators saw the interaction and functionality of the system.

The results show that the teachers made satisfactory the tasks, with the exception of two tasks, since it took them more time than the maximum proposed to the start, that shows us that the application has usabilities problems, that we can fix before the final test.

6.3 User Experience Test of Pistacho

This evaluation was made with the group of 12 blind children the “Instituto de niños ciegos y sordos del Valle del Cauca”, and also counted with the participation of three teachers and the phonoaudiologist of the Institute (see Fig. 7), in order to contribute objectively to the improvements of the system from your professional point of view. It’s important to mention that due to time the test was carried out with one child at a time, this is highlighted since in the context the system can work both as a group and individually.

Fig. 7.
figure 7

Teacher and kids using Pistacho

Full size image

Since the tests were performed with children who are in a situation of disability, the evaluation was required to be made in a dynamic and pleasant manner, for that, different strategies and elements were used for this case, once the child used Pistacho, five stars were given to the child on paper, with which they qualified eight questions that inquire about their experience with Pistachio.

Table 1 presents the percentage with stars obtained for each question, being five stars the maximum score and one star the lowest score that the child could assign. Among these questions, the child is questioned about the understanding of the activities, the affinity with Pistacho and the interest and understanding he had about the cards.

Table 1. Qualification assigned by children
Full size table

From the results obtained, one of the things that were evident in the children’s at the time of asking the questions, is that by having at their hands the five stars to qualify the experience of using Pistacho, most of them gave the meaning of one (1) star as a NO, three (3) stars as a DO NOT KNOW and five (5) stars as a YES. Therefore, it is concluded that in questions like No. 4 a big percentage of children gave a rating of one (1) star, in this case, although the question obtained the lowest rating does not imply that this is a bad grade, due to the way the children interpreted giving a star, as mentioned above. On the other hand, in questions such as No. 2, 3, 5, 6 and 7, most children gave a rating of five (5) stars. In Questions No. 1 and 8 there was a greater oscillation in the answers because, being a more open question, some children decided to give four (4) stars or two (2) and complement their answer with a verbal explanation.

7 Conclusions

Through this work, it was possible to obtain an interactive system called Pistacho, composed by a software component and a hardware component. The software component, develop for the use of the teachers, consists of an application that allows the handling of the hardware component. The hardware component, oriented to the blind children, consists of a wooden structure that represents a being from another planet, with electronic components that allow the user (child) to interact naturally with the system. Pistacho aims to support the development of phonological awareness in blind children.

The development of phonological awareness is a necessary step in the process of acquisition of reading and writing, which plays a fundamental role in the education of children. For this, four activities were implemented that improve factors such as the recognition and segmentation of words in phonemes, in this particular case focusing on the vowels, because they are the beginning of the literacy process.

The creation of an interactive system under the principles of Usability and Accessibility Engineering, as was done by implementing the MPIu+a model, which provided a guide for developers during the project research and execution process, to detect the real needs of the target audience, and reach the development of a product that satisfies them.

The implementation of Pistacho also looks to the benefit of the teachers in the institute, providing them with an additional resource that they can use as a complement to their class, which look up to promote the development of phonological awareness of the children’s. In addition, the implementation of Pistacho not only has the objective of consolidating knowledge in a non-traditional way, making use of technologies that allow interaction through different senses, but also to reach better motivation and interest on the part of children, and thus to reduce the gap existing between technology and blind children.

The implementation of Pistacho also searches for benefit of the teachers of the institute, providing them with an additional resource that they can use as a complement to their class, which seeks to promote the development of phonological awareness of the members. In addition, the implementation of Pistacho not only aims to consolidate knowledge in a non-traditional way, making use of technologies that allow interaction through different senses, but also to achieve greater motivation and interest on the part of children, and thus to reduce the gap existing between technology and blind children’s.

One of the challenges in the design of Pistacho was to catch the interest and motivation of the children’s at the time to do the activities, and to reach this, was necessary the use of some game mechanics, the implementation of activities that generate a challenge in the students to finally receive a good feedback, just as it is sought with the proposed activities impose challenges for users to overcome each interaction, with the aim of strengthening phonological awareness in a different way, using activities that are currently implemented by teachers in the classroom, but generating a different dynamic for the acquisition of such knowledge.

In the analysis, it was perceived that children’s with visual disabilities need mechanisms in their environment that allow them to optimize the information obtained through the senses of hearing and touch. For this reason, Pistacho considered the use of different textures, both in the vowel pieces and the structure of Pistachio, in order to provide the child with mechanisms that allow him to recreate in his mind the image of the elements with which he interacts. In the same way, the personification of the Pistachio voice considers different intonations so that the child perceives it not only as a monster but as a friendly, good and sociable character.

In the analysis, the stage was possible to collect information of great value, for the selection and correct design of the components of the interactive system, resultant in the implementation of Pistacho with low-cost technological instruments and optimal functioning, which would allow compliance of the purpose for which it was developed.

Because the MPIu+a model proposes an iterative process, throughout the process of development of Pistacho, different evaluation techniques were implemented that allowed to demonstrate usability problems, the navigation in the application, ergonomics and the feedback of the users, which were corrected at the appropriate time, allowing to achieve a final product that meets the functional and non-functional requirements.