Keywords

1 Introduction

The number of mobile devices has increased significantly in the recent years. Technological advances provided such devices more resources and reached spaces among different users, bringing new possibilities and challenges to the current society. These devices represent the most ubiquitous interactive Information and Communication Technologies (ICTs) in the world [1]. Globally, it is estimated that the number of mobile users will exceed 5 billion by 2019; in Brazil, the scenario is also positive. According to a research conducted by IBGE (Brazilian Institute of Geography and Statistics) [2], the use of mobile phones for Internet access among Brazilians, in 2010, was greater than the use of personal computers. In addition, Brazil was recognized as the Latin American country with the largest mobile phone market in the year of 2015 [3].

In this context, mobile devices, also considered as digital technologies, may be used for a variety of purposes, emphasizing their use in educational area. According to [4], digital technologies can help learners in the relationship with knowledge and extend their cognitive capacity.

Since mobile devices have been used in the educational area, a new learning modality has emerged, namely mobile learning (or simply m-Learning). Several researches and initiatives have been undertaken to implement mobile learning for both formal and informal education. UNESCO (United Nations Educational, Scientific and Cultural Organization), for instance, proposed a set of recommendations for policy-makers in 2013 so that they could better understand what mobile learning is and how its benefits can help and advance to promote education for all. According to this document [1], mobile learning involves the use of mobile technology itself or combined with other Information and Communication Technologies (ICTs), to enable learning anytime, anywhere. Mobile learning also covers efforts to support across-the-broad educational goals such as: effective management of school systems and improve communication between schools and families. Actually, m-learning has enabled the learning development in several ways: (i) people can use mobile devices to access educational resources easily; (ii) people can connect with others, sharing knowledge and experiences; (iii) people can create content both inside and outside classrooms [1].

Other definitions complement the concept of m-learning. Farooq et al. [5] state that mobile learning is a teaching and learning process that uses specific devices anytime, anywhere. Srivastava and Gulati [6] state that the individual, when using m-learning applications, takes advantage of learning opportunities provided by mobile technologies, associating the concepts of technology and mobility. And, Traxler [7] indicates that the term “mobile” is not only a new adjective, qualifying the timeless concept of “learning”.

This modality of learning has provided great expectation in the educational area for presenting benefits related to the flexibility of teaching and training. It presents a flexibility and adaptation to the context of use of the learners, teachers and tutors, being adaptable regarding space and time [8,9,10,11].

Mobile learning applications should also allow access by different users, including the seniors. According to the World Health Organization, the world’s population over 60 years will pass the 2 billion by 2020, a larger amount than that of children up to 5 years [12]. In Brazil, researches show that in 40 years, the number of elderly will triple and will increase from 19.6 million (10% of the Brazilian population) in 2010 to 66.5 million people in 2050 (29.3%) [13].

Thus, mobile learning applications should be developed considering accessibility and usability criteria in order to reduce barriers that may be found by the elderly, since these users may have their physical, mental and learning capacity compromised due their age.

The word accessibility can be used in several scenarios. The term is usually classified as “quality of access”, whether related to the social or technological environment. Accessibility is a dynamic process, associated not only with technological development, but also with the development of society. The term “accessibility” appears in different stages, it changes from one society to the other according to the attention given to human diversity, by that society, at that time [14].

According to ISO 9241-171 standard [15], accessibility is the “usability of a product, service, environment or facility by people with the widest range of capabilities”. Barbosa and Silva [16] showed that usability is the issue related to how well users can use a given functionality. According to ISO 25010:2011 [17], usability is the “degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”.

Considering these premises, the purpose of this paper is to present the results of an accessibility and usability evaluation in a mobile learning application called Crossword Learning. The results showed that 80% of the participants would use the application in their daily lives, and the user test allowed the categorization of some issues faced by users when using Crossword Learning. The issues raised can be used to better develop m-learning applications or mobile applications.

The remainder of the paper is organized as follows: in Sect. 2, we examine the previous works related to our research. In Sect. 3, we describe the application Crossword Learning and its prototyping process. In Sect. 4, we present the method used. In Sect. 5, we present the procedure used for the evaluation. In Sect. 6, the results are discussed. Finally, conclusions and future work are presented in Sect. 7.

2 Related Work

Other researches related to the development and use of mobile applications with focus on elderly users have been conducted in the academic environment. Melo et al. [18] analyzed the use of an application called Eldernote. Based on a user profile analysis, the application focuses on the use of notes and reminders for older people. In the study, 21 participants were invited to use the Eldernote for one week. The researchers evaluated the actions taken during the period of use and then conducted an interview with the participants. Finally, the results showed a higher satisfaction of the elderly users, because the application was easy to use. The study showed that considering aspects of accessibility and usability in the design of the application provide easier use for the elderly and also their independence.

The study conducted by Balata et al. [19] focused on the use of touch-sensitive keyboards and their use by elderly users. Initially, a survey was conducted with five participants to gather qualitative data. Then, they applied an online questionnaire to 118 participants for the gathering of quantitative data. Considering the data obtained, the users created KoalaPhone Launcher, a keyboard developed concerning the needs of elderly users. Finally, they concluded that mobile devices designed for the elderly should not have complex features, but must support them through user interfaces that are efficient and appropriate to their needs. If the design is created based on the user’s need, the elderly can accomplish more complex tasks in the cell phone.

Similarly, Arab et al. [20] present in their study preliminary results about the use of PhonAge, an adaptable and accessible interface for elderly users. The authors were able to conclude that the proposed interface may make older users more independent, considering the results of an evaluation performed with 20 participants.

In a related perspective, [21] carried out a study that aimed to determine the attitudes of the elderly in relation to the use of mobile phones and the barriers to their use. The study was conducted with 328 subjects and showed that the majority of the participants had unfavorable attitudes regarding the use of the cell phone, although they used it daily. Among the barriers presented in the study we can mention: difficulty in selecting the buttons or using the keyboard, prolonged search for icons and difficulty in identifying them, font type and color, low voice quality and others.

Finally, the study of Al-Razgan et al. [22] present a set of guidelines and design recommendations for touch-based mobile phones for the elderly. These guidelines were created based on a systematic review of the literature. The recommendations are organized in three dimensions: (i) Look and Feel, (ii) Functionality and (iii) Interaction.

3 Crossword Learning: A Mobile Learning App

The Crossword Learning prototype emerged during the course of User-Computer Interaction II: Practice, offered at the University of São Paulo (São Carlos – SP), Brazil. Before we get to the final prototype, presented in this Crossword Learning article, we follow the steps below:

  1. 1.

    Application of an online questionnaire with 28 elderly people;

  2. 2.

    Interview with two experts about accessibility and usability in mobile applications and elderly;

  3. 3.

    Development of three preliminary interface designs that have undergone expert evaluation.

  4. 4.

    Selection of the final prototype and heuristic evaluation performed with two experts in accessibility and usability;

  5. 5.

    Redesign of the prototype from the results of the heuristic evaluation;

  6. 6.

    Test with 10 users from the final prototype of the Crossword Learning.

Although all previous steps have been performed and recorded, this article aims to present only step six (6). However, it is important to note that the application was developed based on the opinion of the users themselves, through the online questionnaire (step 1).

We applied a questionnaire with elderly users in order to understand their real interest in educational applications. The questionnaire was applied online with 28 users aged between 50 and 85 years. We asked: (i) the age of the participant; (ii) the frequency with which he/she uses cell phones; (iii) the activities that usually take place on mobile devices; (iv) the interest in mobile applications. Some questions were more specific and enabled us to create Crossword Learning. The obtained data showed that 89.3% of participants believe that mobile devices can improve people’s education. Participants stated they would like to learn about several topics, as shown in Fig. 1.

Fig. 1.
figure 1

Responses given by participants on topics they wish to learn using mobile devices.

Full size image

After the answers given by the participants and the tabulation of the data be performed, we proposed Crossword Learning.

Crossword Learning is a mobile learning application that focuses on senior users and seeks to teach on several subjects: History, Geography, English, Portuguese, Mathematics and Computer Science. Different levels of difficulty (easy, medium and difficult) are also addressed. The focus of the application is not to be gaming, but rather, that through the crossword, the user can know and learn about different subjects, always having a context for each one of the activities (Fig. 2).

Fig. 2.
figure 2

Crossword Learning application

Full size image

When performing crossword activities, the user earns points and climbs to more complex levels. The user may have access to his/her score and see how much he/she has learned from the application through statistical data. In addition, the application allows the user to participate in competitions and share their results within their internal network of friends. We can highlight the following application features: (i) presentation of various contents by means of video, text or audio; (ii) registration of people in an internal social network; (iii) share the results of activities/phases with your network of friends; (iv) monitoring your level of learning through personal ranking.

Having defined the general ideas of the application, we structured the prototype in Justinmind [23], a software for web and mobile applications prototyping. The prototype was focused on verifying the first feature “presentation of various contents by means of video, text or audio” and, with the structured prototype, accessibility and usability evaluations were performed.

4 Method

Next we discuss the methodological procedures, as well as the data collected and the usability and accessibility tests performed for Crossword Learning evaluation.

4.1 Study Participants

In our study, we considered older users who are not necessarily “seniors”, but those who can benefit from the proposed application. The usability and accessibility tests were performed with 10 participants aged from 45 to 85 years, all residents of Catanduva (SP) in Brazil. From the 10 participants, 7 were women and 3 were men, aged average 63.8.

Before using the prototype, participants were invited to answer a characterization form to get information about users’ experience of using mobile applications. Participants were divided into two groups:

- Group of beginners (GB): participants who had less than 1 year of experience with mobile devices – 3 older users.

- Group of experienced (GE): participants who had more than 1 year of experience with mobile devices – 7 older users.

Table 1 shows some characteristics of the participants of our research as well as the division into groups for better analysis of the data.

Table 1. Number of participants in each group, according to their characteristics
Full size table

4.2 Tasks Undertaken

Participants were invited to perform nine tasks in the Crossword Learning prototype. The mobile device used was a SM-SM-J510MN 5.2 in. screen, running Android 6.0.1. The tasks allowed the participants to go through all the screens of the application and to try out their functions. Such tasks were divided as follows:

 

Task 1 -:

Enter the system. Enter username (1234) and password (1234).

Task 2 -:

Click the Menu to see the available options in the system.

Task 3 -:

Return to the previous screen.

Task 4 -:

Choose History as subject.

Task 5 -:

Choose the Easy difficulty level to start.

Task 6 -:

Choose the option to read about History.

Task 7 -:

Enter the Crossword.

Task 8 -:

See tips on the period.

Task 9 -:

Fill in the crossword in the Period fields.

Task 10 -:

Finish the game.

 

The test was recorded and documented. Through observations and annotations, researchers analyzed the correctness of the tasks performed by each participant and also their mistakes, the time spent of the test and the experience of each user. In the end, the users participated in an interview in order to verify the difficulties of the participants in the use of Crossword Learning and its strengths and weaknesses.

5 Procedure

Usability and accessibility tests were conducted at the participants’ homes. In average, the sessions lasted 15 min, depending on the participant and on the tasks he/she was able to complete.

Firstly, the purpose of the research and the evaluation process that would be carried out was presented. Next, the informed consent form was read for each one. Upon agreeing to participate in the research, the participants answered the characterization form and started the tasks. Finally, the researchers accessed the prototype on the smartphone and the test started. While the participants were performing the tasks, the researchers wrote down all the comments, questions and reactions, and also recorded the participants interacting with the prototype.

At the end of the usability test session, the researchers interviewed the participants. One of the questions asked was about the difficulty level of each task. The levels were measured as follows: Easy; Medium; and Difficult. Participants also answered questions related to the difficulties they faced, the changes they would make in the application and if they would use the application in their daily lives.

6 Results and Discussion

The accessibility and usability tests showed that most of the participants were able to accomplish the tasks, regardless of whether they were from the experienced or novice group (Table 2).

Table 2. Percentage of complete and incomplete tasks by group of participants
Full size table

Table 3 shows the average time in minutes that each group spent to complete the tasks.

Table 3. Average time to perform tasks by group of participants
Full size table

We can notice that the group of beginners took twice the time to carry out the activities. These results may be due or to the inexperience with the use of mobile applications or to the accessibility and usability barriers of Crossword Learning itself. Actually, to statistically verify the results, a higher number of participants would be necessary.

Crossword Learning prototype, was considered easy to use by most participants. In addition, at the end of the test, the participants were asked: “Would you use this application in your day to day?”. The participants had a positive feedback (Fig. 3).

Fig. 3.
figure 3

Responses from participants about using Crossword Learning

Full size image

Among the positive points highlighted by the users were:

  1. 1.

    Simple and easy to use application;

  2. 2.

    Crossword is a common game among senior users;

  3. 3.

    The possibility of learning about various subjects;

  4. 4.

    Access from anywhere.

On these points, users afirmed the following:

Participant A - “I’ve done several distance learning courses on the computer. I think the idea is very good!”.

Participant B - “I do crossword in the paper all week. I found the idea very cool, because I can do the crossword from anywhere”.

Participant C - “It can distract and is easy to use”.

The tests enabled us to reach a set of accessibility and usability issues that may contribute to other researches. We divided the issues based on the W3C/WAI Guidelines. Although they focus on web systems, they can also help in the development of mobile applications [24].

6.1 Principle 1: Perceivable

The perceivable principle emphasizes that the components of the information and the interface must be perceptible by the users in the mobile application. “This means that users must be able to perceive the information being presented (it can’t be invisible to all of their senses)” [25].

The prototype was thought with focus on some characteristics of perception: (i) simple design without using images; (ii) access to content by different media - video, text or audio; (iii) buttons in larger sizes and easy access; (iv) content on a single screen; (v) possibility of contrast on the screens.

With the accessibility and usability test, we identified some issues that can be considered in the perceivable principle:

  1. 1.

    Font size was very small (GE Group: 6 participants and GB: 3 participants).

  2. 2.

    Colors of the text are inadequate or very bright, making it impossible to read in some cases (GE Group: 1 participant and GB: 1 participant).

The issues highlighted by the users in relation to this principle refers to a problem in prototyping. Crossword Learning has an accessibility menu [Increase Letter, Decrease Letter and Contrast], but the features were not enabled for the prototype (Fig. 4).

Fig. 4.
figure 4

Accessibility menu available in prototype

Full size image

However, it is important to note that none of the participants attempted to use the “Increase font” or “Contrast” link despite the difficulty. This can suggest that: (i) the users did not find the options; or (ii) they were not able to see them; or (iii) they were not used to this type of functionality in other applications.

6.2 Principle 2: Operable

The operable principle says that the user interface and navigation must be operable, that is, the user interface can not contain some interaction that the user can not perform [25].

With respect to this principle, the application followed the characteristics: (i) all the functionalities were developed to allow easy access; (ii) the application made it possible for the user to carry out the activities in his own time; (iii) the pages were labeled according to their content.

Regarding operability, three participants had problems in relation to the size of the keyboard (GE Group: 2 participants and GB Group: 1 participant). The problems are related to the difficulty of typing due to the small size of the letters and due they are very close together. However, this question is a issue of the device itself and not of the application. The following comments are from two participants who have faced this issues:

Participant A - “I prefer cell phones with fixed keyboards, like the regular phone. The buttons are easier to tighten and we have better control over them”.

Participant B - “I usually type a little slow. I am not used to this keyboard”.

Another operability problem is that some of the participants did not understand the need to click on a text field to make the keyboard available. In total, four participants faced this problem (GE Group: 3 participants and GB Group: 1 participant) and did not know how they would type the text. In this case, the researchers had to intervene and give hints on how to accomplish the task.

This problem can be related to non-visibility or non-focus in the text field. Another possibility would be the explanatory text of the inbox had not been sufficiently clear to the right use. In one case (GE Group), we noticed some apprehension from the user of breaking the device or making it stop working. This apprehension caused some users to ask many questions during the test and often did not “risk” doing the task on their own.

6.3 Principle 3: Understandable

The understandable principle focuses on the user’s understanding of information and functioning of interface. This means that the content or operation can not be beyond your comprehension [25].

On this principle, we seek to use the following characteristics: (i) the content was prepared in only one language, avoiding abbreviations; (ii) do not perform context change in the application without the user understanding the change; (iii) presentation of error messages and suggestions for their correction.

Despite the care taken in relation to these characteristics, users have faced the following issues that can be classified in this principle:

  1. 1.

    The application did not allow changing the screen (Portrait/Landscape).

  2. 2.

    The icon used to represent the main menu was not clear.

  3. 3.

    In the crossword, the clickable elements were not clear to users.

One participant said that he preferred the cell phone’s screen rotation in the horizontal position (landscape) because this way the letters on the keyboard become more spaced and larger. This is an option that Crossword Learning does not offer since it is still a prototype. However, it is an issue that must be addressed in the app implementation.

Eight participants had difficulty in finding the Main Menu of the application (GE Group: 7 participants and GB Group: 1 participant) and one participant did not complete Task 2 (GB Group). The problem highlights the need to change the icon (Fig. 5). Here is a suggestion given by a user:

Participant C - “I think in the place of the icon, could be written ‘Menu”’.

Fig. 5.
figure 5

Icon that represents the main menu.

Full size image
Fig. 6.
figure 6

Crossword

Full size image

Finally, the users also commented the crossword design (GE Group: 4 participants and GB: 2 participants). The screen of some mobile devices is small. As this was the case of the smartphone in which the prototype was tested, the participants had difficulty in understanding how the letters would be inserted into the application. A suggestion given by one of the users was:

Participant D - “I think I could put a note to say where we should click to appear the lette”.

6.4 Principle 4: Robust

The robust principle states that the content must be sufficiently robust so that it can be interpreted by a wide variety of user agents. This means that users should be able to access the content as technologies advance (the technologies and user agents evolve, the content should remain accessible) [25].

However, as the prototype was developed based on elderly users, we believe that the application can be used by different users and be compatible with the use of Assistive Technology.

No issues related to robustness were reported. However, by verifying the issues found by users in other principles, we can observe a need for different functionalities to be implemented for text insertion, especially in the crossword. Some users (GE Group: 4 participants and GB: 2 participants) had trouble typing the text in the squares of the crossword puzzle. The possibility of insertion of text by speech or gestures could facilitate this interaction (Fig. 6).

7 Conclusions and Future Work

In this study we discussed the usability and accessibility assessment of a mobile learning prototype called Crossword Learning. The application, prototyped based on the elderly users profile, aims to teach senior users about different topics, using crossword.

The accessibility and usability tests showed that older users had a positive view about the prototype (80% of research participants said they would make use of the application). Despite the positive points presented, we focused on the issues of accessibility and usability. Therefore, we identified some issues in the mobile learning application. Such issues were categorized according to the accessibility guidelines of W3C, that is, perception, operation, comprehension and robust. The results highlight the need of improvements in the m-learning application; also, the issues found were the same that could occur in any application of another domain.

Beginning participants took twice as long to perform tasks in comparison to the experienced group, showing that Crossword Learning demands a learning time. However, most participants in both groups (GB and GE) were able to successfully complete the tasks. Also, it is important to notice that many application problems were related to the fact that it is still a prototype and should be upgraded to the final version of the application.

Finally, this study demonstrates accessibility and usability issues that have been identified in an mobile learning application. Such evaluation can serve as input for developers of this type of application and assist them during the development process.

As future work, other features need to be tested, such as including the accessibility menu and the features that involve reward and competition among friends. In long term, the application can become a virtual learning environment where teachers can include new content and customize it according to their needs and their students. Therefore, new usability and accessibility tests must be performed, as well as a larger number of participants to evaluate the new redesign. In addition, it is important to perform tests to verify pedagogical issues of the application, since it is a mobile learning application.