Keywords

1 Introduction

The integration of Information and Communication Technology (ICT) in educational settings has gained a lot of interest. Especially the use of mobile devices can offer great potentials to complement traditional education with new technologies and enrich students’ learning in various ways. Because of the increasing capabilities of these devices, they bring together a multifarious collection of software- and hardware-based functionalities and could finally also replace additionally required hardware resources such as traditional handheld calculators.

GeoGebra [1] is an open source mathematical learning tool that allows students to manipulate, explore, and experiment with mathematical concepts through different dynamically connected representation forms, which can lead to increased motivation and a deeper level of understanding [2, 3]. However, to exploit the full potential of mobile devices and related mobile applications for educational purposes, it is required to guarantee high usability [4], especially for a tool like GeoGebra which is used by millions of students and teachers worldwide. Nevertheless, a search of the literature revealed that only few studies have investigated the usability of mobile graphing calculator applications used in school practices so far. One study that could be found is described in [5]. The authors present an evaluation of the interface of an older version of GeoGebra’s tablet application. However, tablets are rarely carried by students, hence smartphones should also be considered as possible learning tools. This is also confirmed by the much larger number of users of GeoGebra’s new smartphone app [6] on mobile phones vs. tablets (see Fig. 1). For this reason, in this study the smartphone application of GeoGebra is investigated. The aim of this study is the identification of possible usability issues of the GeoGebra Graphing Calculator application used on smartphones. Based on the identified problems, suggestions for an improved user interface and workflow will be given. As a result, these suggested improvements should enhance the educational impact of mobile graphing calculator applications in school practices.

Fig. 1.
figure 1

Percentage of GeoGebra Graphing Calculator users on mobile phones vs. tablets (Google Analytics from Aug 2016 to Jan 2018).

Full size image

2 GeoGebra

GeoGebra is a dynamic mathematics software tool that is used by millions of students and teachers worldwide. GeoGebra can support the teaching and learning of geometry, algebra, spreadsheets, graphing, statistics, and calculus. This open source math tool started out as a set of mathematical applications for the desktop operating systems Windows, MacOS X, and Linux and was later accompanied by HTML5 web applications running in all modern browsers. In 2015, a first version of a native mobile application for Android smartphones was released. This app is based on the existing source code of GeoGebra’s mathematical algorithms, however the user interface was modified in order to address the special requirements of smartphones, in particular their smaller screen size.

2.1 GeoGebra Graphing Calculator

After launching the GeoGebra Graphing Calculator application the graphics and algebra view are displayed on the screen (see Fig. 2 left). At the top of the screen a toolbar is included offering possibilities for access of geometrical construction tools, settings, materials, and the main app menu. Pressing the “pen” button for construction tools in the upper left corner turns the toolbar into a panel that offers an overview of all available categories that contain one or more related tools (see Fig. 2 right). After selecting one of these tools, a subpanel is opened as a second header row, displaying the set of tools of the corresponding category. All of these tools can be used to construct new objects within the graphics view. Furthermore, the graphics view also supports familiar gestures such as one-finger pan, pinch-zooming, and long tap for moving and zooming the coordinate system. Additionally, all of the created objects are also displayed as algebraic representations within the algebra view at the bottom of the screen. It is particularly worth mentioning that the graphics and algebra view are dynamically connected. This means that manipulations in one view are always immediately shown in the other one. Aside from the graphical possibility of constructing new mathematical objects, the algebra view includes a custom formula editor for algebraic input. This editor supports input of mathematical operations as well as GeoGebra commands through a custom virtual keyboard (see Fig. 2 left). Besides, a button is included within the input bar to open a separate view that offers a list of all available mathematical functions and GeoGebra commands as well as a description and short example of how to use those commands.

Fig. 2.
figure 2

GeoGebra Graphing Calculator.

Full size image

3 Design of the Experiment

This study uses quantitative and qualitative analysis in order to gain insights into the usability of the mobile GeoGebra Graphing Calculator application (version of Spring 2017 as seen in Fig. 2). Eye tracking will be used to categorize user interactions with the mobile application regarding active construction, visual search, and actions that are indicative of failures. Additional data will be collected using the system usability scale (SUS) questionnaire [7], that allows to get an overall perception of the investigated application. Particularly, the study is based on the ISO 9241-11 definition of usability and evaluates effectiveness, efficiency, and satisfaction through task analysis. Effectiveness is measured by completion rate. Efficiency is based on the time spent on one task to successfully complete it and the data retrieved from eye tracking. The SUS score is used to determine user satisfaction. All of these study findings are then considered for suggestions of an improved user interface and interaction design of a mobile graphing calculator application.

3.1 Participants

The presented study collected data of 9 adult participants (6 female and 3 male). 5 of the participants had no previous knowledge in using GeoGebra or any other graphing calculator application on mobile phones. 4 of all participants were already experienced in using the desktop application of GeoGebra, but with no experience in using the mobile application.

3.2 Usability Evaluation Methods

System Usability Scale.

In this study the System Usability Scale (SUS) [7] is used to quantitatively evaluate user satisfaction. SUS was created by John Brooke [7] to get a quick overall perception of an application. It is a 5-point Likert scale consisting of 10 items (see Table 1). The questionnaire consists of positive and negative formulated items, that are sorted in alternate sequence. Each of them can be answered ranging from “strongly agree” to “strongly disagree”. For calculating the SUS score for all positive questions the scale position is subtracted by one. For each negative question the scale position is subtracted from 5. The average sum of all calculated scores is then multiplied by 2.5 in order to get a final overall value ranging from 0 to 100.

Table 1. System Usability Scale questionnaire.
Full size table

Eye Tracking.

Eye tracking allows to observe and measure eye movements of participants while they are looking at the mobile user interface. The retrieved data can be visualized as gaze plots. This enables a possibility to observe where participants are looking on a user interface. In particular, gaze plots illustrate fixations and saccades by dots and fine lines (see Fig. 3). The size of the dots depends on the duration of fixation. Small dots indicate short fixations, whereas longer fixations are indicated by larger dots.

Fig. 3.
figure 3

Gaze plot example.

Full size image

Task Scenarios.

In order to guide the eye tracking session prepared work tasks (see Table 2) are given to the participants. These work tasks represent conventional tasks, as users would work with the application in real life situations. In addition, the exercises are defined in a way to evaluate the main features of the GeoGebra Graphing Calculator application, that are plotting and investigating function graphs and geometric constructions.

Table 2. Task scenario of this study.
Full size table

Think Aloud Method.

Additional qualitative data is collected through the think aloud method. While participants conduct the task scenarios they are asked to think out loud. In this way, eye tracking can be combined with additional qualitative data that allows the correct interpretation of the gathered eye movement data.

3.3 Apparatus

For this study a Tobii Pro X2-60 [8] eye tracker was used. Since this research is based on the smartphone application of GeoGebra a special mobile device stand was used (see Fig. 4). This device stand includes a mounted high-definition scene camera to film the mobile device. This allows the recording of the user interface as well as user interactions. Additionally, the analysis software Tobii Studio was used that superimposes the captured eye tracking data on the recording of the mounted scene camera. The mobile device used in this study was a Samsung S6 smartphone with Android operating system version 7.

Fig. 4.
figure 4

Mobile eye tracking setup.

Full size image

3.4 Procedure

At the beginning of each experiment the motivation and goal of this study was introduced. Afterwards the workflow of the whole experiment session was explained. After this introduction each participant was asked to sit in front of the mobile device stand in a comfortable way. The eye-tracker, mobile device stand, and the calibration plate were mounted in advance. Before the calibration was started, the equipment and procedure was explained. For each participant the calibration was repeated until reliable results could be achieved.

During the eye tracking sessions participants were asked to complete a list of prepared tasks (see Table 2), that represents possible natural situations of using the GeoGebra Graphing Calculator application. After the task scenarios, participants were asked to rate their experience with the GeoGebra Graphing Calculator using the SUS questionnaire.

3.5 Data Analysis

For analyzing the gathered data from eye tracking, Tobii Studio was used to identify different segments of user interactions. This data was further analyzed using SPSS and Google Sheets for statistical analysis and graphical visualizations. The total SUS score was evaluated based on the results of the SUS questionnaire, that participants filled after the task scenarios.

4 Results

4.1 Effectiveness

In order to evaluate the effectiveness of the mobile application task completion was considered. As shown in Fig. 5 all of the participants were able to complete tasks 1, 3, and 5. Three of the participants could not successfully complete task 2. Task 4 could not be completed by two participants. The overall effectiveness of the GeoGebra Graphing Calculator application was calculated as percentage of successfully completed tasks, that is 88.89%.

Fig. 5.
figure 5

Percentage of users who successfully completed each task.

Full size image

4.2 Efficiency

For evaluating the efficiency of the GeoGebra Graphing Calculator application the time spent on each task as well as data that was retrieved from eye tracking were used.

The average time that was spent on each task is visualized in Fig. 6. On average task 2 took the longest amount of time to be completed, as might be expected given that constructing a parallelogram requires several steps involving different tools. On the other hand, task 3 took the shortest time to be completed by participants.

Fig. 6.
figure 6

Average time spent on each task.

Full size image

The evaluation of the time-based efficiency of the GeoGebra Graphing Calculator application is defined as follows:

$$ Efficiency = \frac{{\mathop \sum \nolimits_{i = 1}^{N} \mathop \sum \nolimits_{j = 1}^{M} \frac{{n_{ij} }}{{t_{ij} }}}}{NM} $$
(1)

Where,

  • N = total number of users

  • M = total number of tasks

  • \( n_{ij} \) = 1, if user i successfully completed task j, 0 otherwise

  • \( t_{ij} \) = time spent to complete task j by user I

For the mobile application an efficiency of 0.6 (tasks per minute) could be achieved. Figure 7 shows the average efficiency of each task.

Fig. 7.
figure 7

Average efficiency of each task.

Full size image

For further analysis, all of the eye tracking recordings were divided into multiple segments. These segments represent the main actions i.e. active construction, visual search, and repair from failure. Additionally, timestamp, duration, and a short description of the user interaction was defined for each of them. Overall 1625 segments were categorized. Considering all successfully completed tasks, participants spent 65% of total time on active construction, 31% on visual search, and 4% on repairing from failures. Figure 8 shows the average duration of the three categories for each successfully completed task. Participants spent most time on searching for specific features while they were working on task 5. Task 2 caused the most time on repairing from failures.

Fig. 8.
figure 8

Average duration that is spent on each successfully completed task based on the three action categories failure, construction, and search.

Full size image

4.3 Satisfaction

In this study user satisfaction was determined by SUS score. The results of the SUS questionnaire are visualized in Table 3. The average SUS score is 63, which is below the average SUS score of 68 [9], that indicates usability issues in the investigated version of the GeoGebra Graphing Calculator application.

Table 3. SUS score results.
Full size table

5 Discussion and Suggestions for Improvements

Participants faced most problems while working on task 2. This may be caused as this task was more complex, requiring several steps to construct a parallelogram. Five of the participants first started to search the toolbar for a suitable tool. Therefore, they selected multiple different tools and tried what could be constructed with them. This resulted in many unused constructions that were deleted again. Three of those participants found the parallel line tool. However, one of them was not able to use it. Even the displayed tooltip was not helpful for her. To construct a parallel line in GeoGebra, first a line has to be created. After that, it is possible to use the parallel line tool in order to select the previously created line and to construct another point through which the parallel line is constructed.

Another problem that caused multiple actions to repair from failure across all tasks was caused because of the target touch area of objects. For some of the GeoGebra tools it is necessary to select existing objects from the graphics view. For example, if using the angle tool, either two segments or three points can be selected. But participants were not able to hit the objects, thus new points were created. This caused the necessity of removing the unexpected created points and repeat the construction again. Therefore, we suggest to increase the target touch area for objects within the graphics view.

Another issue that could be observed from three participants was that they have tried to zoom the graphics view by pinch-zooming while they were creating a construction, i.e. a construction tool was active. However, instead of zooming, new objects were created. For this reason, we suggest to automatically identify common gestures such as pinch-zoom and allow zooming of the graphics view even though a tool is selected.

GeoGebra uses a custom virtual keyboard that should aid users in typing mathematical expressions. From the task analysis and eye tracking recording it could be observed that participants struggled the most with the input of \( x^{2} \). As shown in Fig. 9 the virtual keyboard includes a button displaying \( x^{2} \), however this button only inserts power of two. Similarly, also the input of \( x^{3} \) caused problems because the intended button \( a^{x} \) was not recognized as the correct one. As known from other custom keyboard implementations, the labeling of the buttons could be modified in a way, that it is clear to the users, which parts of the label is automatically inserted and which parts need to be entered by users.

Fig. 9.
figure 9

Virtual keyboard.

Full size image

One point that caused a lower efficiency rate for the Graphing Calculator application is the selection of a certain tool. This issue could be observed by all participants. GeoGebra provides a set of tools for constructing mathematical objects. These tools are divided into different categories that are displayed within the toolbar. After tapping one of the categories, the displayed tool is already selected and active. However, the eye tracking recordings have shown that users always selected the tool from the opened submenu, even though the tool was already selected (see Fig. 10). As the tools are only used in combination with the graphics view, the user interface could be modified in a way, that the algebra view is replaced by another view that displays all available tools. In this way, the geometric construction tools could be arranged in another way than a row with submenus. This could reduce the time spent on construction because users do not have to tap the tool icon twice.

Fig. 10.
figure 10

GeoGebra toolbar with opened submenu.

Full size image

While working on task 5 participants spent a significant amount of time on searching for a tool that can be used for calculating the integral of a function. For evaluating the integral of a function the input bar and the Integral command can be used. However, participants also searched the geometrical construction tools and the keyboard to find a proper tool that can be used for this exercise. They finally managed to complete the task after they have opened the help view of the input bar and searched for the phrase “Integral”. In order to improve this issue, a button for the integral symbol could be added to the keyboard as it is already known from different other pocket calculators. Since multiple participants also searched for a construction tool, it could also be considered to add an integral tool.

Independent from the tasks, participants further searched a long time for a way to delete existing objects. GeoGebra offers two possibilities: using the delete tool from the toolbar or using the small cross displayed at the end of each algebra view row. Participants tried to select and long tap the objects and further searched within the properties view of the objects for a possibility to delete them. Therefore, we suggest to display a small context menu for each object after selecting it within the graphics view, that includes most used settings and actions such as deleting the object. This would further allow to completely remove the need for long tapping an object to open the properties view. Users were often interrupted in their construction process while moving objects or the graphics view because the touch gesture was recognized as long tap and the properties view was opened.

6 Conclusion

This paper presented a usability evaluation of the mobile GeoGebra Graphing Calculator application (version Spring 2017) using eye tracking and the SUS questionnaire, with the aim of identifying possible usability issues. To do this, usability research has been conducted involving nine participants. These users were given a set of tasks to be performed on a smartphone using the GeoGebra Graphing Calculator application. After performing the task scenarios subjective measures using SUS was collected. The result of SUS questionnaire indicated a system usability score below average, that is supported by the data gathered from eye tracking. Based on the identified usability issues several suggestions for improvements of the user interface of the GeoGebra graphing calculator application were made.