1 Introduction

Augmented Reality (AR) applications aim to enhance the perception and effectiveness of the user by providing additional visual information. The user stays aware about the real world, but, ideally, is not able to distinguish between information coming from the real or the virtual world [1].

The use of RA systems has been investigated since the early 1990s, and includes medicine, production processes, aeronautics, robotics, entertainment, tourism, marketing, social networking and education [2].

The Augmented Reality has also benefited from several technological advances that have occurred in recent years, which have given rise to more natural and low-cost interfaces and human-computer interaction devices. Among these devices, depth sensors attract attention by implementing advanced interfaces in various Augmented Reality applications, from advertising to serious games for physical rehabilitation [3,4,5,6,7,8].

The present work presents a set of classroom practices for Geomorphology based on an Augmented Reality installation, known as SARndbox [9]. This set was developed based on guidelines developed for systems that use Natural Interfaces, as well as the pedagogical needs intrinsic to the teaching practices of the topics covered.

This paper is organized as follows: Sect. 2 presents some applications that use Augmented Reality in Education; In Sect. 3 the concept and characteristics of Natural User Interfaces; Sect. 4 presents the SARndbox, an installation used to support classroom practice; Sect. 5 describes the guidelines that have guided the development of practices and a brief description of practices created and finally, in Sect. 6, the final considerations and future work.

2 Applications of Augmented Reality in Education

Specifically, in Education, there are applications that aim to promote teaching from elementary to higher education, in several areas of knowledge. An extensive listing of examples of the application of RA in teaching, varying from Biology, Physics, Mathematics, to Religious Education is presented in [2, 10]. Some RA systems applied to Teaching are presented in the following sections.

2.1 3D Pop-up Book

The 3D Pop-up Book is an Augmented Reality book used for teaching English with students in Thailand, which uses markers to combine characters and objects in a book (Fig. 1) [11].

Fig. 1.
figure 1

3D Pop-up Book [11]

Full size image

The results presented in [11] corroborate with other usability assessments, which always demonstrate the great interest of users in this technology, highlighting its playful and innovative aspects, as well as its possibilities in other studies.

2.2 Augmented Chemistry Reactions

To facilitate the understanding of Chemistry, especially the spatial structure of molecules and their behavior in some reactions, Maier Klinker proposed a visualization tool to display and control molecules as well as the dynamics between them [12].

This tool uses a cube with a handle that contains imprinted patterns (markers) on its faces (Fig. 2). Students should manipulate the cube to visualize the molecules and thus rotate and be aware of their atomic structure (Fig. 3).

Fig. 2.
figure 2

Marker cube for molecular manipulation and visualization [12]

Full size image
Fig. 3.
figure 3

Visualização de Moléculas usando augmented chemistry reactions [12]

Full size image

This system, implemented with ARToolkit [13], is representative of the first implementations of RA, which makes use of markers to introduce virtual objects into the scene.

2.3 Augmented Reality Magic Mirror

In teaching Anatomy, Meng et al. propose a system that uses an “AR mirror” [14]. In this system, the depth camera is used to track the posture of the user standing in front of a screen. A computed tomography (CT) scan of the user is generated, creating the illusion that it is possible to observe the inside of your body, as shown in Fig. 4.

Fig. 4.
figure 4

Augmented reality magic mirror [14]

Full size image

Gestures are used to select different layers of the tomography and a set of photographic data can be chosen for viewing. This system is also capable of displaying 3D models of internal organs, textual information, and anatomy images. The interaction with this system dispenses markers and uses depth data, increasing, or decreasing the visibility of the hands based on their distance from a virtual interaction plan. According to the authors, this helps the user to perceive the spatial relationships between his body and the virtual interaction plane.

Most of the examples found in the literature have been used by educators to provide preconceived teaching experiences. This can lead to situations where AR only develops lower order reasoning skills, rather than encouraging integrative skills such as analysis, evaluation, and creation.

Thus, using AR applications to stimulate critical thinking should be a main goal, allowing students to interact and act actively in these systems, instead of being mere spectators.

3 Natural User Interfaces

In recent years, the technological evolution allowed the creation of a range of devices that allowed the emergence of a new mode of interface. This new paradigm is known as Natural User Interfaces (NUI). These can be defined as “interfaces designed to reuse existing skills for direct interaction with content” [15].

Historically, NUIs can be inserted into an evolving line of Human-Computer interfaces, in which three paradigms can be highlighted (Fig. 5): Command Line Interfaces (CLI), Graphical User Interfaces (GUI), and, more recently, Natural User Interfaces.

Fig. 5.
figure 5

Evolution of interface paradigms

Full size image

In command-line interfaces, successive lines of text (command lines) are sent by the user to the system. This interface is found on older operating systems, such as MS-DOS, early versions of Unix, and more. Generally, its implementation consists of a prompt that is able to accept the text commands and convert them into calls to system functions. This is the type often preferred by advanced users interface, it provides a concise and powerful way to control a system.

Graphical interfaces are currently the most common, especially those based on the WIMP (Windows, Icons, Menus, Pointer) paradigm. In this paradigm, the mouse, or some similar device, is used to manipulate elements on the screen (windows, buttons and menus) and thus perform tasks. From a usability perspective, graphical interfaces introduce benefits such as the ease of remembering actions and constructing models that remain coherent for several similar tasks. Visual feedback is immediate when indicating the effects of a given action. For example, when you delete a file, its icon is removed from the screen. The great advantage of this type of interface is that they make the operation of the computer more intuitive and easy to learn.

Finally, the Natural User Interfaces can be found in various systems and applications of our everyday life: touch screens on computers and smartphones; applications using voice commands; gesture commands to control televisions and games; BCI (Brain-Computer Interface) systems, where brain commands trigger the execution of tasks.

NUIs refer to interfaces between humans and computers that use unconventional devices and are based on natural elements. The word “natural” in this case is used in opposition to most computer interfaces that use “artificial” devices whose operation needs to be learned. In this type of interface, one can affirm that the interaction is based on previous user knowledge and, thus, it is easy to learn and to become experienced [15].

3.1 Characteristics of Natural Interfaces

According to Blake, there are three important fundamental concepts that greatly contribute to the usability of NUI-based systems [15].

First, NUIs are designed, are premeditated and prior efforts are employed in their design. It is necessary to ensure that the interactions in a NUI are appropriate for both the user and the content and context. The simple fact of grouping several concepts together is not enough for an interface to be considered natural. One must be aware of the role designers need to play in creating natural interactions and ensure that the same design priority is given to the design phase.

Furthermore, NUIS reuse existing skills. Application users are experts in various skills that have been acquired simply because they are human. For several years, these users have practiced communication, verbal or non-verbal, as well as interactions with the environment.

The computing power and technology have evolved to the point where you can take advantage of these skills. Nuis do this by allowing users to interact with computers through intuitive actions such as touch, gesture and speech, and have interfaces which users can first understand with metaphors drawn from real-world experiences.

Finally, NUIs have direct interaction with content, that is, the focus of interaction is on the content itself and on direct interaction with it. That does not mean controls such as buttons and check boxes are totally absent in the interface. It simply means that such controls should be considered secondary when compared to content, and that direct manipulation should be the primary interaction method

4 Augmented Reality SandBox - SARNdBox

The Augmented Reality Sandbox (SARndbox) is an installation that aims to integrate an Augmented Reality system with graphical effects and simulations to physically created topographic models that have their surface scanned by a computer in real time [9].

A SARndbox aims to integrate an augmented reality system into physically created topographic models that have their surface scanned by a computer in real time. These models are used as background for a variety of graphic effects and simulations.

The installation consists of a computer, a projector, a 3D surface reading device (in this case, the Microsoft Kinect) and a box containing material that can be manipulated, such as sand, to create topographies interactively and with low need for supervision by a specialist (Fig. 6). All software required to configure SARndbox is available for free under the GNU General Public License. For its implementation is recommended a computer with Intel Core i7 processor, clocked at at least 3 GHz, a video card with graphics processor (recommended NVidia GeForce GTX 970), and Linux Mint 64-bit operating system.

Fig. 6.
figure 6

SARndbox installation [9]

Full size image

Initially, the potential use of SARndbox in the educational environment was investigated through questions inserted into a usability assessment applied to 100 participants, including students and teachers of elementary, middle, and high school.

The evaluation consisted of two sets of questions: one to obtain the user profile (age and school education) and other to evaluate efficiency issues, prior knowledge of the topic addressed in the application (contour lines), prior knowledge of AR, ergonomics (comfort feeling when using the application, ease of use, satisfaction when using) and whether they consider that the use of the tool would facilitate learning.

The answers obtained in the usability assessment indicate that there is a great public interest in using innovative teaching technologies. The adoption of these technologies can introduce many benefits. The entire evaluation is presented in [16].

5 Geomorphology Classroom Practices

Despite AR technology is widespread in the Computer Science academic environment and used in entertainment and advertising, there is still a niche to be explored in Education. Although about half of the answers affirm already knowing this technology, it can be observed, from the answers provided, that AR applications, with great visual impact, still arise a lot of fascination and interest in people.

To enhance the teaching of Geomorphology concepts, we propose some classroom practices using the SARndbox, with the indication of auxiliary tools.

The development of these classes considered some NUI-specific usability guidelines, regardless of the type of interaction: Instant Expertise, Cognitive Load, Progressive Learning and Direct Interaction [15].

The Instant Experience assumes that the designed interactions must reuse existing skills. By doing so, we take advantage of the previous investment made by the users in their competences and create experts instantly. Both in the real world and in creating interfaces, the most difficult to use a skill is the learning process. Once this ability is learned, it becomes much easier to exercise it. Users have many skills before even using any application, many of which have been exercised since childhood. Interactions that reuse existing skills enable users to become practical quickly and with little effort. To create experts instantly, J. Blake considers two ways: to leverage domain-specific skills (such as the use of tools and devices common to a given activity); and harness skills common to all humans [15]. Regarding the SARndbox, the interaction consists solely of handling the sand contained in the box. Thus, we ensured that all users could interact with the system immediately.

Cognitive Load implies designing interfaces in which most interactions use innate and simple skills. This brings two benefits: most of the interface will be easy to use and the interface will be quick to learn, even if some of the skills are completely new. If the interface uses capabilities based on natural interactions with the real world, the frequency of interactions will be much higher. In fact, there is a trade-off between Instant Experience and Cognitive Load. These guidelines may conflict when users are already endowed with a useful skill, such as using the mouse. The focus, in these cases, should keep on minimizing a cognitive load for a majority of interactions, reusing it as existing simple skills. There was no concern about this guideline for the development of classroom practices, since the interaction was quite simple and straightforward.

Progressive Learning provides a smooth learning curve from basic tasks to advanced tasks. Natural Interfaces should allow your users to learn and evolve progressively, from beginners to expert level. At the same time, the interfaces cannot make it difficult for experienced users to perform advanced tasks in the best way. This guideline implies allowing the application to be used at the same time as more complex tasks are learned. In the case of class practices created, there was a concern to establish a growing and constant learning curve of the concepts addressed, from basic concepts to the subjects that need more critical thinking and analysis.

The Direct Interaction, in its turn, states that interactions should relate directly to the content; occur at high frequency and be appropriate to the context. Since our interactions with the real world show this effect, the adoption of direct interactions results in more fluid and natural interfaces, and allow access to many features without overwhelming the user presenting them all at once. In the case of SARndbox, the means of interaction is not only direct, but also tangible, allowing free manipulation (within the limits of the box), and giving this system a highly playful appeal to users.

Moreover, in [17] is highlighted the need to provide constant feedback to the user, so that it is aware of what is happening and what to expect as a result of their interactions. In the case of SARndbox, the results of interactions (sand movement) occur immediately with each action taken by the user (Fig. 7).

Fig. 7.
figure 7

User interacting with the SARndbox

Full size image

The broad sense of these practices involves activities related to both basic and higher education, as well as consulting projects, public and/or private companies, among other projects that have the landform as the focus of analysis. The classroom practices proposed include:

  • types of terrain, which consists of building different landforms, working concepts related to each shape, its genesis and evolution. The landforms to be built can be hills, plateaus, flatlands, among others;

  • hydrographic basins, that is, to build in the sand a basin with its characteristics: main river, watershed, tributary and sub-tributaries rivers;

  • slope, that is, building landforms for calculus of declivity and, later, classification of the landform as flat, slightly hilly, hilly, strongly hilly, hilly terrain and steep terrain;

  • relief and the Brazilian Forest Code, that is, to reproduce relief forms with declivities within the limits of use established by the New Brazilian Forest Code (Brazil’s Law number 12.651/2012) [18];

  • rivers and the Brazilian Forest Code, which consists of constructing different types of riverbeds and applying the provisions of both the former Brazilian Forest Code and the new Brazilian Forest Code [18], showing the differences between them in terms of delimiting the Permanent Preservation Areas along the riverbanks.

The Fig. 8 illustrates some contours created for each of the classroom practices.

Fig. 8.
figure 8

Sample contours for each classroom practice

Full size image

The application of this set of practices should be associated with a set of pedagogical guidelines, which aims to evaluate the efficiency of this approach.

Basically, the instructional aspect of SARndbox should be well understood before using it, and after the activity, the topic addressed should be discussed.

The instructor should be able to assess whether the use of the tool was more efficient than the use of conventional teaching techniques, such as lectures, animations, videos, and illustrations.

6 Conclusions and Future Work

In this paper, we present a set of classroom practices based on an Augmented Reality tool that uses as a interaction device a depth sensor created to provide Natural User Interfaces, called the SARndbox. This set of practices was created considering both characteristics required for Natural User Interfaces and specific usability guidelines for this interaction paradigm.

The great visual appeal of this tool, associated with its inherent playful aspect, mainly due to the interaction mechanism, were the main factors that motivated the investigation of the viability of its use in the classroom environment.

The practices created take into account issues addressed in both higher education in the areas of Environmental Engineering, Civil Engineering, Geography, as well as in High School courses, and can be used, with different degrees of depth, at all levels of education.

Future Works include measuring the impact of technology adoption on teaching. To do so, it is necessary to investigate ways to obtain an accurate, results-based, replicable and applicable evaluation not only for the area of Geomorphology, but also to contribute to the evaluation of other Augmented Reality devices applied to the teaching of several other areas of knowledge.