1 Introduction

Recently, the development of digital technology has been followed by active discussions on the use of virtual reality in various fields. Virtual reality (VR), in particular, provides users with active participation and realistic information. These advantages of virtual reality have attracted a great deal of attention as an educational medium by providing realistic information and enabling direct activity of learners. Thus ease of learning and immersion can be two essential factors for designing User Interface (UI) in educational VR contents.

Psychological immersion, which is also known as flow [1], refers to a psychological state in which the user can be completely immersed with the surrounding environment. The UI design in virtual environment can crucially affect the user’s immersion and satisfaction [2]. The immersion process in VR can be achieved not just by an approach through sensory stimulation with multi-modalities [3], but also by the psychological immersion through the environment and the naturalness of the situation in flow [4].

In the VR environment, interactivity is pivotal especially when we consider the user involvement process and conditions. The interaction between the virtual world and a user is generally performed through a user interface (UI). The game UI can be described as “the way players can interact with the game and receive feedback of their interaction” [5]. Thus it is necessary to maximize the user’s feeling of immersion by natural interaction between the user and the contents in the VR environment. That is, a user needs a UI that can naturally interact with the user’s behavior or movement without learning how to interact with the content. Regardless of the quality of its content, players would not be immersed with the virtual environment if they are not comfortable with interacting and receiving feedback on the content.

In this paper we introduce our ongoing VR learning program for fjord that can enhance the user’s sense of immersion using the diegetic UI in virtual reality environment. A fjord is “formed when a glacier retreats, after carving its typical U-shaped valley, and the sea fills the resulting valley floor”Footnote 1. Fjord is a typical example of glacier terrain, which is not very well known to our target users - junior high school students in South Korea.

2 Story

The narrative in our proposed VR project is an adventure genre. The player first accidently finds the amulets in the Viking-history relating books which were left by her grandfather. From the Viking history book, the player finds out a map of Norway. As soon as touching Norway on the globe in the room, the player finds herself that she just moved from her room to the inside of a boat in the sea of Norway, encompassed with fjords. Now the player’s goal is to identify a place called Seven Sister Waterfall which was described in the book and move the boat safely to reach there using a given map. When the player almost comes to the Waterfall, she experiences a sudden change of weather with thunder and lightning. At this point the player recognizes an emergency bell that is being highlighted with an alarm sound. As soon as the player touches the emergency bell, the boat is struck by lightning and she falls into the floor. The player loses her consciousness and the scene fades out. In the next scene, the player wakes up in consciousness and finds out that she had time slips to the Viking era. Then, seeing a silhouette of a man at the outside of the boat, the player goes outside and meets a Viking. The player hears the story of fjords-related legend from the Viking, witnessing the creation process of the fjords.

3 Diegetic User Interface

The UI can be largely divided into Diegetic UI and non-diegetic UI in the opposite concept. Diegetic (or in-game) UI is “an interface that is included in the game world” [6], which is seamlessly included in the game world together with characters in the virtual world, background, props, and sounds. It can be seen or heard by the game characters. Non-diegetic UI, on the other hand, refers to a typical UI that is “rendered outside the game world, only visible and audible to the players in the real world (e.g., heads-up display)” [6]. While it is arguable to say which type of UI is better for VR games, the choice seems to depend, more or less, on the user’s preference in the first person shooting game genre at least [7].

As a method to naturally enhance the sense of immersion, we are currently focusing on the natural progression of the immersion senses by avoiding the use of the non–diegetic UI (e.g., explicit head-up display menus) and employing the diegetic UI (such as the involvement of story characters who are helping the story progression, the use of props and interactable objects, and sound effects that can provide clues and aid in the process) in VR environments. This section describes three elements in diegetic UI – character, props and interactable objects, and sounds.

3.1 Character – Informing the Progress of the Story

Virtual Reality (VR), compared to other media such as hypertext for interactive fiction or virtual environment for digital interactive narrative, could be an effective medium for building empathic relationship between the player and story characters [8]. In this project, we are creating UI design in connection with character design by presenting a diegetic UI through character. For instance, the player hears the fjord-related legend from the Vikings and experiences the creation process of the fjords as the story goes. Figure 1 shows the example of a diegetic character who plays the role of explaining about fjords.

Fig. 1.
figure 1

Screenshots of diegetic character

Full size image

3.2 Props and Interactable Objects – Providing Clues and Aid in the Process

The props and interactable objects of VR need to be located precisely where they can naturally interact with each other according to the user’s actions or movements. The props and interactable objects in our project are as follows:

  • Props: Wooden chest, Viking history book, Amulet, Map, clock

  • Interactable objects: Globe, Emergency bell

For example, the emergency bell is highlighted with an alarm sound when an emergency situation (e.g., storming in the sea) occurs. Then, the player can recognize that it is time to interact with it to progress the story. Figure 2 shows the example of props and interactable objects that we use in our project.

Fig. 2.
figure 2

Screenshots of props and interactable objects

Full size image

3.3 Sound - Drawing the User’s Attention

Interactable objects in the virtual world have their own characteristic (diegetic) sounds. Along with the sounds, the player can recognize when she needs to interact with some key objects in order to progress the story. For example, the wooden chest has specific sounds that can draw the player’s attention so that the player can approach and open it. So has the emergency bell.

4 Conclusion and Future Work

In this paper we introduced our ongoing project focusing on the diegetic UI in which the player can explore some educational content on the fjords through three diegetic game elements – story characters, props and interactable objects, and sounds – without the explicit use of non-diegetic UIs (such as heads-up display) in VR environment. We are now implementing the project using Unreal game engine and HTC Vive.

As future work, we plan to conduct a user study after completion. We also plan to find out how usability factors such as ease of manipulation can affect the user’s learning process.