Keywords

1 Introduction

Collaborative learning is, by definition, a situation in which two or more people learn or work on something together [1]. Collaborative learning has a benefit in which people can share their expertise to make something faster or help others learn new things or understand more clearly about a problem or create new knowledge from sharing of their expertise. Examples of collaborative learning are group projects, joint problem solving and other activities [2]. With today technology, collaborative learning can expand beyond collaboration in classrooms to global collaboration in which two or more people can interact to make effective learning. Augmented reality technology is one of the new technologies that have been adopted for this purpose. Unlike its counterpart, virtual reality, in which students are immersed into virtual world, augmented reality generates 3D virtual objects which students can interact as in the real world, making them more interesting to learn. In collaborative augmented reality systems, users can collaborate by face-to-face, remote or a combination of both. Researches on augmented reality and their applications in the field of education have been investigated by many researchers but for collaborative learning, it is just the beginning. To make collaborative learning more interesting and useful, participants in the group should be able to interact on the same topics, or the same object. Interaction with the virtual object in augmented reality technology means people can interact, rotate or even grab the object while others can observe how that reaction affects the virtual object.

In this research, we have set up an experiment for two peoples working at different locations but in the same network to perform an AR collaborative learning. These two persons will attempt to help each other to finish the same puzzle board using their real hands in an AR environment.

2 Related Work

Research on computer supported collaborative work has been done for many years. At first, past research emphasized on communications, meeting support, and coordination and procedures [3]. Recently, due to advance in technology, especially in computer graphics, it has been possible for virtual reality (VR) and augmented reality (AR) applications to emerge and be applied to computer supported collaborative work [4]. Collaborative augmented reality can be applied in many areas ranging from education, manufacturing, health sciences, entertainment and others. Research on developing prototype of virtual and augmented reality applications in remote collaboration for process modeling has been done by Poppe et al. [5] with the aim to help increasing sense of immersion in an intuitive shared work and task space. Interaction amongst users is also a point of interest both in the situation of face-to-face and remote interaction. Billinghurst and Kato [6] have shown that using the characteristics of augmented reality such as seamless interaction between real and virtual environment, the ability to enhance reality, the presence of spatial cues for face-to-face and remote collaboration, different interfaces for face-to-face and remote collaboration were developed. In their demonstration, users could share physical workspace, create an interface for three dimensional Computer Supported Cooperative Work (CSCW). Testing for remote collaboration, AR conferencing interface were developed for this purpose. Another interesting experiment on interaction between two persons in augmented reality environment was developed by Yan et al. [7]. In this experiment, two users who were in different places played ping-pong against each other by using real paddles over the network. Studies on multiple users interacted with two and three dimensional data in augmented environment using tangible user interfaces have also been investigated [8]. Some researchers preferred to work on mobile collaborative augmented reality. For example, Reitmayr and Schmalsteig [9] built a mobile collaborative augmented reality system that supported true stereoscopic 3D graphics with user interface and direct interaction with virtual objects. Wichert [10] developed a mobile augmented reality 3D Tetris game for multiple users as an example of a collaborative augmented reality system. The game shows how to distribute the functionality in collaboration.

For specific areas of collaborative augmented reality applications, education or learning seemed to be the most popular. Kaufmann [11] gave a brief insight into the potential and challenges of using Augmented Reality in education with an example of experience gained from development of collaborative AR applications specifically designed for mathematics and geometry. Gonzales et al. [12] showed how to use mobile technology and augmented reality to enhance collaborative learning on cultural and natural heritage. The project involved forty students and four faculty members. The evaluation results have shown a high rate of success in both pedagogic and technical aspects.

Research on the interaction of users and virtual objects in augment reality environment are also important to collaborative training. There are many concept of interaction that can be used in collaborative training but the most effective one is real hand interaction. There are many techniques for tracking hands and virtual objects such as using the execution of grasp and release gestures [13]. The grasp and release gestures are recognized by estimating the positions of the user’s thumb and forefinger visually. If there is a virtual object within the region between these fingers, and the surface is touching both fingers, then the object is grabbed. If the distance is far apart, the object is released. There is another method proposed by Boonbrahm et al. [14]. They used virtual fingers to grab the virtual object. In this case, distance between virtual finger and virtual objects must be calculated carefully. With this technique, grabbing the virtual object will be done naturally and can be used in collaborative work as well especially for the work that required moving virtual object from one place to another.

3 Methodology

In terms of collaborative learning, people usually prefer face-to-face interaction using conversation, gesture, gaze and non-verbal cues to communicate. However, there are many cases within real environment face-to-face interaction is not possible due to either geographical location or other factors. In this paper, we aim to prove the concept that interactive augmented reality can be used as a new way for collaborative learning. The concept is that more than two people can collaborate by interacting with the same object as long as they are in the same network, even though they are in different locations or using different tools. In our set-up experiment, there are two or more users locating at different area but in the same WiFi network, using their hands to interact with the environment in order to help each other to accomplish a common job, which in this case is completing a puzzle. In fact, the collaboration can be anything ranging from building a small toy to making a virtual house. But for demonstration purpose, we used jigsaw puzzle to prove the concept. In this setup, Unity 3D game engine was used on the Vuforia platform. Interaction with the virtual objects by hand includes touching, lifting and grabbing. Friction was applied to the surface of both virtual fingers and objects. For the devices to perform in this experiment, Apples’ iPads was chosen due to its ease of use and its good camera and display quality for AR applications. There are three components that have to be accomplished in order to facilitate the experiment which include communication framework, virtual interaction and network requirement.

3.1 Framework

In this experimental set up, there are two users at different locations, connected to the same WiFi network, trying to help each other to complete the puzzle with their own hands. The proposed framework is shown in Fig. 1.

Fig. 1.
figure 1

Proposed framework for interactive collaborative augmented reality system

Full size image

As shown in Fig. 1, there are two users collaborating to interact with the model which in this case is the puzzle. Each user has one set of markers to transform their fingers to become the virtual fingers so that they can interact with the piece of puzzle. The piece of puzzle is also a virtual object and it interacts with virtual fingers. The shared model as shown at the center of Fig. 1 represents the puzzle that is one of the four puzzle pieces as shown in Fig. 2.

Fig. 2.
figure 2

Puzzle before and after completion through collaboration

Full size image

Since the pieces of puzzle are shared at the center for both users to interact with, there must be some signs to tell whether that piece of the puzzle is being used by other user or not. To solve this problem, we have assigned them to change color when one user touching them or picking them up. Figure 3 demonstrates this change.

Fig. 3.
figure 3

Color of the puzzle changed from blue to green or red when users touch or grab the puzzle respectively (Color figure online).

Full size image

Another part that needs attention is the output. In this case, output is the display that shows what happened when the interaction occurred. In order to interact from one side, users at both sides need to know what happened to the system. When one user grabs or touches the puzzle on his or her side, the user can see his or her virtual fingers touching or grabbing the puzzle as shown in Fig. 4 but the user at the other side can only see the movement of the puzzle and the change in color. This condition will make collaboration more effective because users realized what is happening.

Fig. 4.
figure 4

User1 can see his or her virtual fingers in the left picture while user2 knows from the change of color indicating that user1 is now touching the piece that have turned green as shown in the right picture (Color figure online).

Full size image

3.2 Interaction Between Fingers and Objects

Since interaction can occur only between virtual objects, if we want to use our fingers to pick up things or grabbing something, we have to change them into virtual fingers. In order to do that, we stick AR markers to them and change them into virtual objects. In our case, we choose cylindrical markers for this purpose. Since cylindrical marker can fit onto the fingers, so touching or picking objects up will make the participants feel like using their own fingers. Figure 5 shows the cylindrical markers and their virtual models which they appear just like real fingers.

Fig. 5.
figure 5

Cylindrical markers and virtual models

Full size image

To make virtual fingers to be able to grab the virtual object, some physical properties such as surface, volume, density, friction and collision detection properties for rigid body have to be applied to them. With this setup, the virtual finger will have better grab due to the properties of the surface.

3.3 Network

As mentioned before, interactive augmented reality for collaborative system will work best if they are in the same network. This network can be either WiFi or Bluetooth. If the location of each user is not far apart, Bluetooth will work fine but if it is not, we need to use WiFi network. For this experiment, the regular household Wi-Fi with a bandwidth of 8 Mbps was used. We can setup the network in two ways depends on resources and stability of the system. The first setup used only mobile devices as a host and client. Figure 6 shows how the network operates. If one device is assigned as a host, it will create a virtual object to be used in operation which in this study, is the puzzle. In the case that another user joins the network as client, the host will clone its virtual object to be used by the client device. This cloning object will behave just like the parent object, except that it will only be seen by the client.

Fig. 6.
figure 6

Network setup for mobile-mobile collaboration

Full size image

In case that the system needs to be more stable, the second way to setup is to use a powerful desktop computer attached to the LAN implemented as a host and the joining mobiles using WiFi as clients. Figure 7 shows how to setup such network. In this case, the virtual object which is the puzzle system will be created only once and both users can share this object.

Fig. 7.
figure 7

Network setup using desktop computer as a host for mobile-mobile collaboration

Full size image

4 Experimental Setup

After setting up all the requirements for testing the concept of using interactive augmented reality in collaborative learning, we have started the experiment by having two users using iPads (as host and client) with augmented reality software installed. The users were asked to help each other to put the puzzle pieces together by using their hands under augmented reality environment. Since we have applied all the physical properties to both fingers and objects, all the parts behave like the real puzzle, so when it touches each other, it slipped. To avoid that, we have to add one piece of virtual model performed as a frame to keep the puzzle within the range as shown in Fig. 8.

Fig. 8.
figure 8

Puzzle sheet with frame

Full size image

Figures 9, 10 and  11 show how the two users performed and what they saw on their display screens.

Fig. 9.
figure 9

Activity of user1 (a) grabbing a piece of puzzle and user2 (b) touching one puzzle piece (Color figure online)

Full size image
Fig. 10.
figure 10

Activity performed by user1 (a) and user2 (b)

Full size image
Fig. 11.
figure 11

Finishing job as seen by user1 (a) and user2 (b)

Full size image

When user1 grabs one piece of the puzzle, that piece changes color from blue to red as shown in Fig. 9(a) while user2 touches the other piece and that piece becomes green see Fig. 9(b). Since user on one side cannot see the finger of user on the other side, color will be used to tell whether it was grabbed or touched by the other user. Figure 10 shows the activity that happened by two users, but this time the second user did not interact with the puzzle.

After collaborate in finishing the puzzle, both display will be the same but may be in different viewpoint as seen in Fig. 11.

5 Results and Discussion

From the experiment, we can see that two users can collaborate in helping each other to complete a job without interacting on the same virtual object at the same time. The only thing that we found which have caused some issues is the stability of the network and the compatible of the devices. In case that one device is more powerful than the other, the movement of the 3D objects on the powerful one will be faster making real time collaboration not possible. The bandwidth of the WiFi is another issue. From our experiment, the suitable bandwidth should be at least 8 Mbps. This concept can be applied to other kind of teaching and learning such as in project based learning, active learning and etc. Besides that, since augmented reality is the technology that deals with virtual object, so all topics related to some kinds of interaction between objects can be simulated for easy understanding. For example, it can be applied in chemistry, physics and biology. With new way of learning like flipped class room in which collaboration amongst students and students with teacher is necessary, collaborative augmented reality seems to fit in nicely as well. Beside teaching and training, some kinds of work that need collaboration such as manufacturing, architecture and business could exploit this opportunity.

6 Conclusions

With the advance in technology like augmented reality in which virtual objects can be visualize in real world along with new user interface and ability of the users to interact with the virtual objects, it is possible to create a new way of learning. This new way of learning not only saves time and budgets for learning new things or to understand difficult topics, but it also creates a sense of working as a team. Groups of student or workers can share their expertise and to create new knowledge with this new way of learning.