1 Introduction

Learning environments and methodologies have continuously evolved along time, in particular, with the advent of new technologies in order to improve learning and knowledge acquisition. This evolution is clearly verified when we consider the transition from the old blackboards, to the retro-projectors with slides, until the current use of projectors and computers. Nevertheless, when most of the students leave class and go home, the only support they have for studying are the printed (e.g., books) and digital material (e.g., Internet) [17].

In general, the current resources for studying are more theoretical and based on books or school manuals, associated with the computer. However, when it comes to more practical courses such as chemistry or biology, the same does not apply. In this context, it is generally necessary a physical laboratory with minimal facilities to replicate successfully the exercises and experiments.

Another solution currently applied is the utilization of virtual laboratories, in general accessible through the Web, where students can study, review or replicate the experimental procedures carried out at school. A virtual laboratory is a simulation software that provides a wide availability and mobility, so that a user is able to interact with the application whenever they want regardless of their location, time of day and learning pace.

In this paper we present the proposal and development of VirtualLabs@UMa, a virtual laboratory able to be customized easily to new activities and experiments, being accessible to students regardless their localization. This tool is useful to assist teachers in order to enhance the cognitive development of students since they are able to replicate at home the experiment carried out at school, having a richer perception of how the activity can be executed. This tool allows students to consolidate what they learned about the experiments carried out in real environments.

This paper is organized as follows: Section 2 introduces some related works; Section 3 characterizes a virtual laboratory and its main components; Section 4 introduces how experimental protocols can be customized by the utilization of an XML-based language; Section 5 discusses the main issues related to the implementation of VirtualLabs@UMa; Some validation issues are presented in Section 6; Finally some conclusions are presented in Section 7.

2 Related works

In order to further understand the existing virtual laboratories, and to better determine and compare the functionalities of the proposed application, it is important to carry out a literature review. Some of the most promising virtual environments available and presented in this section are Howard Hughes Medical Institute Virtual Labs [10], ChemCollective [39], Virtual Laboratories [34] and VRLabQuim [25]. Nevertheless, some other existing Virtual Laboratories were also considered such as Virtual ChemLab [38], VLabs [5], LiveQuem [31], Model QuemLab [4], VirtLab [30], Quemgapedia [18], VMSLab-G [32], labsvirtuais Pearson [33] and OLI [22].

2.1 Howard Hughes Medical Institute virtual labs

The virtual laboratories of the Howard Hughes Medical Institute (HHMI) won the first prize in the Pirelli INTERNETional Award competition [24] in 2002, due its sophisticated interface for the simulation of laboratory procedures. With this platform the community of medicine students and also non-expert users are able to explore and easily understand the scientific concepts studied for didactic or personal reasons.

Figure 1 depicts the Howard Hughes platform where it is presented as a 2D interface organized into two sections. In the first section there is a Flash application where the user can interact with the available objects and complete all the experimental protocol. In the second section there are some tabs where the user can obtain further information about the different steps of the procedure or about the components being used. The user can also be informed about the goals of the virtual laboratory and he can also verify the correct answers to the different questions proposed during the activity.

Fig. 1
figure 1

Interface of the Howard Hughes Medical Institute virtual laboratory

Full size image

Although these virtual laboratories are well conceived and provide the easy understanding of how the user has to proceed in order to carry out an experiment, the users’ actions enabled are limited to the choices offered by the interface (buttons), thus constraining the user interaction.

2.2 The ChemCollective

The ChemCollective is a collection of virtual laboratories and learning activities based on scenarios and concept tests that can be applied as an alternative to the existing activities available in school manuals. The target users are high school and college teachers that are willing to use, evaluate and create on-line activities for their chemistry classes.

This project is maintained by a group of teachers and students of the Carnegie Mellon University, and is funded by the National Science Foundation and National Science Digital Library.

Figure 2 illustrates ChemCollective, which is also a 2D laboratory. Currently a 3D version of this application has been developed however the main focus remains the 3D representation of the objects to be manipulated. In the 2D version, these laboratories are composed of three sections. In the first section the reagent can be selected from a hierarchy of components (on the left). The second section (middle) is related to the user working space. In the third section (on the right) different information is available about the selected reagent/solution.

Fig. 2
figure 2

Interface of the ChemCollective virtual laboratory

Full size image

Contrarily to the Howard Hughes virtual laboratories, the user is free to interact with any objects and reagents he wants in order to blend them and to analyze the obtained results. This interaction is intuitive and carried out by drag-and-drop actions. In order to use an object, the user just has to select it from the first section and drag it to the working space. Also, in order to add, delete or blend the reagents, the process is similar by dragging the reagent over the recipient and indicating the amount of reagent that will be poured into it.

2.3 Virtual laboratories

The virtuallaboratory.net inc. is a non-profit company with its headquarters in Colorado, which until 2009 was developing a set of applications classified as virtual laboratories under the thematic of evolution and creation of life in earth. Although they are called virtual laboratories, there is no real connotation to the execution of experimental activities. In fact, under the different thematic proposed, the applications present a strong theoretic component, followed many times by a short slideshow or a quiz in order to consolidate the knowledge introduced. The user interaction allowed during the activities is limited either to the selection of the correct answer (or option) or filling out fields from which the results are further calculated.

Figure 3 depicts the execution of an activity that aims at illustrating the growing rate of a bacteria distribution under a certain temperature. In this activity the user has to select a temperature and indicate the amount of time that will elapse between each result obtained. The result is presented as a graph and a table with all the generated values. Compared to the previous platforms, this virtual laboratory presents a very simplified interface based on buttons and text.

Fig. 3
figure 3

Interface of Virtual Laboratories

Full size image

2.4 VRLabQuim

The application VRLabQuim is different from the previous virtual laboratories since it presents a 3D virtual environment. VRLabQuim was developed using VRML [37] in order to describe all the objects and their behavior inside the virtual environment being a good illustration of a flexibility and good performance.

Due its 3D features, this virtual application becomes a more captivating learning environment providing the users with a more playful and interesting chemistry laboratory.

Figure 4 illustrates VRLabQuim after it has been initiated. From this point on, the user can access a set of existing modules in the application, for instance, the instruments module where the user can visualize all the existing objects in a laboratory or the molecules module where he can visualize the molecules obtained in the reactions. Nevertheless, the user is not able to register his learning process in this application. Indeed, VRLabQuim is mostly used as a learning environment not allowing the teachers to assess the students’ learning activities.

Fig. 4
figure 4

Interface of the VRLabQuim virtual laboratory

Full size image

2.5 Analysis of the studied platforms

The study and comparison of the presented virtual platforms allowed a further understanding of the requirements of these learning environments and also of the users’ needs concerning the types of interaction allowed in order to enable an intuitive learning. Most of these applications are well conceived in order to convey the understanding of the topics studied providing a reasonable interface for carrying out experimental procedures. However, as discussed previously, some of them constrain users due their limited number of actions available in the application. Some other tools, such as Virtual Laboratories provide a strong theoretical basis sometimes followed by a slideshow or a short exercise for the consolidation of the studied topic, with no practical experience on these topics. The application VRLabQuim differs from the other applications since it offers a 3D environment for experimental procedures. However, this environment is applied mostly for illustrating the experimental procedures, and this application does not allow students to register their learning process. In general, these virtual laboratories are very domain specific, and they are not flexible enough to be adapted to new activities and experiments.

By the comparison of the platforms, it was possible to identify some research gaps still to be fulfilled, as depicted in Fig. 5. The comparison was carried out concerning some relevant criteria such as Environment, Interaction, Information and Interface, Knowledge Log and Adaptability.

Fig. 5
figure 5

Comparison among the studied virtual laboratories

Full size image

The Environment is related to the dimensions of the working space provided to the users. Depending on the activities being carried out, the user might need a larger working space (also considering it as a 2D or 3D environment) in order to have a broader view of the working environment providing a better and straightforward understanding of the experimental activity being executed. The Interaction is related to the flexibility and broad choices of actions a user has during an experimental activity, such as select, fill out, pour into, move, add, delete, etc. Information and Interface is related to the amount and depth of information provided concerning the experimental activity being carried out. Knowledge Log is related to the capability a platform has to register all the users’ actions and consequently his learning process. Based on this information it is possible to assess the students’ assets and difficulties related to a particular experimental activity. Finally, Adaptability is related to the capability a platform has to process new experimental activities and adapt the virtual environment to the needs and requirements to the execution of these new experimental activities.

Each platform was evaluated based on these criteria and assigned a value from 0 to 5 corresponding to the degree of flexibility or level related to each one of these criteria (0—Not feature supported; 1—Weakly supported—minor efforts towards the implementation of the respective feature; 2—Basic support—introductory functions or characteristics implemented; 3—Intermediate support—some well-defined key features implemented; 4—Strong support—basic and advanced features implemented however still existing some usability issues, and; 5—Fully supported—all functional features implemented having an intuitive usability degree). In this case, the previous criteria aim at determining the degree of implementation of a particular feature has within a respective platform. Since these metrics were applied for the evaluation of usability and functional requirements, the results were obtained based on the evaluator’s subjective assessment with no specific measure for each metric. Table 1 presents the results of this evaluation.

Table 1 Comparison of all the studied platforms
Full size table

Overlapping the results related to the evaluation of each platform (Fig. 5), it is possible to notice that in general they provide an average level of Environment dimensions (3), a high level of Interactivity (5), a high level of information (4), however they provide a low level of Knowledge Log (2) and Adaptability (1). As such, there is a clear relevance in the proposal of a virtual laboratory able to improve the criteria Environment, Interaction and Information, and to propose new solutions for Knowledge Log and Adaptability.

As for the other existing Virtual Laboratories considered such as Virtual ChemLab [38], VLabs [5], LiveQuem [31], Model QuemLab [4], VirtLab [30], Quemgapedia [18], VMSLab-G [32], labsvirtuais Pearson [33] and OLI [22]: Virtual ChemLab provides a set of simulations covering general and organic chemistry laboratories, where students can work on a virtual environment where they are free to make the decisions that they would confront in an actual laboratory setting and, in turn, experience the consequences. This platform provides a high level of Environment dimensions (3) offering a 3D working interface, a high level of Interactivity (5), an average level of information feedback (5), Knowledge Logging and data exporting (3) and, however, a low level of Adaptability (1) since it allows students and teachers to customize some existing experiments but not to create new ones; Considering the other tools, they provide in general, an average level of Environment dimensions (3) since the students work in a 2D interface sometimes being offered some multimedia content, a basic support of Interactivity (2) since students are proposed to select the desired elements in the interface, a strong level of information feedback (5), no Knowledge Logging and data exporting (0) and no Adaptability (0).

Based on the analysis discussed previously, it is possible to verify that most of the virtual laboratories propose 2D interfaces, where there is a direct relation between the level of interaction provided and the degree of understanding of the concept studied. Concerning the Knowledge Log, only one platform provided this functionality enabling to save the state of the application (not the students’ activities) while the other applications did not save any information at all concerning the current activity. Finally, all the studied platforms present a low degree of Adaptability since they support only the experimental activities developed with them. Therefore, there is a clear gap for contributions that improve the environment characteristics and the user’s information and learning management.

3 Virtual laboratory: characterization

The virtual laboratory should resemble the actual hands-on laboratory and thus offer a lot of extra features and functions. Therefore, it represents a very useful tool for teachers as well as for students. In a virtual laboratory, students deduce scientific principles by performing laboratory experiments [2]. Most of the existing virtual laboratories are not only applied for scientific purposes. Instead, they can be used by people in general interested to explore or learn further about a given domain, not having to worry about the required facilities or resources needed to carry out the experiments.

The main advantages of a virtual laboratory are not only related to time, space and resources. A virtual laboratory is a simulation software that provides a wide variety of alternatives for carrying out an activity, not constraining the student to a single solution sequence. This type of application provides a wide availability and mobility, so the users can interact with the application whenever they want regardless of their location, time of day and learning pace. These three aspects have a major impact when considering that a user needs to replicate an experiment several times, until it is fully understood. Besides, with these applications they are not constrained to the lack of facilities or resources of any kind.

When interacting with a virtual laboratory there is a set of concepts that the reader is expected to come across, including [19, 26]:

  • The reagents and tools which are the different objects present in a laboratory with which the user must interact to achieve a particular goal. A reagent is a substance that will be used during the experimental activity, and a tool is an object used during the activity for measuring, transporting, mixing, or perform any other further specific action;

  • An experimental protocol represents all the logic of an experimental activity, being described by a set of actions ordered in a way that an activity can be completed successfully. Scientists use experimental protocols to plan for an experiment. Parts of an experimental protocol include a statement of purpose, materials to be used, control groups to assess effect of the experiment on the tested group, data interpretation methods and references to enable readers to understand the reasoning behind the plan, and;

  • Finally, the results are the objective of all the experimental activity. Based on the results the user can draw his conclusions regarding all the activities he has just performed. The results may also be useful to confirm the user’s expectations, thus validating his knowledge about a particular topic and ensuring that he has completely understood the procedure.

A normal experimental protocol used in laboratory classes is usually divided into four main parts:

  • The first part generally consists of a brief introduction to the experimental activity which refers to their reasons and objectives;

  • In the second part there is a list of all the material necessary for the activity. Although not required, as a rule, this list is divided into two parts, one reserved for the reagents and another for the tools that will be needed;

  • The third part contains the procedure, which is the set of logical steps as previously mentioned in this section, and;

  • Finally, the fourth part is optional and does not exist in many protocols. This last part is a set of questions and tips to guide the student to draw conclusions about the activity undertaken.

Another important feature of a virtual laboratory concerns the time needed to carry out and conclude experimental activities. In this case, this duration is relative since the user can simulate the elapse of time and anticipate the presentation of results with a single click.

4 Customizing experimental protocols

The main goal of the implementation presented in this paper is to build an application that can be customized and adaptable to a wide range of experimental activities. For this purpose, XML [35] is adopted to describe all the experimental procedures to be carried out inside the virtual laboratory.

Figure 6(a) illustrates the first and second level of tags used in the adopted XML file. As it is possible to verify in this representation, this file contains the same required information to describe the experimental protocol, which is carried out in a classroom. The experimental protocol is divided into four parts, identified with different colors. The blue part corresponds to an introduction, the green part indicates the description of objects, the procedure is identified in orange and yellow is the outcome or expected results.

Fig. 6
figure 6

XML file with the structure of an experimental protocol (a) and of a respective container object (b)

Full size image

The introduction consists of the following tags: Focus—describes the main purpose of the experimental activity; Time—indicates the duration of the procedure; VerboseLog—indicates de level of detail used on the log generation; ShowAction—defines if the application shows the action log after user’s actions, and; ProtocolDisplay—describes the information access level that the user has over the procedure. This access has four levels—no access, partial group access, full group access and full access.

The second part, Object, contains the necessary data to create the different objects in the virtual environment and some extra information about those objects. For this reason they are composed of the following tags: Scale, Position, Rotation, and Data Model, as depicted in Fig. 6(b).

The first four elements represent respectively the scale, position, rotation, and the model that will be presented in the virtual environment, and Data contains the necessary information about the respective object such as, Identifier, Object Name, User Action Available, initial and maximum of quantity of content supported, etc.

The Procedure is the third part of the experimental protocol and represents the actions that the user has to perform in the virtual environment. Since it was necessary to determine a certain order among actions, a Procedure was defined as one or more groups of tasks (Group) that will be executed in the order they are described and each group consists of a description (GroupDescription) of a set of tasks (Task) that can be carried out in any sequence. Figure 7(a) illustrates how the Procedure is composed.

Fig. 7
figure 7

Structure of the containers Procedure (a) and Results (b)

Full size image

Finally, in the fourth part of the protocol the results table (Results) is defined. This part indicates if the results are displayed per column or all at the same time (ShowAll). If the display per column is chosen, the label firstColumn indicates which columns the visualization process starts. The last two tags retain the data needed to fill the result table. The Header tag describes the table header and the Row tag describes the different lines of the table. Both Header and Row consist of different columns (called HeaderColumn) and Column, respectively. Figure 7(b) illustrates this structure.

The proposed XML file describes completely the experimental protocols that can be supported by VirtualLabs@UMa. This representation allows the easy customization of new experimental activities within the application (Fig. 8).

Fig. 8
figure 8

Excerpt of an experimental protocol in XML

Full size image

5 Implementation

This section introduces some implementation issues such as requirements, architecture, functionalities and tools which provide a broad understanding about the main concerns related to VirtualLabs@UMa’s development.

5.1 Requirements and use cases

The requirements reflect the needs that a client wishes to be fulfilled by the application being developed [20]. These requirements serve as guidelines, and most of the times, they may represent objectives to be achieved in the project development. Some of the main requirements of the proposed application are:

  • Create experimental protocols for the VirtualLabs@UMa;

  • Support the execution of more than one activity in the experimental laboratory;

  • Enable users to perform different actions within the virtual environment (actions ‘select’, ‘move’, ‘pour’ and ‘rotate’);

  • Visualize materials / reagents needed for each experimental activity;

  • Show the user’s actions during the execution of the activity;

  • Visualize a chart or table with the results of the experiment;

  • Keep track of the various user actions, and;

  • Create test reports.

A use case diagram allows the understanding of all the possible interactions with the system, under the standpoint of the user (actor), called use cases [11].

After the definition of the requirements, two actors were identified and their possible interactions with the system, one who is responsible for managing the experimental protocols in the context of the classroom, which could be a teacher for instance, and those who could carry out experimental activities using the protocols previously created, these actors can be represented by the students. Note that managing protocols involves not only its creation, deletion and editing, but also making them visible or concealed to students. When undertaking an experimental activity the users’ actions may include downloading an experimental protocol from the server, load the virtual laboratory from the Web platform, perform the experimental activity itself, or interact with objects through the actions select, move, rotate and pour existing objects and finally generate reports so that the process can be registered (Fig. 9).

Fig. 9
figure 9

Use case diagram

Full size image

5.2 Architecture

The architecture of a system aims at detailing all the software components, the externally visible properties, and the relations among these components. The architecture of a system allows the verification if all of its initial requirements were correctly taken into account [3]. For the development of the proposed architecture the “model-view-controller” (MVC) [15] architectural model was adopted, as illustrated in Fig. 10.

Fig. 10
figure 10

Architecture of VirtualLabs@UMa

Full size image

The MVC architectural model divides the application components into different layers, such as presentation layer, logic layer and persistence layer. The presentation layer describes all the components of the system responsible for providing feedback for the users. The logic layer implements all the main functionalities of the system, and the persistence layer describes all the database access and management. MVC was introduced in Smalltalk-80 and since then has been widely used in software design [7]. MVC improves scalability, reusability, maintenance, easing personalization, etc. The main benefits expected from using this model include guaranteeing the quality and decreasing time and cost of engineering processes [9, 23]. Most of the existing Enterprise Application Architecture development patterns [6] (such as Domain Logic, Data Source Architectural, Object-Relational Behavioral, Object-Relational Structural, Object-Relational Metadata Mapping, Web Presentation, Distribution, Offline Concurrency, Session State and Base Patterns) provide recurring patterns of classes and communicating objects in many object-oriented systems. These patterns solve specific design problems and make object oriented designs more flexible, elegant, and ultimately reusable [14]. Model-View-Controller (MVC) in particular is a standard for design and a generalized solution in software in general and web development [27], moreover it promotes and improves the tasks of developing (prototyping), testing and maintenance of systems.

Figure 10 illustrates the architecture of the system, outlining the virtual environment which is related to the presentation layer (View), the XML and Moodle [13] files corresponding to the persistence layer (Model) and Data processing corresponding to the logic layer (controller). In particular, Moodle was applied in order to provide the management of the content of the courses, allowing the teachers to create and configure new experimental protocols to be applied within the VirtualLabs@UMa. Therefore, from our Moodle pages it is possible to include links to the experimental protocols and to trigger the execution of VirtualLabs@UMa.

Based on the XML description of the experimental protocol (Fig. 8), it is possible to define the initial state of the virtual laboratory. The analysis of this XML file by the controller allows the creation of an instance of VirtualLabs@UMa and the start of an interactive process between the logic and presentation layers. This process is based on the identification and processing the actions carried out by the user within the virtual environment, the necessary changes to the state of the virtual environment and presentation of this new state to the user, thus restarting the cycle of interaction.

5.3 Main functionalities

After having discussed the main implementations aspects, this section introduces the main features of the system developed. The interface of VirtualLabs@UMa can be divided into two parts as illustrated respectively in Figs. 11 and 12: (i) the Moodle module and (ii) the VirtualLabs@UMa virtual environment. The Moodle module allows the user (such as teachers) to create and configure new experimental protocols using as the main platform the Moodle application. The VirtualLabs@UMa virtual environment is the 3D interface where users (such as students) will load and carry out the experimental protocol.

Fig. 11
figure 11

Creation of a new experimental protocol within Moodle

Full size image
Fig. 12
figure 12

VirtualLabs@UMa environment

Full size image

As for the Moodle module, the teacher responsible for a course having a permission to edit or to create new protocols, logs in and accesses the respective Moodle section of his course, there he/she will find a link available for the creation/edition of a VirtualLabs@UMa experimental protocol. When he/she chooses either to create a new protocol or edit an existing one, a form will be presented for this purpose, as illustrated in Fig. 11. This form is composed of four groups of information such as Base Configuration, Objects, Protocol and Results Table.

In the Base Configuration group the user can describe all the main configuration aspects of the experimental protocol such as its name, goal, duration and information to be available to the user during the session (list all tasks/do not list, all the users action/no actions and validate actions).

In the Objects group the user will be able to add all the 3D objects (reagents, test-tubes, test-tubes support, shelves, workbench, etc.) he desires to present in the virtual environment for the respective experimental protocol, indicating its position on the X, Y and Z axis and its respective 3D model.

The Protocol group allows users (teachers) to determine all the procedures that should be carried out during an experimental protocol. A procedure is defined as a set of tasks (actions) to be executed by the user. In order to accomplish successfully an experimental protocol the procedures created must be executed in sequence. However, the tasks inside a procedure can be executed in any order. To create a new task, the user can add this task on the dialog box and choose the respective type of action (move, pour or rotate), providing the remaining information according to the action selected. For instance, as illustrated in Fig. 11, the user can define two main procedures, “Selection of materials” and “Blending the reagents”. In the procedure “Selection of materials” different tasks can be created such as “Move Iodide Potassium to the workbench”, “move Lead Nitrate to the workbench”, etc.; and in the procedure “Blending reagents” some other tasks can also be defined such as “Pour 2 ml of Lead Nitrate into the test-tube”, “Pour 2 ml of Iodide Potassium into test-tube”, etc.

The last group of this form aims at describing the results table expected for the reactions of the respective experimental protocol. The reader should observe that VirtualLabs@UMa is not a simulation platform for experimental activities, instead this is a platform where the student is free to replicate the experimental activities he learned at school in the sequence he thinks this might be correct, and the platform can correct and guide the student to choose the right reagents and execute the procedure in the correct order. If the student was successful to choose the correct reagents and blended them in the correct order, the platform is supposed to present the expected results, as the student makes the time elapse (in a click of a button). As such, based on the generated reports, the teacher is able to verify the learning process for each student and detect possible difficulties one student might be going through. After confirmation of all the configuration of the new protocol, the respective file is made available on Moodle, so that students can download it.

As for the VirtualLabs@UMa interface, once the students download it from Moodle and install it locally on their personal computer, they are able to launch the application and load the XML file related to the experimental protocol they want to work on. Figures 12 and 13 illustrate some snapshots of VirtualLabs@UMa. This interface is composed of three main areas: the main menu, the message panels and the virtual environment.

Fig. 13
figure 13

Presentation of the results table inside VirtualLabs@UMa environment

Full size image

In the main menu the File selection allows users to load the protocol, reinitialize the virtual environment, generate reports and close the application. In the actions menu the user is able to visualize all the actions history he executed during the experimental protocol.

The message panels are used for presenting to the user information about the protocol execution. The panels (from top to bottom) indicate which object is currently selected, the goal of the activity, the actions that were performed and other information required by the user (e.g., actions validation), and warning messages. Below these panels a progress bar is also presented together with some buttons to allow the users to verify the experimental protocol and to visualize the results table.

Since VirtualLabs@UMa is built as a 3D learning environment, it offers a more intuitive interface for the user to interact with, allowing the user to replicate the actions he would usually take in a real environment inside this virtual environment. These actions can be to select an object, move it, pour its content into another container object and rotate it.

In particular, Fig. 12 depicts the VirtualLabs@UMa after it has been initialized. At this moment, all the available objects and reagents are available in a cabinet. From this moment on, the user is free to choose the objects and reagents he needs in order to carry out the experiment. Such as in a real environment, the user has to take these components over a workbench, and there he has to perform all the activities required by the experience (e.g., blending, selecting quantities, etc.). Also, the user is allowed to restart over his experiment, if he desires so.

When the user accomplishes 100 % of his experimental protocol, an interactive button is enabled in order to allow the visualization of the results of the experiment. The user should click on this button to visualize the results table for the respective protocol, as illustrated in Fig. 13. When visualizing the results table, the user has three options: (i) to generate automatically two files related to the experiment report template and to the actions history report; (ii) to make the time elapse on the results table until the maximum time is reached, and; (iii) leave the results table dialog, saving the current state of this table for further visualizations.

One important feature of VirtualLabs@UMa is the possibility of recording all the actions of the user during one session. Once the user has completed an experiment, it is possible to generate automatically a report which can be sent to a teacher for evaluation or be archived for further utilization. Thus, evaluation can be based not only on the final result of the experiment, but also taking into account the process undertaken by the student. Clearly, this feature allows teachers to follow each students’ activities, enabling them to help students and improve their learning experience.

5.4 Implementation tools

For the implementation of VirtualLabs@UMa Java3D was applied using the open source integrated developing environment called Eclipse [28] to create all the java classes needed for the virtual laboratory. During the modeling phase, the application 3ds max was adopted [1]. However, since Java3D was not able to support more advanced textures, such as raytrace to simulate glass, the final result lacked some more realistic appearance providing a sketchier look to the virtual environment.

We also applied PHP and JavaScript for the implementation of the Web platform and further integration with Moodle. For testing and debugging the code we applied XAMPP server and a firefox plug-in called firebug [16] during the development of the web platform.

During the development of VirtualLabs@UMa different solutions (e.g. programming languages, 3D languages, metadata, etc.) were considered such as C++, Java, X3D [36], O3D [8], OGRE [21], XML, among others. The final choice relied on aspects such as performance, interoperability and compatibility among the solutions adopted.

6 Validation and performance issues

Since VirtualLabs@UMa is composed of two main components (The module based on Moodle and the virtual environment VirtualLabs@UMa), it is important to plan the validation of both components separately through different use case scenarios. This section first introduces how these use case scenarios were determined and after that, the analysis of the validation results is discussed.

6.1 Planning the use cases sessions

VirtualLabs@UMa was designed to support learning of high school students, supporting the execution of experimental activities. Nevertheless, since this platform allows teachers to plan and propose new activities, it is possible to convey different degrees of difficulty and therefore VirtualLabs@UMa can also be used among university students. However, validation tests were conducted with students of 10th/12th years who attend Biology, in which they performed activities related to the enzymatic catalysis.

As initial tests the usability of VirtualLabs@UMa was evaluated by a group of six volunteered students. Each session was estimated to last around 10 min. Within this time the platform was introduced to the students, then they had to complete a set of tasks and afterwards fill out a short form. The tasks the users were supposed to carry out were:

  1. 1.

    Launch the application using Moodle;

  2. 2.

    Load the existing protocol (XML file) from Moodle platform into VirtualLabs@UMa;

  3. 3.

    Carry out the protocol procedures and visualize the activity results after 1 h and 30 min of reaction. After that, generate and save the respective report, and;

  4. 4.

    Visualize the created documents.

Before users begin to work on the tasks, they were asked to express with words everything they were thinking about the work to be done. This technique is called think aloud [29]. This method allows the observer to know in a simply and immediate way how the application influences and guides the users throughout the tasks. Nevertheless, this method requires some specific skills, for instance, if a test user does not participate verbally, the observer should encourage him to do so without influencing his results. During the execution of the tests some notes were taken of all the observations the users have made which were considered relevant. The students also filled out a questionnaire in the end of the session in order to evaluate their experience. In order to compare the results obtained, an expert user carried out the same tasks executed during the test sessions.

In order to test the usability of the Moodle module it was proposed to two high-school (secondary) teachers to carry out a list of tasks. The tasks to be carried out were:

  • Create a new experimental protocol;

  • Fill out the Base Configuration template for the new protocol in order to determine the tasks that can be executed within 1 h, allowing the visualization of the procedure by groups, not presenting the users’ actions after each intervention in the system;

  • Add the object “room”, using the model Objects/CloudRoom.obj, scaling it up 10 times;

  • Add the object “workbench”, using the model Objects/BalcaoSemPia.obj, scaling it up 1.5 times so that it is placed on the ground of the room;

  • Modify the protocol TesteMoodle.xml;

  • Create the following procedure composed of two groups:

    • Materials and reagents selection group: (1) Move to the workbench the reagents Iodide and Potassium, and; (2) Move to the workbench the objects support case and one test-tube;

    • Reagents blends group: Pour into the test-tube 1 ml of iodide and 1 ml of Potassium.

  • Create the following table of results:

    Reagent 1 (color)

    Reagent 2 (color)

    Product (color)

    Iodide (colorless)

    Potassium (colorless)

    Potassium iodide (yellow)

  • Make available the protocol TesteMoodle.xml for download.

Similarly to the tests carried out with VirtualLabs@Uma the technique think aloud was also applied. However, after the execution of the tasks a short interview was carried out instead of filling out a questionnaire in order to confirm some notes that were taken, and to try to figure out some actions that were not clear during the think aloud session.

6.2 Analysis of the users’ results

Despite the small number of users initially recruited (volunteers) to test the developed platform, the results of these tests were helpful in order to detect potential mistakes and usability limitations in this solution. After the execution of the tests with the students, the initial results concerning the time spent to carry out the proposed list of tasks is presented in Fig. 14.

Fig. 14
figure 14

Time needed to complete the tasks by each user

Full size image

This Figure depicts the time needed to complete the tasks required to each user. It is possible to notice the clear division between two groups, the first one took between 5 and 8 min do complete the tasks, and the second group needed between 11 and 13 min to finish them. These two groups were naturally formed during the experiment according to their degree of expertise and former experience with virtual environments. It is also remarkable the difference between these two groups and the expert user (who intended to be used as a contrast to measure the students’ performance).

Based on the questionnaires filled out by the users and on the notes taken during the think aloud sessions, the following results were achieved, illustrated by Figs. 15, 16, 17, 18, 19 and 20, where the y-axis indicates the number of users that have chosen the respective option:

Fig. 15
figure 15

Answers for questions 1 and 2

Full size image
Fig. 16
figure 16

Answers for question 3

Full size image
Fig. 17
figure 17

Answers for question 4

Full size image
Fig. 18
figure 18

Answers for question 5

Full size image
Fig. 19
figure 19

Answers for questions 6, 7 and 8

Full size image
Fig. 20
figure 20

Answers for questions 9, 10, 11 and 12

Full size image
  1. Q1.

    How often do you use the laboratory facilities?

  2. Q2.

    How often do you use a virtual environment?

  3. Q3.

    When using VirtualLabs@UMa the understanding of the goals of the experimental protocol …

  4. Q4.

    The results presented are …

  5. Q5.

    The knowledge record (report generation) is …

  6. Q6.

    The application reacts as expected to the users’ interactions?

  7. Q7.

    The interface is intuitive?

  8. Q8.

    The interface promotes an easy interaction?

  9. Q9.

    The action “move” is …

  10. Q10.

    The action “pour” is …

  11. Q11.

    The action “rotate” is …

  12. Q12.

    The action “deselect” is …

  13. Q13.

    Do you have any comments or suggestions for modifications?

  14. •

    Facilitate the deselect action

  15. •

    Propose other actions inside the virtual environment (e.g. keys AWSD)

  16. •

    Provide users with additional information when the application is launched

Based on the results of the tests sessions, questionnaires and interviews, it was possible to verify that there is an important gap between the time the expert user and the novice users take to carry out the same tasks. This result indicates that the interface of VirtualLabs@UMa is still not intuitive enough so that users are balanced concerning the time they take to carry out these tasks.

Still, based on the same results it is possible to observe that factors such as laboratory utilization or familiarity with virtual environments can help decrease the time one user takes to finish his tasks. Although these factors produce an interesting division on the users sampling, it was observed that the higher the experience with real laboratories, the better was the users’ performance within VirtualLabs@UMa, regardless their level of experience with virtual environments.

As for the level of interaction, it was possible to observe that most of the users in a first phase did not interact with the virtual environment, and when they tried to change the perspective view, they applied the mouse for that aim. In order to overcome this problem it would be important to provide the users with a short (multimedia) tutorial on how to execute the actions and to move inside the virtual environment. Also, a Help section could be made available on the main menu for quick references.

Concerning the generation of the experiment reports, it was observed that five out of the six users when consulting the generated files, actions record and report template, considered only one file, ignoring the other. This happened as a consequence of the process of creation of these files, where the users are only supposed to indicate the name of the files and the directory where they want to save them. Since only one file name is provided, the users normally expected to have one file generate, not two files with the same name (one text file and the other a pdf).

Briefly, VirtualLabs@UMa presents eventually some difficulties for novice users, however these barriers are overcome as their utilization frequency increases. Also, the action “deselect” which allows users to withdraw the focus from one selected object was considered as confusing or unnecessary. An appropriate solution should be proposed to replace this action. Once these difficulties were identified, the interface of the proposed application should be reconsidered in order to provide a more intuitive experience during the first interactions.

Despite the initial misunderstanding found in the interface due the lack of further or complementary information, VirtualLabs@UMa represents a solution that can provide a positive influence over the students’ experience and performance in a real laboratory. Indeed, since students can replicate the experimental protocol at home as many times as they want, they will be better prepared to carry out this experiment inside the real laboratory.

As for the tests carried out with the Moodle module, the users were able to carry out the proposed activities for the configuration of new experimental protocols following the provided form. During this test some usability and functionality limitations in this interface were identified. For instance, when composing the virtual laboratory the user has to place the 3D objects in their correct position inside the virtual environment. Unfortunately, this operation can only be currently carried out through the method of trial and error. This task is rather slow, complex and error-prone, in particular when composing environments with many objects. To overcome this limitation, an intuitive graphical interface should be provided in order to enable users to easily place the objects inside the virtual environment.

The results of tests with the students and teachers were helpful to detect some limitations of the proposed platform. The correction of these limitations will help providing a more complete and intuitive learning environment. Indeed, VirtualLabs@UMa can be intuitive enough once users get acquainted with the virtual environment in order to carry out the activities required for the experiments. Furthermore, an important advantage remarked by the teachers was the possibility to track their students’ actions during their experiments, which allow them to better help their students with the topics they still lack to be improved.

Comparing the solution proposed in this paper with applications introduced in Section 2 is a hard task since these environments are domain-oriented. Nevertheless, Table 2 presents a tentative comparison between the studied platforms with VirtualLabs@UMa concerning the adopted analysis criteria.

Table 2 Comparison of all the studied platforms with VirtualLabs@UMa
Full size table

As presented in Table 2, when compared to the other platforms, VirtualLabs@UMa provides users with a higher degree of freedom related to the possible actions they can take within the environment without losing the focus to the goals the user has to achieve, at the same time this platform evaluates each one of the users’ actions according to the main steps of the experimental protocol being carried out. Also VirtualLabs@UMa provides a good mechanism for recording the users’ actions for the further generation of an experiment report. Although the interface of VirtualLabs@UMa is not intuitive enough for new users who are not comfortable with it, this limitation is overcome as the user gets used to the application.

The main feature that distinguishes VirtualLabs@UMa from the other platforms definitely is the Adaptability. Through the mechanism for the creation of new protocols (based on Moodle), this application becomes able to support new experimental protocols created from scratch by the teachers and additional 3D objects when available, according to the learning needs of the studied contents. All the laboratories studied were limited to a single activity whereas the developer has to create a new laboratory every time a new activity is needed. In the case of VirtualLabs@UMa, once the user defines new experimental protocols with the proposed XML representation, the application is able to adjust itself automatically to support this new experiment.

The validation tests carried out demonstrated that the continuous utilization of VirtualLabs@UMa by the students help them to learn easier and improve their participation in real laboratory activities since the students demonstrated motivation and were involved with the activities proposed.

7 Conclusions

In this paper we presented a new proposal for Virtual Laboratories, called VirtualLabs@UMa [12]. The main goal of this application is to provide students with a new customizable platform where they can carry out different experimental activities. To achieve this goal we applied an XML-based language for describing experimental protocols. With this description, teachers can create and customize their own experimental activities, which can be easily executed and validated within VirtualLabs@UMa.

The main advantages of the developed application are: its high capability to adapt to any given protocol; it allows users to perform any actions inside the virtual environment such as in the real environment, and; the possibility to record all the actions of a user during one session which can used to generate a final report. This report provides teachers with a follow-up of student’s activities which can be helpful for improving the student’s learning experience. So far, the existing experimental platforms did not allow students to provide teachers any feedback from their activities during an experiment.

VirtualLabs@UMa is introduced as a solution for motivating students and proposing an added value for complementing laboratory courses. The evaluation and comparison with some existing platforms allowed us to demonstrate how this Platform can fulfill some research/implementation gaps. The results of initial tests enabled to identify some usability weaknesses and to correct some of them. Further utilization results and learning evaluation will be presented in forthcoming contributions, after more in-depth and larger utilization of the Platform.