Keywords

1 Introduction

The manufacturing sector is a foundational pillar of the European economy that continues to grow, but it is continues to suffer from a lack of skilled human resources able to support them [1]. The shortage of skilled workers will most likely increase as the digitization of the manufacturing industry gains momentum [2]. There are many causes for the skill shortage, such as an aging workforce, an outdated workforce planning, the limited education efficiency, the changing nature of work and a poor image of manufacturing among youngsters [3]. The latter cause is of significant relevance as it affects the potential supply of workers going through the educational pipeline. Some exploratory work has been carried out on how to address the poor awareness and interest in manufacturing education amongst young talent [4, 5]. In particular, some manufacturing concepts (e.g. servitization, sustainability) are hard to communicate by traditional teaching approaches [6], which benefit from the use of innovative learning technologies such as augmented reality and virtual reality that has allowed the design of more complex learning experiences that can involve the user in a more sophisticated educational route [7].

This paper is focused on Life Cycle Assessment (LCA), which is highly relevant topic in the teaching subjects in engineering related to sustainable manufacturing. The traditional teaching approaches neglect the complexities associated the application of LCA within a real manufacturing context. To address the limitations of current teaching practices, the LCA game was created by HighSkillz, Polimi and Sintef. The game was co-developed with undergraduate students over a period of three years, involving in total 265 students. The paper also presents demonstrable results derived from a quasi-experimental study involving 33 master students of management engineering with the final version of the LCA game. The results show a statistically significant improvement in factual, conceptual and procedural knowledge about LCA of the participants, together with high levels of enjoyment in the learning experience.

2 DGBL for Manufacturing Education

There is an increasing body of evidence of the potential of Digital Game-Based Learning (DGBL) to support effective learning [8] in different domains of knowledge, including manufacturing. The use of DGBL allows for the sequencing of task and activities, reducing the typical complexity of many manufacturing concepts and providing a structured learning path to be followed [9]. Additionally, DGBL continuously provides real-time feedback that allows the student to do self-assessment and thereby indicating where they should improve [10]. It allows reflective practice, clarifying the objectives and expected outcomes of the learning activity [11].

DGBL provides learning contexts where students may experiment [12], where they are given the opportunity of making relevant connections among the different disciplines of manufacturing [13]. For example, Coller and Shernoff [14] discuss the use of a DGBL (NIU-Torcs) to support a numerical methods course for undergraduate mechanical engineering students. The students are required to write a C++ program to command a racing car to win a race, exploring concepts such as numerical root finding, curve fitting and optimization, and apply them in the coding of the race. The results show a higher engagement and motivation of students when compared to traditional teaching. Gomes et al. [15] present the 5S Game with the purpose of teaching lean manufacturing to mechanical engineering MSc students. The motivation of the participants and the usability of the game were in general encouraging. Li et al. [16] introduced GamiCAD, an interactive tutorial system for AutoCAD able to provide a gamified real-time feedback on the tasks performed by the learner. Results in terms of engagement of the learner and tasks’ completion ratio were higher than the non-gamified version. Hauge and Riedel [5] presented COSIGA, a multi-student simulation-based game to teach concurrent engineering (CE) for new product development, where students take different roles to develop a type of truck. Evaluation of the game showed the cognitive change of participants according to the nature of the different roles interpreted.

Despite the benefits identified and the different applications developed, the empirical evidence of the educational effectiveness of DGBL remains scarce in manufacturing education.

3 The Life Cycle Assessment (LCA) Game Co-design Process

The Life Cycle Assessment (LCA) Game co-design process covered a period of two years from December 2013 to December 2015 involving a total of 265 university students with a total of six co-design sessions. The LCA game, which was developed with unity3D, is accessible with a browser, which can be used both by small groups (2–3 people) or by an individual.

3.1 Learning Goal of the LCA Game

An important tool in supporting sustainable manufacturing is the use of Life Cycle Assessment (LCA) which consists of a quantitative methodology quantifying the environmental impacts of a product, good or service, throughout the whole lifecycle. The LCA is standardized by ISO, in the 14044 and 14044 regulations [17], where a LCA framework is presented and the general guidelines are highlighted. The knowledge to use and understand LCA is taught in different engineering degrees, supported by case studies and simulators. However, the reality of a manufacturing context raises barriers for an individual to effectively use LCA in their analysis and decision making.

To this respect, the main learning goal of the game is to provide an interactive case study where the difficulty lies not only in the application of the methodology according to the guidelines but also addressing the search, selection, elaboration and synthesis of the results, overcoming the issues of everyday working experience (e.g. lack of information, ambiguity of the guidelines, stakeholder bias). For this reason, the student assumes the role of a newly hired sustainability manager in a manufacturing company of household products where a real LCA on a coffee machine should be performed. The student must evaluate the life-cycle environmental performance of the product and make the best recommendation to the CEO.

3.2 Co-design Process

The LCA game was developed with an iterative co-design process (Fig. 1), inspired by the work of Hansen et al. [18]. The users were systematically involved from concept to refinement of the solution through field testing and evaluation in real classroom settings.

Fig. 1.
figure 1

Steps of the LCA Game co-design process

Full size image

The first step of the process defines the learning objectives based on the underlying learning goal. Each time feedback was collated from users, the learning objectives were refined to reflect new needs that were considered relevant by both pedagogical and subject matter experts. So the initial versions of the LCA game focused mainly on the collection and analysis of the data, which required debriefing in the classroom to better facilitate the impact of the analysis. Consequently, the synthesis of the LCA simulation results into a report and recommendation to the CEO on what decision to make was added in subsequent versions of the LCA game.

With a set of learning objectives defined, the second step consists on the definition of the specific requirements (e.g. tasks to be performed, information to display) that shaped the game design of the LCA game. These requirements were always driven by the learning objectives, but also covered user experience and technical constraints of deploying a solution within an educational organization. In the third step, the requirements resulted in the implementation of a new iteration of the LCA game, which would be used in classroom setting.

In the fourth step, the LCA game is to be evaluated by both students and teachers. The type of evaluation depends on the maturity of the game (paper prototype vs digital game integrated in the organization’s learning management system) and what questions are necessary to be answered to feed the next iteration of development. For the purposes of assessing the learning effectiveness, the LCA game would be evaluated by students in a classroom setting using pre-test/post-test questionnaires targeting both usability and knowledge, which would be complemented with open-ended questions. One may conduct one or more evaluation studies for the final fifth step of analysis, which would apply a variety of different tools. However, the analysis would not take into account the data from the evaluation studies of the iteration cycle, but across all iterations thus assessing the progression and differences as the LCA game evolved.

3.3 Final Learning Objectives Resulting from the Co-design Process

The final version of the LCA game was achieved once the results of the development iteration cycle demonstrated a maturity level where any improvements would result in negligible gains. The final set of learning objectives that resulted from the co-design process were:

  • Objective 1: To understand the LCA data gathering process in the context of a manufacturing enterprise;

  • Objective 2: To critically assess the data collated, many contradictory, biased, erroneous, and select the data to be used in the LCA simulation;

  • Objective 3: To interpret the results of the LCA simulation for impact assessment;

  • Objective 4: To decide on how to improve the sustainability of a product based on the interpretation of the LCA simulation results.

Starting from those specific learning objectives, the main tasks to be performed were identified, keeping them coherent with the four main LCA phases defined by the ISO 14044:

  • Phase 1 – Goal and Scope Definition: The problem and the objectives are presented to the student as part of their briefing in the LCA game.

  • Phase 2 – Inventory Analysis: The student must define a simplified process tree of the product life cycle which will limit the data gathering process.

  • Phase 3 – Impact Assessment: With the identified inputs from phase 2, the game will carry out an LCA simulation that will determine the environmental load and impact.

  • Phase 4 – Interpretation: In this phase, the student needs to interpret the impact assessment and identify opportunities for improvement in order to make a recommendation to the CEO.

3.4 Overview of the Game Design of the Final LCA Game

The gameplay of the LCA game starts with a briefing, where the student is provided with an initial context regarding the product (coffee machine) and activities of the production system. In addition, the student is given an overview of the different stakeholders (CEO, production manager, shift manager and R&D manager). The briefing is required to understand the context and consequently, the student is required to navigate it before engaging with the game itself.

Once the briefing is done, the student finds themselves (snapshot of Fig. 2) in their office as the sustainability manager of the company. The key elements of the interface are:

Fig. 2.
figure 2

Beginning of the LCA Game

Full size image

  • LCA Toolbox: a tablet where the student finds different apps in order to implement the LCA analysis (boundaries selection, flowcharts, data entry and impact charts), and recommendations based on interpretation. In addition, the student can find the briefing with all the explanation and the app “my notes” with all the dialogues.

  • Personal Computer: the student can consult the ERP System (where info about the bill of materials is stored) and the Coffee Machine Instructions (where information about Middle of Life and Recycling is stored).

  • Books: can have information regarding other lifecycle phase not directly related to the company.

When activating the LCA Toolbox, the student should start with the system boundaries definition. The snapshot of Fig. 3 shows the main steps of coffee machine lifecycle to choose from to scope the analysis.

Fig. 3.
figure 3

LCA Game – Boundary Selection

Full size image

The boundary conditions have information on the upstream processes, considering information about the coffee, with the coffee bean production, and the filter production used for the coffee maker. The core process is the coffee machine manufacturing directly related to the company. Finally, the downstream processes after the gate are related to the coffee machine usage and the final disposal.

Once the student has chosen the appropriate system boundary, the next phase of the LCA game is gathering the data to fill in the data entries to do the LCA simulation. The snapshot of Fig. 4 illustrates the case when the student chose only the end-of-life of the coffee machine as the boundary condition. In this case, the data entry form shows only the relevant inputs and outputs as delimited by the scope of the boundary condition.

Fig. 4.
figure 4

LCA Game – Data entry

Full size image

The design of the user experience with regards to the data entry evolved over several iterations of the gameplay. In the final version of the LCA game, with regards to the learning objective 2, the focus is on understanding and discerning the quality of data for making decisions. When data is found, it is shown in the respective column of the interface with the source clearly identified and unlocks the corresponding field for data entry. For each field, there may be multiple data entries and the student needs to determine which is the correct value to consider.

There are multiple sources for the data within the scenario, the student can consult the books, information systems, shopfloor production cells, and the four stakeholders (CEO, Production Manager, Shift Manager and R&D Manager), within the company.

The four stakeholders have different roles in the company, which shapes their vision and predisposition towards sustainable manufacturing. The student when talking to the stakeholders needs to be aware of the potential bias and choose carefully how to make pertinent questions. However, the purpose of the game is not to develop soft skills thus the LCA game (Fig. 5) has fact-oriented dialogue with focus on simplicity.

Fig. 5.
figure 5

LCA Game – Example of the interaction with a character

Full size image

With regards to the information systems, the student can have access to them via their computer in their office. However, for the production cell data, the student needs to go directly to the shopfloor (Fig. 6) by means of an elevator and visualize the information on the terminal cell.

Fig. 6.
figure 6

LCA Game – The shopfloor

Full size image

Once data entry is completed, the student shall complete the environmental impact assessment report in order to understand the relevant environmental indicators. However, aligned with learning objective 3, the student is required to justify the impact assessment results, facilitating reflection. As evidenced in the snapshot of Fig. 7, the student has four sentences characterizing the impact assessment, and for each sentence they have to define the impact category choosing the correct justification.

Fig. 7.
figure 7

LCA Game – Impact charts

Full size image
Fig. 8.
figure 8

LCA Game – Example of final score and individual evaluation

Full size image
Fig. 9.
figure 9

LCA Game – Revamping of the production flowchart

Full size image

Finally, the student sends an improvement suggestion to the CEO (Fig. 10), choosing among five different options made available by the system. The game ends when the student submits the LCA Report to the CEO by clicking the Submit button in the LCA Tool. After that, an email from the CEO (Fig. 8) provides the student with the final score and an individual evaluation of his/her work. The student will hence discover if he/she will be confirmed or not as the sustainability manager of the company:

Fig. 10.
figure 10

LCA Game – Recommendations to the CEO

Full size image

3.5 Main Feedbacks Collected from the Users

The collated feedback from the students contributed significantly to the iterative improvements of the LCA game from both user experience and pedagogical perspective. Regarding the user experience, efforts were done to facilitate the interaction between the student and the game, trying to minimize the need to open many different windows at a time in order to compare and visualize information and to virtually reproduce as much as possible the context and the tools (e.g. notebooks, laptops) available to a worker in a real company. To this respect, the following main changes were implemented, i.e. the revamping of the boundaries selection and production flowchart; the improvement of the Heads-Up Display (HUD), where the initial buttons were replaced by a mobile device that consolidated all the information together; the development of more sophisticated dialogues; the expansion and enhancement of the ease of viewing of the impact charts (Fig. 9).

Regarding the learning content, the initial releases of the LCA game focused only on finding the data concerning different phases of a product lifecycle, which entailed talking to three different characters (the R&D manager was added in subsequent releases), searching information systems and consulting the production cells on the shopfloor. In addition to finding the data, the student then had the task of discerning the correct information and avoiding atypical pitfalls concerning the data, namely biases of individual, inaccurate understanding of the domain, data is not in real-time, the units of the data source requires conversion, etc.

As a consequence, in order to improve the pro-active and critical involvement of the user in the game and hence improving their learning experience, the main following changes were implemented, i.e. introduction of the interpretation and justification of the results obtained after data retrieval; introduction of the provision of recommendations to the CEO on the basis of the justifications identified, hence finalizing the LCA report; introduction of a further character representing the R&D knowledge and biases within a manufacturing company; introduction of more explicit feedbacks during the game (while filling in data and while talking with characters) as well as a final score with detailed evaluation of the user’s work.

Interestingly, all the main changes suggested and then implemented were in line with the theoretical underpinnings of DGBL. As a consequence, as a final result an effective impact on both motivation and learning should be expected.

4 Methods

The assessment of the four specific learning objectives should be translated in assessment methods able to properly track their changes. The final version of the LCA Game was tested with a class of 33 master students of management engineering by means of a quasi-experimental pre-test/post-test design based on questionnaires. The questionnaires aimed at measuring Factual, Conceptual and Procedural Knowledge, as well as the Usability and Enjoyment of the LCA Game.

To measure Factual, Conceptual and Procedural Knowledge, it was not possible to use a standardized questionnaire to assess LCA knowledge, thus a tailor-made questionnaire was made, which was piloted during the previous development iterations. The questionnaire included 5 items for each of the three knowledge sections, totaling15 items, each of them structured as a multiple-choice question with just one correct answer. For Usability, the System Usability Questionnaire (SUS) was adopted [19]. For the enjoyment, the iGEQ [20] questionnaire was adopted. The ratings of 1–2 have been grouped under “No”, 3 as “Neutral”, and 4–5 have been grouped under “Yes” [21].

The experimental design procedure was composed of four consecutive steps:

  • Lecture. The first step consisted of a theoretical lecture, supported by a set of slides, to introduce the general context of LCA and its relationships with sustainable manufacturing.

  • PreTest. Then, the participants completed the 20-min pretest aiming at assessing their initial levels of Factual, Conceptual and Procedural Knowledge about LCA.

  • Engagement session. This step consisted of a 1 h30-min session where the students would engage with the LCA. The students were encouraged to play at least once the whole scenario, but most played several times to improve on their performance. During the session, a teacher and a teaching assistant were present to offer support in troubleshooting any technical issues, without providing any further information about the strategy to approach the game or about LCA itself.

  • PostTest. After the engagement session, the 20-min posttest containing Knowledge, Usability and Enjoyment questionnaires was given to all the students.

5 Results and Discussion

The results (Table 1) of the experiment show a statistically significant improvement in Factual, Conceptual, Procedural and overall Knowledge about LCA of participants. Therefore, the user assessment implemented provides evidence of the positive impact of LCA Game on the specific learning objectives identified. The most significant impact is reached on Factual and Procedural Knowledge, while the weakest is on Conceptual Knowledge. This can be explained by the time needed to develop in a robust way concepts, theories and structures about a given topic, which partially contrasts with the quite brief exposition to the LCA Game the participants were submitted. For this reason, the results anyway obtained for Conceptual Knowledge can be considered extremely positive.

Table 1. Results of paired-sample t-tests between pre and post scores for Factual, Conceptual, Procedural and Overall Knowledge
Full size table

The LCA Game enables an effective teaching approach of manufacturing concepts that would be otherwise hardly explainable to students. The students are able to engage in just 1 h30-min, directly from their laptops, with the development of an LCA report without the need for teachers to set up a huge quantity of material from a lot of different sources. Despite the impersonality of the approach, the learning process of the student is anyway extremely effective, because the ideal conditions for learning are directly reproduced in the virtual environment where the users are immersed. Indeed the specific conditions already identified as effective for manufacturing education are all guaranteed by the DGBL approach embedded in the LCA Game, i.e. the sequencing of task and activities (the user cannot access the next activity until the previous one is completed); the real-time feedbacks for self-assessment (in case of mistakes during data collection, proper advices are used); the application of reflective practice (the final objectives of the activity are made clear from the beginning and drives the whole user flow); the stimulation of creativity and problem solving ability (just some solutions but not just one is allowed, making the user able to explore different options); the interaction with different roles (the other characters of the company); the multidisciplinary (the tasks to be performed range from the understanding of production line power consumption to that of supply chain processes). The abovementioned conditions are always supported (as DGBL theory argues) by the fun element, that feeds the motivation of the student to actively interact with the application. Also in this case, the results support the initial assumptions, with good results both in terms of Usability (3.26, SD 0.44) and Enjoyment (3.22, SD 0.53) of the game, with the former ensuring the persistence over time of the latter.

The findings suggest the high potential for scalability of DGs for manufacturing as the LCA Game is, as well as their implementation in current manufacturing curricula. These applications can be in fact implemented without any specific previous work by the teachers and students, and make them particularly suitable for the introduction of basic manufacturing concepts, leaving anyway the floor to professional software tools in case their application is strictly needed. In any case, we should also remember that the final objective of DGs and LCA Game is not that of fully substituting the teacher, but rather supporting him/her, allowing the differentiation of educational activities able to complement traditional frontal lectures and to make the learning process more engaging and effective.

6 Conclusion and Next Steps

DGBL is for manufacturing a promising approach to support the development of students’ advanced knowledge about the latest advancements of the sector. Soundly based on cognitive constructivism and experiential learning theory, DGBL adds to interaction the fun element, able to support the motivation of the user in the learning process. Despite this high potential, the development and validation of DGs for manufacturing is still in its nascent phase. In particular, proper evidence of their educational effectiveness and suitability for the sector are still largely lacking.

For these reasons, we have presented in the paper a structured methodology to identify the proper way to validate the educational effectiveness of DGs for manufacturing, identifying as final objective that of assessing the learning progress of the students. To this respect, the specific learning outcomes to be used as a reference have been identified (i.e. knowledge, skill, competence) and further linked to Anderson’s types of knowledge (i.e. factual, conceptual, procedural and metacognitive). On this basis, the design and development process of the LCA Game has been presented, identifying its specific learning objectives and features and how they have been progressively refined by means of different co-design cycles with the final users.

The approach for the final user assessment of the specific learning objectives of the LCA Game has been then presented. The results have shown that the engagement of the students with the LCA Game was able to increase their Factual, Conceptual and Procedural Knowledge about LCA concepts. These results should be considered as extremely important, since they are giving evidence not only of positive levels of usability and enjoyment of the DG, but especially of a significance effect on two of the relevant learning outcomes (i.e. knowledge, skill) for manufacturing education.

Therefore, the evidence provided is able to give important indications in the definition of the educational approaches based on interactive learning environments most suitable for manufacturing and hence for the preparation of advanced professionals ready to face the industrial changes underway. Indeed DGs should be integrated in the current manufacturing curricula, making available to educators a stimulating and highly scalable tool able to enhance student skills otherwise hardly developable in a traditional teaching context. As a next step, further results should be then collected about the effects of DGBL compared to other interactive teaching approaches, providing a stronger evidence of its specific limits and strengths.