Keywords

1 Introduction

Nowadays the society requires better prepared professionals that must be able to work with others to succeed in their work. In this sense, many initiatives have been developed to promote teamwork for developing projects and address challenges. Companies and Educational institutions are giving to teamwork competence special relevance [1] and try to support its acquisition by applying methodologies such as collaborative learning and more specifically Project Based Learning (PLE) [2] or Challenge Based Learning (CBL) [3].

With the aim of promoting teamwork competence and entrepreneurship the authors of this paper have launched in the University of León a laboratory for developing collaborative projects. The idea is to define an open and common space where any student and/or teacher could exchange knowledge and collaborate in order to perform multidisciplinary projects. Projects that aim to promote issues such as of creativity, innovation and collaboration and that can be developed with different proposes as: final degree projects, products for companies, prototypes to be applied as proofs of concepts, etc.

Some of the individuals involved in the project development could have some previous knowledge in project management issues. This is because most teachers have participated in projects previously and most of engineering students have taken subjects related to project management or have used project management methodologies to address problems or assignments in their subjects (some examples can be found in [4,5,6]). However, not all the lab users have the same educational/professional background (they are not only coming from engineering degrees) and should solve problems from different areas that are not always related to their expertise. Moreover, the projects in the lab could not be associated to a subject, so the way in which the team members deal with the project tasks is something they should manage by themselves if they want to succeed. This means that common project management methodologies should be adapted.

Given this context the present work aims to assess three main issues: (1) the success of the lab for producing useful projects; (2) the possibility to adapt a project management methodology to such heterogeneous context; and (3) the evaluation of the individual acquisition of team work competence (TWC) by each project team member.

With this information it is possible to know if the lab is working as expected, if it is useful for students and teachers and if it is promoting the acquisition of teamwork competence by the users.

This research work is then focused in clarifying and supporting these assertions. In order to do so the paper is structured as follows. Section 2 describes the research context, especially the lab and how it works and the methodologies employed by the lab users. Section 3 describes how the previous described issues where measured. Section 4 presents the results and discuses about them. Finally, in Sect. 5 some conclusions are posed.

2 Research Context

2.1 The Lab

The lab was equipped and launched in 2016/17 academic course. In the kick off several teachers and researchers both from the Department of Electronic Engineering and the Department of Mechanical, Computer Science and Aerospace Engineering were involved. The main goal for this space is to develop collaborative projects in which can be involved students from different areas, teachers, researchers and professionals.

The lab, that has around 120 m2, is divided in two parts. One is employed for prototype development. This part includes two 3D printers, a numerical control milling machine and several toolkits (Fig. 1).

Fig. 1.
figure 1

The numerical control milling machine

Full size image

The other part is used for academic activities such as workshops and courses related with the lab activities, there are 10 computers, a video projector and an electronic board for this aim (Fig. 2).

Fig. 2.
figure 2

Students working in the computers and with the 3D printers.

Full size image

Regarding the staff in charge of the lab include the researchers mentioned above and two grantees that made managerial tasks during 2016/17 academic course (related to access control, equipment maintenance and description of the supplies required for the normal operation of the lab). Also, two students’ associations have supported the use of the lab by promoting it and developing out several workshops and roundtables.

Given this context two type of actions were carried out:

  • Academic activities. Several actions of this type have been developed. Specifically: 3 free workshops related to software use and software engineering; 4 additional courses for students; and an optional subject in the 4th year of the Degree in Computer engineering, Specific and Embedded Architectures, that uses this lab for developing projects.

  • Projects. Beyond those developed during the subject described above, students/teachers and researchers. 29 projects were developed in this sense. These projects and the methodology applied in them is described in the following subsection.

2.2 The Projects and the Methodologies Used

As commented in the previous section, one of the main activities of this Lab is to support the development of research projects. These projects are all developed by teams, and the methodology applied depends on the type of project we are dealing with. The possible categories of projects are the following:

  • Projects related to subjects. In this case the project can be the final or partial outcome of a subject. The lab provides tools and materials to develop these projects. The groups that addresses the project are formed by students (from 4 to 8) and applied a project management methodology. Their work is supervised by teachers. During 2016/17 academic year 8 projects were presented and in 2017/18 they were 13.

  • Final Degree projects (FDP). This is an activity located at the end of the studies and it integrates the skills assessment that a professional must have once he/she finishes the degree. The lab provides tools and materials to implement these projects, that in this case are developed in pairs. The methodology applied for teamwork and for project managing use to be suggested and assessed by the FDP advisor. 8 FDPs have been developed in the collaborative lab.

  • Free projects (FP). This type of projects is developed not only by students but also by teachers and professionals. In this case the projects are carried out by a team of people that could have a different educational context and work together in order to obtain a research output or a product. For instance, veterinary researchers and computer scientist can work together in order to develop a product or a solution to a specific problem. 4 projects of this type have been carried out and 3 of them were granted by different Spanish prototype calls (Results are shown in Fig. 3). These are:

    Fig. 3.
    figure 3

    The four granted projects. A is the robotic arm, B the system for prosthesis manufacturing, C the system for interrupt feeding of material for 3D printers and D the app for measuring vertical jumping

    Full size image

    • Development of a robotic arm controlled by brain activity and muscular sensors. This involves health professionals and Computer Scientists. It has been granted by several Spanish Universities (Part A of Fig. 3).

    • Development of a system for low cost manufacturing of prosthesis for healing animal injuries. It involved researchers from veterinary, mechanical engineering and Computer Science engineering. It has been granted by the regional government of Castilla y León in Spain and the Veterinary Hospital of the University of León (Part B of Fig. 3).

    • System of uninterrupted feeding of material for 3D printers by means of pellets. It has been granted by the regional government of Castilla y León within the TCUE 2015-2017 Plan (Part C of Fig. 3).

    • Low cost software prototype to measure vertical jump in smartphones. This project was awarded a prize in the competition of prototypes of the University of León (Part D of Fig. 3).

In these projects it is also necessary to apply teamwork and project management methodologies. However, the activities carried out are unattended, they are not linked to a specific subject or degree. It is necessary to define/adapt methodologies that help to guarantee the project success.

By exploring the learning programs of the subjects and asking the teachers involved in several subjects that carry out learning activities we find out that two main methodologies were applied.

The first methodology is an adaptation of the integrated model of effective teamwork, IMO [7], developed for nonacademic organizational environments but accepted in academia as a valid conceptual framework with high value. This model represents teamwork as a set of processes conditioned by some previous factors that lead to results. It also incorporates the cyclical and dynamic nature as results become new process inputs. The above model has been adapted and completed to include other skills and promoting employability of students. Specifically, the work proposed by Viles et al. [7] used a questionnaire to provide feedback to teams on their performance. To do this, they created a chart radar called “footprint” with the scale of assessment for each operating process. In this way each team could analyze the strengths and weaknesses. The methodology in the University of León was applied in the optional subject of Specific and Embedded Architectures that develops the projects in the collaborative lab. As the subject has a few students and lectures deal with them daily, the feedback was immediate. It involves a verbal communication of those deficiencies that were observed in the laboratory sessions and tutorials, considering the detailed aspects in “Direct Observation” section.

Other teamwork methodology applied in several subjects of different degrees of university of León was Comprehensive Training Model of the Teamwork Competence (CTMTC) [6]. CTMTC explore the group results and how each individual has acquired the competence. The methodology relies on the analysis of learning evidences from data generated by the use of IT-based learning tools by teams during a project development [5]. Moreover CTMTC application entails that teams develop the project in several stages adapted from the International Project Management Association (IPMA) [8]. CTMTC is a proactive method that draws on three aspects of group-based learning: teamwork phases (mission and goals, responsibility maps, planning, implementation and organization of documentation), collaborative creation of knowledge, and cloud computing technologies (wikis, forums, social networks and cloud storage systems) [5]. In the CTMTC, faculty continuously monitors team members’ collaboration and individual evidences along the teamwork phases. Monitoring also enables teachers to guide students’ individual learning. CTMTC allows teachers to do partial summative assessments of TWC [9]. This method has been tested in prior settings [9,10,11,12]. The methodology was applied and adapted for its application in different degree subjects of the University of León (between them Specific and Embedded Architectures) with a high rate of success and acceptation [4, 6, 13].

Given these two options, and taking into account that the projects in which this research are focused are not going to be continuously tracked, it is necessary to use a methodology flexible enough; which, given the previous experiments in different subjects makes CTMTC the most suitable one. With this methodology it is possible to measure not only the final outcome produced by a team during a project but how their members acquire teamwork competence. Although, also the methodology requires some adaptation.

3 CTMTC Adaptation

First of all, we should say that CTMTC methodology is a very flexible methodology. It has been applied in very different subjects and adapted to them as shown in other works [4, 6, 13]. In fact, it has been applied in the lab during the subject Specific and Embedded Architectures with a high participation rate [4].

CTMTC have associated a rubric that allows measuring both the group work and each member work in the group tasks [13]. The rubrics is applied over the evidences gathered during the project development, and theses evidences are mostly stored in forums (group members interactions), wiki (group outcomes) and sometimes cloud technologies (to store final results). However, in case of the FPs the team members are not always employing a learning platform or this kind of tools to carry out their work, so if we aim to employ this methodology both the tools employed to gather evidences and the rubrics should be adapted.

For compiling evidences three actions were carried out:

  1. 1.

    Each group will have a meeting every two week during the project development. In this weeks, the members should describe what they have done in this two weeks and what are they going to do in the following, describing the problems they have found and if the planning has been affected. This is similar to the meetings employed in agile methodologies for software development [14]. But in this meeting the members are also surveyed individually and asked about what they have done during this two weeks and what their partners have done.

  2. 2.

    Every month there will be a meeting with the results obtained since then. During these meetings the team members should present what the group has done, specifying the mission and goals, how they have distributed works, how team members interact between them and the results obtained until this meeting.

  3. 3.

    A final presentation. At the end of the work, the team should present their outcomes to other students/teachers working in the lab and to their advisors.

With these three actions and as commented above the rubric needs to be adapted. Tables 1 and 2 shows how this is done.

Table 1. Questions employed in the rubric to measure group evidences
Full size table
Table 2. Questions employed in the rubric to measure individual work of team members
Full size table

Table 1 shows the different issues that were evaluated about the team work developed. It is adapted taking into account how information is now gathered and stored. Explicit references to forums and wikis were removed.

Table 2 shows the part of the rubric employed to assess individual work of team members. It was adapted for this our FPs. The advisor should attend to team member behavior when they are developing the project, during the meetings and in the surveys. However, as not all the interaction could be gathered as it happens by employing other tools, this table should be supported by other tools. This is done by applying the self-perception of students about the acquisition of TWC using Team Work Behaviour Questionnaire (TWBQ) [15]. The students involved in the experiences should fulfil a questionnaire before and after the development of the projects (one sample of the questionnaire can be accessed here: https://goo.gl/kHYskM). TWBQ has two parts: one in which students have to assess their own ability, TWBQ (Self), and another in which they assess the ability of the group as a whole, TWBQ (Others). In each item (statement), participants have to evaluate their own behavior or the other members’ behavior in terms of an appropriate behavior, on a 7 points Likert-type scale ( l= ”not at all”; 7 = “very much”). The test gives each part a total grade [16]. Although this test is based on self-appraisal opinion, research has found that a person’s beliefs about teamwork behavior predict the generic teamwork behavior that this person displays as a team member [15].

In order to check how is this methodology working we have applied it as a proof of concept in the 9 projects that are being developed during 2017/18 academic year, results can be seen in the following section.

4 Results and Discussion

4.1 Experiment Description

For this experiment 13 FPs were studied, they involved 34 researchers (2 or 3 per FP). The projects began in September of 2017. The researchers were requested to answer the TWBQ at the beginning, later each group developed their work with the associated meetings (8 2-weekly meetings and 4 outcomes presentation). After the second presentation the researchers have fulfilled the TWBQ as if they would have finished the project. This is done for this proof of concept although projects will last at least until June 2018.

4.2 Results

The advisor has been observing the team members during their work and had surveyed them in the 2-weekly meetings. With this it was possible to apply the rubric to evaluate TWC acquisition, which results are shown in Tables 3 and 4.

Table 3. Average grades taking into account each group outcomes
Full size table
Table 4. Average grades taking into account individuals work in each team
Full size table

Table 3, shows each of the rubric elements valued a 0–10 points scale. The last column includes the final grade associated to the work each group has done.

Table 4 shows the individual work by each team member taking into account the rubric. It is possible to that not all the members of a team have the same grade.

Another analysis that is carried out is to compare the perception of teamwork behavior by using TWBQ. Students are asked to score several items regarding their TWC development and the other members in their teams. To do so, a 7 value Likert scale was used. Table 5 shows the average results for each of the groups before and after the experiment.

Table 5. TWBQ with answers per each group about own and others perception before and after the experiment
Full size table

It is interesting to explore if the difference between self and others perception before and after the experiment is significant. In order to check this, first normality is explored to define if parametric or non-parametric test can be applied. Normality is explored taking into account the answers of the whole sample before and after the experiment, which imply more than 50 answers to be explored, this means that Kolmogorov-Smirnof tests can be applied. Both for the own perception of TWC (with a signification of 0.475) and the perception about others work (with a signification of 0.065) null hypotheis is retained, which means that the answers distribution is normal. Taking this into account we have selected a Student’ T test for related samples, because the same students answer the same questions before and after the experiment. Table 6 show the results for the whole sample.

Table 6. Results of the student T test for related samples, with self-perception and other perception about teamwork behavior compared before and after the experiment
Full size table

4.3 Discussion

The adaptation of CTMTC methodology has allow us gathering some results as shown in previous section. When exploring group results in Table 3 it is possible to see that the work per each team is correct with an average grade of 7 over 10. These grades were assigned by the project advisor in the outcomes meetings. Grades are based in the difficulty of the project compared with other carried out previously in the laboratory and the necessity of support by experts and teachers to finish it successfully. It should be noted that grades could have been better if students had described the normative they were using in the group. As this was not clear in any of the projects the grade assigned was 2 over 10 which has a negative impact in the group average grade. For future projects the advisor will clarify the necessity of the specification of team normative.

Regarding the individual grades, and as in other applications of the methodology [4, 13], it is possible to differentiate which team members are working more and which less. However in this case the grade assigned is based on advisor perception as has also happened in other project based learning initiatives [17] and not only in objective data. This could be a problem if the same advisor should review an important number of projects.

Regarding the results shown in Tables 5 and 6, it is possible to say that self-perception and others-perception about TWC development has been increased for all the groups after the experiment which is something positive. In addition, when taking into account the whole sample we can see that the difference is significant. TWBQ has been used in other works with different results. In [16] several subjects TWBQ results are compared in order to see how specific training improves students teamwork capacity. Other works such as [18] also use TWBQ but do not find differences in teamwork acquisition before and after the application of two methodologies. In Conde et al. work [13] this is compared in a compulsory subject and in an elective one. In this case we are exploring other possibility by comparing not different subjects but projects.

5 Conclusions

The development of projects that involve people from different contexts is not easy. The researchers involved could have not the same knowledge, could not work in the same way, have different aims, etc. This makes necessary using methodologies that guarantee the project success in such context. One of the key issues to explore during the project development is teamwork. Results show what each group has done, but it is also required to measure how TWC is developed individually in those teams. In order to do so this work has explored different possibilities and adapted CTMTC methodology for these projects, including also some other tools to take into account students self-perception about teamwork behavior.

The application of the methodology was tested in an experiment that involves several projects. From the gathered results it is possible to assert that the methodology was properly adapted, that it makes possible to measure individual development of teamwork. Also experiment let us know that the groups require some more training about the methodology, so they include team normative when doing their job, which could help solving problems between group members. It would be also useful to include more members in each team so more difficult projects can be addressed.

The experiment has also limitations. The number of projects should be increased and also the number of components in a team, so we can check not only team work behavior in the whole sample but look in each of the project teams. In addition, it would be desirable to explore a complete project and not only when it is not already finished.

As future work it would be helpful to analyze team members and advisor opinions about the methodology, which could help us to improve it. Moreover, it would be desirable to compare projects from different academic years and what are the difference when the methodology is applied and when it is not.