1 Introduction

This study explores a course in human-computer science education at Aalborg University. In all the classes, it is the teachers’ intention to involve every student individually and in group participation, even though the students cannot meet face-to-face as they are geographically dispersed. The teachers’ intentions were reified in the learning design, which included various technologies. However, once a learning design has been created and presented to the students, the teacher can never be sure that the students will respond to the design as intended. Kress and Selander [1] discuss this issue with the concepts of design for learning (the learning design that the teacher has created and shared with the students) and design in learning (the activities that actually take place when the learning design is in use). In this paper, we will look at the learning designs (i.e. designs for learning) that were created by the teachers, and at the students’ reactions to these designs (as designs in learning).

The pedagogical foundation of Aalborg University is Problem- and Project-Based Learning (PBL) [2]. The idea behind this type of learning is that thinking begins in a forked-road situation, a dilemma with two or more alternatives [3]. Since different people think differently, it is preferable to think with others; consequently, the students are supposed to work in groups and to defend their projects during the exam (assessment) as a group. PBL is the pedagogical expression of the theory that learning is context-based and collaboratively constructed instead of individual and independent of context [4]. PBL requires that the students be curious. They must also be able to define problems and grasp theories that help them understand and examine these problems in order to come up with solutions, which they then evaluate and revise until they are satisfied with the result. Moreover, PBL requires teachers that be willing to give up their position of external boss or ‘dictator’ and take on instead the role of the leader of group activities [3]. This means that teachers present fewer lectures, exert less control over the content and assign fewer fixed literature lists than in more traditional pedagogical models. Instead, the teacher suggests content related to the aim of the course and the course description and supervises the students in all phases of the PBL [4]. As final assessment, the teacher leads the examination being the students’ defense of their project report.

Some studies on PBL have found that it can be difficult for novice students to identify resources and to define the relevant problems [5], so facilitation or scaffolding of self-directed learning is key in PBL courses [6, 7]. Moreover, the teacher’s facilitation and supervision are needed to scaffold complex conceptual tasks [8] when the PBL course takes place online, although technology can also be used to effectively scaffold the students’ problem-solving [5].

This study is a keen reflection and a critical discussion of the teachers’ dilemma of, on the one hand, providing efficient online lessons and scaffolds in order to help students understand important theories and find relevant resources, and, on the other hand, letting the students take responsibility for their own learning.

2 Methodology

This study utilizes online field-work in which participants’ activities were observed by gathering data from the internet. We have used data logs obtained user logins for different internet sources to learn about computer-mediated communications between teachers and students, between students and technology, and among students. Hence, this participant-observational research may be classified as netnography [9, 14], a type of research in which the people under study interact and communicate via the internet. Generally, the role of the netnographic researcher is heavily influenced by the fact that the data are gathered online without contact with the subjects. On the one hand, the researcher experiences the same screens and online interactions as the informants, and on the other hand, the researcher has access to logged data that the participants normally do not have access to. In this study, however, we were familiar with the subjects, as they were students in courses that we taught. Consequently, we possessed – and were aware of – a double role, first as teachers (in September and October) and later as researchers (from November to January).

When the research area addresses an educational setting, some netnographic data can be collected from analytic data. Learning analytics is a relatively new field in educational research [10, 11], and although a common understanding of the concept is difficult to find among researchers, it is usually defined as ‘the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs’ [12]. The data thus consist of ‘[…] traces that learners leave behind’ [13] in various educational settings, and these data can be used to understand and improve learning. Learning analytics often use big data, but researchers in the field also find that multifaceted data from mobile devices and from the physical world [10] may contribute to a picture of the way students respond to a learning design, interact and learn. Thus, in this study, our netnografic empirics consist of learning analytics data from two learning designs lasting eight hours in an online module with 70 university students.

3 Research Setting and Empirical Data

ILOO is a human-computer science-related master’s program provided by the Department of Learning and Philosophy at Aalborg University. ILOO is the Danish acronym for IT, Learning and Organizational Change. Students and the teachers from two campuses, Aalborg and Copenhagen, attend the program. The students come from different backgrounds. Some have professional bachelor’s degrees such as teaching; others hold bachelor’s degrees in humanities or social science. The aim of this master study is to enable the students to work with learning, training and IT-related changes, including global perspectives in organizations. ILOO consists of 120 ECTS points (European Credit Transfer System) that should be completed within two years.

As mentioned, this study draws on two teaching sessions from autumn 2016. The course is obligatory, but attendance at the sessions is voluntary. Our data were created in two different sessions in the course ‘Global Perspectives on Competence and Educational Development’. The two authors of this paper both taught two different sessions that lasted four hours each. The course lasted six weeks, that is, 24 h of teaching in total. Seventy-two students signed up for this course, but it took place during a period in which the students were encouraged to participate in internships in and outside Denmark. Therefore, the course was provided online, and the students attended from different countries and areas of Denmark. The learning designs and the data are presented below. The session ‘Cultural Concepts in Global Perspectives’ is presented first, followed by ‘Citizenship in Online Settings’.

The lesson ‘Cultural Concepts in Global Perspectives’ was presented for the students on the Learning Management System of Moodle a few weeks before the course took place. Moodle provided a description and the aims of the four-hour lesson and showed the learning design (Table 1). On Moodle, the students could find two videos with two different cultural perspectives, the literature list and the three discussion fora, which were activated during the session. The students were supposed to read the literature before the session took place.

Table 1. Learning design for ‘Cultural Concepts in Global Perspectives’ (pauses are not indicated). The underscores indicate the websites that generated the data.
Full size table

The teacher introduced the session in a video conference using a system called Adobe Connect. Thirty-six of the 72 students participated in the introduction to the session, which included a brief overview of the agenda and information about how to participate, such as pressing the ‘raise your hand’ button if you want to say something and utilizing the chat to make comments or ask questions. Then the teacher asked the students to work individually by watching the first video presentation of a cultural perspective. The video lasted 15 min and 20 s. The students identified a problem and suggested solutions in group sessions in Adobe Connect, and they described their findings in a written discussion forum in Moodle, which the other students could then comment on. The five groups drew conclusions from the discussion, which they then posted in Moodle. The teacher read the conclusions and summarized them in the videoconference. Then the students considered suitable methods for studying culture and added their suggestions in the discussion forum in Moodle. The second part of the lesson focused on five dimensions of culture, which were presented in a video that the students were supposed to watch during the lesson. Then the students were to provide examples of the five dimensions from their experience at the university or their workplace. Part three consisted of a summary of the four-hour lesson followed by feedback from the students to the teachers in the form of written comments in a Moodle discussion forum.

The learning design for the session ‘Citizenship in Online Settings’ was introduced in Adobe Connect by the teacher, who listed the content for the day and the students’ assigned activities. The first student task consisted of watching a video presentation on citizenship, which had been uploaded to YouTube. The students could watch it during the session but also before or after the session. For the second task, the students discussed questions about the video in Adobe Connect groups, and each student group created two questions for the others to discuss in writing in Moodle. The second part of the lesson focused on netnography as a method for investigating citizenship. After watching the second video presentation, which introduced the method of netnography, the students were to create a blog while pretending to be and see the world from the perspective of someone unlike themselves (such as a person with a chronic disease, a CEO, an old black man, a prince, an unemployed musician, a former beauty queen, etc.). Afterwards, they asked the bloggers questions in the role of an online researcher with a focus on citizenship. The teacher participated actively in all activities. At the end of the lesson, the students had time to sum up and evaluate the activities of the day in the Adobe Connect chat (Table 2).

Table 2. Learning design for citizenship in Online settings (pauses are not indicated). The underscores indicate the websites that generated the data.
Full size table

Next, we explored how the students engaged with our learning design in order to find the patterns in their activities and the relationship between our design for learning and the students’ design in learning [1]. First, we tracked how many students were present in the online teaching space (Adobe Connect) at the beginning of the lessons (part I), in the middle (part II) and at the end (part III) in the early evening. Secondly, we used YouTube analytics to track how much our four videos were watched during the lessons and before/after. Thirdly, we counted the number of comments by current students in the four discussion fora on Moodle.

4 Analysis of the Data

In this project, we analyzed our own teaching designs in order to learn how the students reacted to our design for learning (the learning design that the teacher created and shared with the students). We also wanted to uncover the activities that actually take place when the learning design is enacted, i.e. the design in learning [1], Furthermore, we investigated how the technology supported or did not support full participation in the learning activities (Table 3).

Table 3. Overview of the collected data.
Full size table

Video conferences: lurking, listening and leaving

There were 36 students in the video conference (Adobe Connect) at the beginning of the lesson ‘Cultural concepts in global perspectives’; that is half of the 72 students enrolled in the course. Only 25 students attended the second part of the lesson, and only 11 hung on for the third part. The lesson ‘Citizenship’ started with 26 students, which is a third of the students enrolled in the course. Twenty-five students stayed for the second part of the lesson, and only 12 remained to the end of the lesson. The number of students declines over time: from the beginning to the end of the lesson, and from the lesson in September (36 students started) to the lesson in October (26 students started). The design for learning included 72 students, but the design in learning meant that the number of actual participants declined, after starting with half of the population of 72 students. It is worth considering the impact of the technology, as it is surprising that the students left the session before the end, This caused them to miss an opportunity to learn from the teacher and from their fellow students, as intended in the learning design.

The students were familiar with both Moodle and Adobe Connect, as they had used both the semester before this study. Therefore, the reason for the lack of participation could not be a lack of knowledge regarding the technology.

However, fewer than half of the students entered the lesson in the video conference, and an increasing number of students dropped out during the lessons. Thus, we consider it essential to determine whether our learning designs at least engaged the few students who participated.

Video presentations: snooping and sneaking

Both lessons included two video presentations with important content. These videos were available before, during and after the course. Even when the learning design granted time for viewing the videos, far from all of the students took advantage of this opportunity. The first video in ‘Cultural Concepts in Global Perspectives’ had 68 views but the students had watched only 61% of the video. The second video in that lesson had 53 views, and the students only watched 50% of that video. The first video in ‘Citizenship’ had 32 views, but the students only watched 39% of the video. The second video in ‘Citizenship’ had 29 views, but the students only watched 53% of the video.

The design for learning included time for watching the full video, but to our surprise, we found that the design in learning showed that the students chose other activities. From the numbers, we construe that the students started to watch the videos, but after some minutes they stopped and maybe started over later on. Only a few students watched one or more of the videos in their entirety.

The technology provides easy access to content that teachers consider important, but students often fail to take advantage of this opportunity, even if they are familiar with the technology. The easy access to the video conference also makes it easy for the students to leave the online space without the teacher noticing it. First, the students can appear present in Adobe Connect when they stay signed in, but mentally they may not be present. The numbers show that some students participate less actively in the tasks, although they count as present in Adobe Connect. Second, the teacher cannot – even if she might have wished to do so – give students a disapproving glare as they leave the online learning space with a click.

Online discussions in Adobe Connect and/or Moodle: The few

In order to enhance engagement, learning and reflection, both lessons included online group discussions. Some group discussion took place in separate video conference rooms, and other discussions took place via posts in discussion forums in Moodle or via chat in Adobe Connect. In the lesson ‘Cultural concepts in global perspectives’, all five groups provided content for the exercise in part one without any teacher interaction, and eight students posted 21 comments on the individual exercise in part two, where the teacher posted only one comment. In the lesson ‘Citizenship’, nine students posted 25 comments in the first part of the lesson and the two teachers posted 15. In the second part of ‘Citizenship’, ten students posted 40 comments supported by eleven comments from the teachers. The design for learning invited all the students to participate actively in the reflections and discussions in order to support their learning and to enable them to construct reification of their learning in a social context. But the design in learning shows that very few of the students actually participated.

Evaluation of the session: Lack of engagement

Part III of the lesson included an online evaluation of the session. In the session ‘Cultural concepts in global perspectives’, the students were supposed to write their feedback in a Moodle Discussion forum within the last five minutes of the lesson. The last 11 students snuck out of Adobe Connect without posting any comments in Moodle. In the lesson ‘Citizenship’, 12 students provided 17 comments in the Adobe Connect chat forum, saying that the design was great and that time had flown during the lessons. This suggests that it was easier for the students to give feedback in an online environment where they and the teacher were present already. Likewise, it seems that it was harder for them to visit another website to provide feedback. The teacher’s presence in the forum seems to support the students’ engagement in the feedback, even though the students do not pay much attention to the feedback.

5 Discussion and Conclusion on the Research Question

How does an online learning design impact student engagement in problem-based learning?

The general tendency in this module is a lack of student participation. This tendency surprised and frustrated the teachers, who would like to find an explanation for the lack of student engagement. During September and October 2016, many students were busy with internships, which may have reduced their ability and/or motivation to participate in the sessions. Also, the course is a part of the PBL tradition [2, 15, 16] so the students worked on their own projects. The lack of engagement may be caused by the temptation to only participate in content that was directly applicable to their own PBL project.

Participation also declined over time – both over the four-hour sessions and during the six-week course. At the beginning of the lessons, 30 to 50% of the students participated, but only a sixth of the students remained at the end of the lessons. Furthermore, no students gave feedback as requested if they had to go to a new website to do so. Over the six-week course, student participation declined, perhaps because the project deadlines were approaching.

The pedagogy of PBL might suggest the interpretation that lessons are resources that the students pick and choose at their own convenience. Even when the teachers scaffolded the sessions, supervised the students and actively participated in the online discussions, the students’ own projects seemed to define the relevance of each session for each student.

Moreover, the online learning environment seems to make students feel less obliged to participate, and they often failed to watch all of the teachers’ presentations in the video presentations and instructions in the videoconference. The students also did not participate in the discussions and collaborative reflections that were supposed to be formulated in oral or written formats online. The online environment makes it easy for the students to sneak out without seeing teacher’s disapproving look. Our PBL online designs for learning were designed to make it easy for the students to participate, and the designs offered many ways of interacting with the session content. But the learning designs also afforded many PBL reasons for different reactions to the teachers’ learning design, many of which did not include participating in the collaborative online learning online. For us, the teachers, this was very frustrating; for the students, however, this might be a natural reaction to an online PBL and student self-directed learning.

The teachers had designed for learning and shared this design with the students, whereas the students had designed in learning by engaging in activities that related only to their own projects. In this study, the intersection between the design for and the design in learning was remarkably smaller than expected. Moreover, the teachers intended to provide technology that supported the students’ learning, but this study shows that the students did not utilized the technology as planned in the teachers’ learning design.