Keywords

1 Introduction

Blended learning, a new mode of learning based on Information and Communications Technology (ICT) has received great interested the past years in China. Blended learning combines the advantages [1] of traditional face-to-face learning with e-learning. It allows the instructor to lead, inspire and control the teaching process and also enables students to be proactive, enthusiastic and creative in the learning process [2]. In blended learning, the instructor has a variety of instructional design and teaching methods that can be employed and these methods can be flexible depending on which aspect the instructor wishes the student to focus on: lessons help develop the students’ basic abilities (memorization and understanding) while group discussion and workshops facilitate a students’ advanced abilities (application, evaluation, analysis, creation) [3].

Currently, there are multiple studies on the teaching effectiveness of blended learning. Dziuban et al. [4] did a comparison study on the course completion percentage of students in e-learning, face-to-face learning and blended learning courses. Absalom et al. [5] investigated the teaching effectiveness of e-learning where the test group used e-learning and control group used blended learning. They found that the latter achieved better grades. Furthermore, Cosgrove, B.S. & Olitsky et al. [6], Liu [7], Francis et al. [8], Kirwin et al. [9], Zhang et al. [10] investigated the effects of blended learning on one specific course through experiments and questionnaires. However, there is little research on how to implement blended learning in various disciplines and the effects of different teaching methods in blended learning.

SPOCs are the version of MOOCs used locally with on-campus students. However, China had a late start in MOOCs and SPOCs give little guidance in how to allocate the learning time between e-learning and offline sessions. Only few Chinese Universities such as Tsinghua University (THU) utilize the blended learning technique through existing MOOC resources in several courses. Moreover, how to organize the instructional design of offline learning sessions and whether this kind of diverse teaching method affects teaching effectiveness still requires further investigation. This paper explores the applications of SPOCs in Tsinghua University (THU) and attempts to provide a blueprint for the future SPOCs implementations in China.

This paper examines currently existing SPOCs in THU and poses the following two questions:

  1. 1.

    What are the different ways that SPOCs are implemented in THU and how are the offline sessions organized?

  2. 2.

    Do these different implementations in THU have an effect on the teaching effectiveness?

2 Method

In order to investigated the instructional design and teaching effectiveness of SPOCs in Tsinghua University, this study chose the following 5 classes from the 25 SPOCs offered at THU’s 2016 Spring Semester: an art class on eastern and western art (arts course), a history course on contemporary Chinese history (history course), an introductory computer science course (CS course), an introductory physics course (physics course) and an introductory ergonomics course (ergonomics course). Arts, history, CS and physics courses are undergraduate courses and these courses were selected based on class size(More than 100 students enrolled in these arts course and history course respectively while less than 50 students enrolled in these CS course and physics course respectively). In addition, this study chose a graduate ergonomics course to compare with other courses. Two phases were conducted in the study.

2.1 Phase One: Interview

The first phase of the interview aimed to explore the different instructional design of each respective course from instructor’s perspective. The current section introduces the methodology and results of the interview.

Interview Question.

The interviews question were mainly about basic class information, online learning information, offline learning information and the suggestion of SPOCs. Main questions were: (1) How many credits in this course? (2) What is the form of offline sessions or online sessions? (3) How to allocate the discussion time? (4) What is the grading policy? (5) How to encourage students to take part in the discussion?

Participant and Procedure.

A physic course instructor and 4 teaching assistants of remaining courses were interviewed. The whole interviews were conducted face-to-face in Shunde Building in Tsinghua University. The interview time for each participant was from 28 to 35 min.

Data Analysis.

The interview data were analyzed to compare offline learning forms, discussion time allocation and grading policy in these 5 courses. More details will be approached in the interview result.

2.2 Phase Two: Survey

This study designed a questionnaire and surveyed students from these 5 courses. The objective was to explore the teaching effectiveness from a learner’s perspective.

Questionnaire Construction.

There were three parts in questionnaire: first, the basic personal information includes students’ majors and grades. Main questions were: (1) What is your major? (2) What grade are you in?

In the second part, 29 items were designed based on the Chinese version SEEQ (Students’ Evaluations of Educational Quality) questionnaire, a survey developed by March [11] and later revised by Meng and Liu [12] to adapt to Chinese students. In this study, the survey questions were consisted of six dimensions: learning/value, instructor enthusiasm and organization, group interaction, individual rapport, breadth of coverage, examinations/grading and assignments/readings and we amended some words to adapt to this survey. For the “group interaction” dimension, for example, an instance of item was “I think the instructor or TA encourages students to participate in the discussion.” The 5-point Likert scales were used to measure different levels of agreement to the items in this part from “1 = totally disagree” to “5 = totally agree”.

For the third section, students were asked to answer the question about discussion. The discussion satisfaction following three 5-point Likert scales questions: (1) The discussion (both online and offline) allowed you to gain a deeper under-standing of the class content. (2) You were able to freely express your opinion or ask questions during discussions (both online and offline). (3) The teacher or TA were helpful during the group discussions. We also asked the frequency of online discussion post.

Participants and Procedure.

Questionnaires were given out to the five chosen classes via Internet. In order to keep the validity of each questionnaire, the teaching assistants of five courses forwarded the questionnaires to the students respectively. After filling the survey, each participant received 5 RMB as a reward and the teaching assistants received 40 RMB as a reward.

Data Analysis.

The questionnaire data were analyzed to compare six dimensions of Chinese version SEEQ and the discussion satisfaction in these 5 courses. More details will be approached in the survey result.

3 Result

3.1 Interview Result

Summarizing the in-depth interviews from the teachers and TAs’ from the five courses, several patterns of SPOCs were discovered.

Instructional Design of Offline Learning Sessions.

The interviews revealed that different types of classes differed in the content and form of offline sessions, as shown in Table 1. Physics, CS and arts course only held discussions during offline sessions while history course and ergonomic course combined lessons with discussions. In addition, history course conducted offline learning sessions in the form of lectures where experts were invited from relevant fields to share their viewpoints. For discussions, the five courses had two forms of discussions: the discussions for the physics course consisted of student discussions on the solution for a specific question from the course homework, followed by time to complete individual work while in the other four courses, the TA would lead the discussion with a predetermined topic, followed by presentations from each group and a discussion of the presentation content.

Table 1. Offline session type, session length, grading policy, group number and group size of the five classes in 2016
Full size table

Discussion Time Allocation and Group Size.

For history, arts and CS courses, discussions lasted two periods (approximately 90 min), physics course discussions lasted 3 periods (approximately 135 min) and ergonomic course discussions consisted of one period of lesson and one period of discussion. To ensure participation in discussions, the group size for all courses were consistently around 4–6 students which conforms to the findings of Kanchanachaya [13].

Grading Policies.

Traditionally, course grades are based on the students score on a midterm or final exam/paper, but research revealed that in some SPOCs, the e-learning progress of each student is also included in the final grade. However, the percentage varied across the different courses, as in shown in Table 1, the final grade consisted 70% of the final exam and 30% of class participation in physics course. The teacher from the physic course states, “If there are no final exams, the group discussions will not be in-depth and for such an introductory courses, only exams can show if a student has really learned the material”; In CS course, the programming assignments made up 40% of the final grade (4 assignments, each of which makes up 10%), pop-up quizzes, final exam and participation made up of 30%, 15% and 15% of the final grade respectively; In ergonomics course, the final grade was consisted of 20% reading assignments and homework, 30% midterm project, 30% final project and 20% final exam while video e-learnings were not a part of the final grade; For arts course and history course, both final grades consisted of 40% video e-learning, 30% offline discussion participation and 30% project. The TA for history course believes that “Students from non-history majors are usually not interested in history courses, hence it is important to use grades as a motivation for each aspect of the course to ensure student participation.”

Participation Level.

In SPOCs, offline discussions are usually led by teachers or TA and results in a higher participation level and the “online + offline” teaching method further motivates the students to take the initiative to learn. The participation for online learning varied: Courses with relatively less offline discussion periods such as arts course, history course and ergonomics course had a higher participation in online discussion. The arts course in particular had a highly active student body due to special online sessions where the two teachers and nine TAs would engage in real-time online discussions, resulting in high participation. On the other hand, courses with weekly offline discussions (physics and CS course) had a relatively lower participation in online discussions.

The discussion type, length, grading policy, group number and group size for the five courses are as shown in Table 1.

3.2 Survey Result

Subject Information.

Questionnaires were given out to the five chosen classes and 100 surveys in total were collected from the five courses, 71 of which came from history course (630 enrolled students), 12 from ergonomics course (17 enrolled students), 8 from Physics course (40 enrolled students), 5 from CS course (19 enrolled students), and 4 from Arts course.

Questionnaire Analysis.

Cronbach’s alpha was used to analyze the questionnaire’s reliability. The internal consistency α was 0.961, which suggests the results were reliable and suitable for data analysis.

As shown in Table 2, the general satisfaction for all five courses are relatively high, each dimension with an average higher than 4.1 points. History course had the highest score in “Breadth of Coverage” (4.5 ± 0.5) with little fluctuations between “Group Interaction” (4.2 ± 0.6), “Individual Rapport” (4.2 ± 0.6) and “Examinations/Grading and Assignments/Readings” (4.2 ± 0.5). Arts course had an even score across all six dimensions. The CS course and ergonomic course had the higher “Learning/Value” score (4.2 ± 0.9 and 4.3 ± 0.5 respectively) while physics course scored higher in “Group Interaction” (4.3 ± 0.5) and “Instructor Enthusiasm and Organization” (4.3 ± 0.5).

Table 2. Mean and standard deviation in SEEQ and discussion satisfaction
Full size table

As shown in Table 3, during online discussions, students in the arts course posted 11–15 times because of intermittent offline classes and the real-time online discussion sessions; History course had a larger class size and had more total replies; for the physics course which had weekly offline discussions, the online discussion was relatively inactive; lastly, the ergonomics class’s online discussion functioned more as a homework submission and project presentation area, with around 6–10 posts per student. The TA from CS course said that due to the weekly offline discussions, the online forum had a relatively lower participation in online discussions. However, 4 students in CS course are active on the online discussion forum (more than 6 times to posted), that is different from what the teaching assistant said. This demonstrates that the posting frequency is related to the course’s general organization and function of the discussion area.

Table 3. Frequency of online discussion post
Full size table

4 Conclusion

With respect to student evaluations of each course, the history course had “online learning + offline discussion + lecture” teaching model and scored the highest in the “Breadth of Content” dimension (an average of 4.5 points). The students thought the varied form of classes enabled them to broaden their knowledge. In comparison, the ergonomics course followed a strict “1 + 1” teaching model (1 period of lesson and 1 period of discussion) scored relatively high in “Breadth of Content”, “Learning/Value” and “Instructors Enthusiasm and Organization” (4.2, 4.3, 4.2 points respectively). The physics course scored higher in “Instructors Enthusiasm and Organization” and “Group Interaction” (4.3, 4.3 points respectively), possibly because the teacher participated into every offline discussion, encouraged students to voice their opinions and responded in time to students.

Hence, for arts or history course, discussion should be allocated more time and percentage in the final grade, which will also allow more interaction between the teacher and students to ensure full understanding of the course content. For introductory science courses, courses with offline discussions but no exam requirements might result in less comprehensive understandings of the course content so that it may be plausible to include offline lectures to reinforce the online lessons. Lastly, while considering a form of discussion (no offline lectures), exam scores may need to allocate a higher percentage in the final grade to warrant active participation and teaching effeteness.

5 Limitation and Future Research

This study is an elementary exploration of Instructional design and teaching effectiveness of SPOCs in China so that there are several limitations to the study. First, the sample size of questionnaires is insufficient. As shown in phase two, we only collected four questionnaires in art course and 5 questionnaires in CS course. The number of such questionnaires cannot explained that the instructional design of these SPOCs whether meet the students’ expectation of blended learning. Secondly, we only chose five small private online courses and the number of courses is not enough to reveal the instructional design of each fields (such as history fields and engineering fields). In addition, four of these five courses are undergraduate courses, we know less about the instructional design of graduate SPOCs. Lastly, the frequency of online discussion post could not be explained in the CS course. On the basis of teaching assistant of CS course, the online forum in this course is not widely used so that the situation that one student posted more than 20 times is ridiculous.

Given the limitations of this study, future research should increase the number of samples size, in order to obtain more subjective data on the scope of classes, along with selecting more graduate SPOCs to understand the instructional design of graduate courses. And the in-depth interviews with students taking different courses and observations of the offline classes are necessary becausenot only have we wanted to know the quantitative teaching effectiveness of students, but also want to understand the qualitative subjective perception of the course learner. That we can to understand which way to organize the instructional design of offline learning sessions and whether this kind of diverse teaching method is Suitable for different courses’ students in blended learning. We attempt to provide a blueprint for the future SPOCs implementations in China.