1 Adaptation Systems in Collaborative Contexts

Computer-supported collaborative learning (CSCL) is a branch of learning science concerning how people learn together with computers [11]. Collaboration differs from cooperation. Cooperative learning occurs when individuals accomplish assigned tasks independently and combine their work to arrive at a single output. Collaborative learning, on the other hand, occurs when learning is constructed socially, though a process of negotiation and sharing [11]. Thus, successful collaborative learning means successfully learning the subject matter and successfully working with others [9].

Traditionally, CSCL environments provided learners with a variety of communication tools to support their activities: email, chats, discussion fora, audio and video conferencing, and others. Recent years have seen a growing interest in the application of artificial intelligence (AI) to automatically and adaptively provide learners with the scaffolding they need in order to learn the content and to work with each other productively [see 5, 11]. Indeed, CSCL applications aim to capture knowledge of group activity and use it to better support group interactions [5].

Magnisalis and colleagues [5] characterize adaptive CSCL along several dimensions. Pedagogical objective is the general pedagogical goal of the system, i.e. whether its goal is to present the values of selected activity types (mirroring), information on what productive activity might be (metacognitive), or advising learners towards behaviors that might improve collaboration (guiding). Target of the intervention refers to that aspect of learning task that the systems intelligence supports. The target could be group formation, content support, or peer interaction support. Modeling refers to the aspects of the domain, pedagogy, activity, and learners that the system internally represents in order to make informed adaptations. Closely linked to modeling are the technologies used to generate these models, which include AI and non-AI techniques. Finally, design space refers to how these adaptations are presented to the learner—explicitly or implicitly, directly or indirectly.

Creating adaptive CSCL is a non-trivial undertaking for at least two reasons. Group interactions are inherently complex. The best possible group interactions are marked with idea generation, constructive criticism, a plurality of viewpoints, shared understanding, and peer support [9]. In the more usual instances and in the absence of pressure from a teacher, learners can fail to interact productively with their peers [4], loafing instead and depending on a few competent, responsible group members to complete the work [2].

Second, existing models for adaptation or intelligent support tend to assume individualized learning [5], possibly because theoretical frameworks and models for team interactions have not yet reached a level of maturity that translate easily to software [3]. In group contexts, how do we balance the giving and withholding of assistance? What type of assistance should be provided—cognitive, social, metacognitive, affective? Should the assistance be explicit or implicit? Should the assistance be presented to a specific group member or to another group member, through a change in the learning environment? These are all open questions [8] and are likely dependent upon individual considerations (e.g., student knowledge) as well as culturally-specific expectations.

Researchers have made inroads into finding some answers. At the conceptual level, Chopade and colleagues [1] offer a framework for constructing intelligent tutoring systems for teams. It augments classic intelligent tutoring systems architecture components (learner model, domain knowledge, interface) with new units needed to support team work, e.g. a team model and a dynamic team adaptation module. Soller and Lesgold [10] compare an array of computational methods of modeling the task-based and social processes of collaborative learning through fine-grained analysis of team interactions.

At the level of implementation, Viswanathan and VanLehn [12] made use of tablet gestures and superficial speech features to measure levels of collaboration among pairs of students solving complex math problems. Vizcaino and colleagues [13] created and tested a collaborative, adaptive environment for supporting programming. The system compares group behaviors against known patterns of behavior to decide which content the group needs or what behaviors it should encourage.

Researchers have looked to the automation of collaboration scripts, i.e. teacher-formulated activities, guidelines, and procedures that structure non-computer supported student group work [4]. Examples of these scripts include the assignment of students to specific roles and the distribution of learning materials among members to force consultation (also known as jigsaw scripts) [2]. Rau and colleagues [6] successfully created adaptive collaboration scripts to support chemistry learners. Their script posed questions that resulted in student discussion or prompted the students towards certain behaviors.

We observe that the proposed conceptual frameworks require the use of high fidelity data, including dialog and sensor data [1]. We also observe that these systems are usually deployed in complex problem solving STEM environments [6, 12, 13]. In this paper, we attempt to consider the questions from [8] in the context of Ibigkas! a team-based mobile game for English language learning. The game keeps interaction logs but makes no use of video or audio recording, or sensors. The objective of this paper is to collect design considerations and directions for the development of an adaptive version of the game. The design considerations and directions are drawn from the literature and from focus group discussions with students and teachers representing the game’s target user groups.

2 An Overview of Ibigkas!

Ibigkas! is a collaborative drill-and-practice style game that helps learners develop fluency in identifying rhymes, synonyms, and antonyms in English. It was developed (as discussed more fully in [7]) by the Ateneo Laboratory for the Learning Sciences of the Ateneo de Manila University in the Philippines, and it is available free of charge for both Android and iOS. It was intended for use by under-resourced students in grades 4, 5, and 6 students in Philippine public schools, where English is an official language, but is not necessarily the language of instruction.

Ibigkas! allows both multiplayer and single player modes. To play in multiplayer mode, each student must first have a mobile phone with the game installed. The game does not require Internet access, but each device must be connected to the same network hotspot in order to communicate.

When the game begins, a random player from the team receives a target word (in Fig. 1a, the target word is KIT). All players receive lists of words, only one of which is the correct answer, i.e. the rhyme, synonym or antonym of the target word (in this example, the list of words received are MISS, NO, WRONG, ANOTHER, HIT and BAD and the correct answer is HIT which rhymes with the target word KIT, in Fig. 1b).

Fig. 1.
figure 1

Sample of a rhyming task. (a) Player 1 has target word KIT. (b) Player 2’s screen has the correct answer HIT.

Full size image

The player presented with the target word must say it aloud, so that the other players can hear it. The requirement to say the word aloud is the origin of the game’s name, as ibigkas is the Filipino word for “pronounce” or “say out loud”. All other players then check their list of words to see if they have the correct answer. The player with the correct answer should say the answer aloud and tap it. Once the correct answer is tapped, the round is over and a new round begins.

Because data suggests that mobile phone access is relatively common among poorer Filipino families, a non-collaborative version was also designed. This single-player mode enables students to practice on their own. In this mode, the target word’s answer is always among the three choices at the bottom half of the student’s screen.

However, even if the students we are hoping to serve with Ibigkas! may have access to a phone for individual practice at home, their schools are unlikely to provide one-to-one mobile devises, and even less likely to be able to provide reliable network or Internet access. For these reasons, a card game equivalent of Ibigkas! (See Fig. 2) has also been designed. Like the mobile game, printable cards and rule book are also available for download, free of charge, from http://penoy.admu.edu.ph/~alls/downloads-2.

Fig. 2.
figure 2

Samples of the Ibigkas! printable cards showing the three types of linguistic relationships that the game teaches (synonyms, antonyms, and rhymes).

Full size image

In its simplicity, the multiplayer version of Ibigkas! fits the criteria for a jigsaw CSCL. The learners have a piece of the question and the answer. Bringing question and answer together requires working together. At the time of this writing, the Ibigkas! mobile game is not adaptive. Players can select levels of difficulty, but the game has no native intelligence that enables it to make automatic adjustments based on players’ behaviors. To consider an adaptive version of this game, we first have to identify the characteristics of collaborative activities in the normal classroom, without technology interventions. From these characteristics, we draw considerations for the design of adaptive collaborative support.

3 Workshop Sessions

In early February 2019, we conducted workshop sessions with Grades 4, 5, and 6 students and English teachers in a public elementary school in Quezon City in the Philippines. The workshop session consisted of Ibigkas! game play followed by focus group discussions. Participating teachers and students and their parents gave their written consent to participate in the study. Immediately after the focus group discussions, teachers were given a token of PhP200.00 (approximately US$4.00) while students were given a token of PhP50.00 (approximately US$1.00).

3.1 School and Student Profile

School A was one of two participating schools in the study reported in [7]. As of 2017, School A had 7,419 students. Class sizes averaged 50 students. Dependent wholly on government support, the school is often under-resourced. Teachers told Rodrigo and colleagues [7] that they bring their own laptops, projectors, and Internet hotspots to class, which they procure at their own expense. During the focus group discussion, teachers added that even traditional materials like colored markers were already in short supply, since the school was ending in four weeks.

The profile of the student population mirrors that of the school. Most of the students come from difficult socio-economic circumstances. While some parents hold managerial positions, many work as day laborers or at jobs with relatively low skill requirements. Many of the children have to work before or after school to augment family incomes [7].

3.2 Study Participants

Six teachers participated in the workshop, two from each grade level. All teachers were female. When combined, they shared 82 years of teaching experience (see Table 1). Eleven grade 4 students, 12 grade 5 students and 12 grade 6 students joined the workshop. Ages ranged from 8 to 13. 46% of participants were male. Most students owned their own cellular phones (see Table 2). For the purposes of the workshop, the research team brought cellular phones owned by their laboratory, with the game already installed.

Table 1. Teacher profile.
Full size table
Table 2. Student profile
Full size table

3.3 Data Collection Methods

Each grade level had separate workshop sessions lasting about one hour each. All sessions took place within the same day. The students were divided into three groups of three to four members each. One trained facilitator was assigned per group. The teachers were in a separate group with their own facilitator. The first author of this paper began each session with a reiteration of the consent form’s contents. We then asked the students to complete a brief demographics questionnaire.

Once the students finished answering the questionnaire, the facilitators taught their groups how to play Ibigkas! The students and teachers played all versions of the game—the mobile game, in both single and multiplayer mode, and the card game. The facilitators then conducted the focus group discussion in which they asked members of their groups two sets of questions. The first set was about the game experience and the second was about group work within their classroom context (see Table 3).

Table 3. Focus group discussion questions.
Full size table

4 Results

We discuss the results of the focus group discussion, with emphasis on the responses to the second set of questions. From the first set of questions, we focus on the responses to the questions about the mobile game rather than the card game, to stay within scope.

4.1 Game Experience

When teachers and students were asked for their opinions about the game and the game experience, much of the feedback was positive. They found the game easy to learn and fun to play. The students said they learned new words.

Many students said that they enjoyed the multiplayer mode of the mobile game. They found it exciting, and they liked that they were able to help each other if they didn’t know what to do. The multiplayer mode gave them the feeling that they were “in it together.” They also enjoyed seeing teamwork improve over time.

The teachers also liked the multiplayer mode. They agreed that it was exciting and challenging. They liked that it required students to stay alert and provided opportunities for interactivity with both the device and with classmates. For example, one teacher praised the fact that students could check each other’s pronunciation.

Both teachers and students had a number of suggestions. In terms of content, they suggested adding more words to the program’s corpus, especially words that appear in the students’ textbooks, and they hoped that we would consider expanding to other languages or even other subject areas (e.g., building games to teach mathematics). They suggested categorizing words by types (e.g. verbs vs nouns). Students indicated that they were motivated to improve their game performance by studying, even suggesting that the game give them time to take notes or else provide a reviewer so they could memorize the word lists.

While teachers said that they liked the collaborative nature of the multiplayer mode, they made an almost contradictory request: increase the competition between students. They wanted individual students to be able to level up and they wanted the game to track student rankings, which would require the game to track individual progress.

4.2 Group Work

Group work is a staple teaching/learning format in Philippine public schools, sometimes to a fault: one teacher shared that the latest basic education curriculum required group work formats, whether or not they were appropriate for the subject matter.

The students said that they liked groupwork because working as a team makes school work easier. They could share ideas, help each other, and in the process have more fun. Individual work, one student said, was boring.

The teachers offered a more nuanced perspective that reflects the culture of Filipino classrooms. They said that students liked groupwork because it gave them license to chat. Also, while it is observed that in some cultures, the stronger students would prefer to work alone than to learn with peers [14], they interestingly reported that it is the better students who prefer group work. The less proficient students preferred to work individually because incompatibilities in working styles led to arguments or fights between classmates.

Students said they were not often given a chance to choose their groupmates. When they did, people tended to gravitate towards the smart students. Hence, groups were usually assigned by the teachers using a variety of different strategies: they had students count off, they grouped students by rows or alphabetically, and so on. The teachers said they tried to make sure that all groups had a mix of stronger and weaker students.

Group sizes tended to range from four to eleven members per group, although one teacher said she only grouped students into dyads or triads. These large group sizes were consequences of large class sizes and limited classroom space. Groups often had to spill into the corridor in order to convene.

Once groups convened, students said that they would make sure that everyone understood the task and shared the work. They tried to teach each other, give feedback, assist, and direct those who were lost or confused. It happens on occasion that students disengage from the learning task, partly out of boredom or lack of interest. They refuse to cooperate, preferring instead to chat or play. In these instances, students tried to self-manage by scolding these members, giving them work to do, or reporting them to the teacher. Teachers then try to address the specific needs of these members. They break down the task into simpler steps or give alternate tasks that might appeal to student interests or abilities.

Teachers, in the meantime, went from group to group to check on group dynamics and the learning process. They also reported using other strategies to ensure collaboration, some of which may not be typical in other cultures. For instance, Filipino teachers sought to make the smarter members of a group responsible for less-able peers. Their grading rubrics include group discipline, which subsumes teamwork. During the evaluation of groupwork, teachers ask each member to account for their individual contributions to the group, but it is also common for teachers to assign a single grade to all members of the group. This strategy is meant to make group members responsible for each other, but student leaders within the group are, in some extreme cases, allowed to drop a particularly uncooperative student from the group roster. When this happens, the student who has been removed from the group receives a lower grade than the rest of the group or no grade at all.

5 Design Directions

The game format of Ibigkas! circumvents many of the problems and issues that the teachers and students raised in our pre-design workshops e.g. students disengaging from the tasks and refusing to work with the rest of the group. Students are automatically excited to try it out. The instructions are relatively simple. Students must pay attention or the entire group suffers.

We now attempt to reconcile the other focus group discussion responses with what we know of adaptive CSCL, referring in large part to the characteristics from [5].

Pedagogical Goals.

Given the simplicity of Ibigkas! it seems inappropriate to aspire for the system to guide learners towards more productive interaction, at least during game play. In this game’s context, metacognitive support might be the most appropriate pedagogical objective, as students mainly need to stay alert in order to succeed in the game.

However, as discussed below in the subsection Targeting specific students for intervention students could also be pushed to try words that are considered more difficult—based either on measures typically employed in Natural Language Processing (NLP) tools (e.g., age of acquisition) or based on the students prior performance with the word and/or its linguistic properities.

Likewise, it might be possible to have the game incorporate new words, either based on past student performance, or with some input/guidance from their teachers. For example, future designs of Ibigkas! could allow a teacher who is aligning their English instruction with the science curriculum to specify that students practice the vocabulary associated with those lessons.

Targeting Specific Students for Intervention.

Teachers said that they try to form heterogenous groups, with a balance strong and weak learners. Ibigkas! only provides a group score; there is no differentiation in score among members. For the system to support group formation, it would have to track individual students’ performance and report these to the teacher.

One way to implement individual tracking is by penalizing students who answer incorrectly or who fail to tap the correct answer when it appears on their screen. If the system tracks individual performance in this manner, it would be possible to accommodate student and teacher suggestions for levelling up, ranking, and overall providing a more competitive environment.

This also opens the door for content adaptation. While in-game, the level of difficulty of the content could be increased or decreased, depending on student performance. That is, students could be given progressively less-common synonyms or antonyms for a target word they have proven familiarity with. Likewise, if a student seems to have mastered the rhymes involving one phonological class of sounds, the game could shift these examples out of rotation and instead require students to practice words containing other classes of sounds. Future versions of the game may also take advantage of the devices being connected to each other to learn how difficult the words are as more games get played. Words that majority of the students seem to get wrong may be labelled as more difficult. After some time, a hierarchy of word difficulty for a given context of respondents may materialize and this could be made as additional basis for shifting the in-game difficulty.

While the design of the multi-player version of the game, especially, does not lend itself naturally to hints or clues, it may be possible to provide bottom-out hints. Particularly when the game is being played in single-player mode, it could be useful to prompt students to advance to the next set of words. This could be done by, for example, having the correct answer change color if the student does not correctly answered within a certain amount of time. Empirical research is needed in order to determine the appropriate amount of time to allow a student to struggle (e.g., [5]). However, in general, finding some way to discouraging them from wheel-spinning, as in [15]), is probably an important design strategy.

Building on student suggestions, future designs of Ibigkas! could also provide post-game notes. These notes could be personalized to individual learner needs, but with further help from teachers, they could also be provided to encourage group study activities outside of the game. For example, students in a group that has struggled with a particular set of synonyms or antonyms could be directed to specific reading/writing tasks outside of the game.

Providing students with content support would hypothetically have a positive impact on peer interaction. Because the game mechanic mainly requires knowledge of the subject matter and attention, students who know the subject matter should be able to perform better at the game.

Modeling and Technology.

The domain needs to be structured based on level of difficulty of the words. At the moment, a language expert binned the words in the Ibigkas! corpus based on word length and familiarity, and words have been semi-automatically coded for phonological (sound-based) patterns that are related to the rhyming tasks. Moving forward, it may be possible to automate this process using NLP tools, however, we will need to take into account contextual factors (i.e. a word that is familiar in a Western context may not be as familiar in a Philippine context), and we will also need to consider the perceptual factors that differ among non-native speakers when they are asked to identify rhymes.

As discussed earlier, learner performance can be tracked and modeled in terms of the timing and correctness of response patterns, and these transactions might be able to provide further information about group collaborations. For example, a student who is repeatedly mispronouncing a word (e.g., the player in one of our workshops who pronounced gorgeous with two hard “g”s, resulting in gor-gee-us) might impede his or her classmates from correctly identifying the synonyms or antonyms in their lists. One can imagine, for example, that if the group’s response is delayed each time the target word appears on one particular student’s screen, it could be that the group needs help remediating that student.

Design Space.

Automatically changing difficulty levels would constitute an implicit design because it takes place without providing the student with directions, clues, or hints.

The design is both indirect and direct. It is indirect because when in-game difficulty is shifted/adapted for one person, the way the game is played allows this change to affect the whole team. A student who advances in difficulty will, as per the rules of the game, cause team mates to receive answers that match the question originally for the target student. This allows the target student to gain the adapted instruction indirectly through another student.

It is direct because at certain points in the multi-player mode (where the correct solution appears in the device held by the target student) and throughout the singe-player mode, the instruction is directly received by the student it targets.

6 Conclusion

We describe design considerations and directions for an adaptive version of Ibigkas!, a collaborative mobile game for English language learning. Based on a reading of the literature and feedback from teachers and students who participated in a focus group discussion about the game, an adaptive version of the game would include metacognitive support, a scoring system that enables monitoring of individual performance, adaptive difficulty level adjustment. Post-game content support would be an added benefit, especially if it addresses learners’ individual needs.

While it does not have the same level of sophistication as the systems discussed in [6, 12, 13] the simplicity of Ibigkas! might also be its strength. The game offers an opportunity to design a few simple, adaptive interventions and measure their effects on the target population. Future work will look into the possibility of implementing these adaptations and testing them in the field.