Keywords

1 Introduction

Adaptive instruction is a critical concept in realizing self-regulated instruction where computer-based intelligent tutoring systems (ITSs) jointly manage the pace, flow, and complexity of instruction along with the learner. To begin a discussion of adaptive instruction, we should first discuss what we mean by “adaptive” and how adaptive systems differ from adaptable systems. Adaptable systems may be tailored by the user to support individualized needs or preferences. Adaptable systems offer flexible control of information and system performance resides in the hands of the user [1]. A good example of an adaptable system is a smartphone or tablet where the user can change the interface, layout of applications and decide on specific applications to support the recall of information toward their educational or entertainment goals.

Adaptive systems demonstrate intelligence by altering their behaviors and actions based on their recognition of changing conditions in either the user or the environment. This change is usually managed by software-based agents who use artificial intelligence techniques to guide their decisions and actions [2]. Adaptive instructional systems are “intelligent” in that they are able to observe and interact with both the learner and the training environment (Fig. 1) based on their ability to recognize “learning opportunities” and “teachable moments”. Learning opportunities include instances where the adaptive system is aware that the learner is at a point in the instruction where they are able to bridge from existing knowledge to new knowledge. Teachable moments are opportunities for reflection by the learner after a new experience and may be guided by the adaptive system through the use of metacognitive prompts.

Fig. 1.
figure 1figure 1

Adaptive training interaction

Full size image

The motivation for developing/maintaining shared mental models of cognition is much the same as for maintaining individual models of cognition. For individuals, we refer to the learning effect model (Fig. 2), where selective mining of learner data (e.g., behaviors and physiological sensor inputs) informs learner states (e.g., cognition, affect), which inform strategy and tactics selection by the tutor and ultimately influences learning gains.

Fig. 2.
figure 2figure 2

Learning effect model [3, 4]

Full size image

Better models of learner cognition are purported to result in more accurate strategy and tactic selection and result in improved learning (e.g., knowledge acquisition, skill acquisition). They most certainly result in more efficient learning as content is tailored to provide instruction on either new material or remediation of old material where the learner is struggling to grasp needed concepts. This results in more efficient learning in terms of instructional contact time and in reduced time to competency when compared to traditional classroom training where everyone is in lockstep.

We define learning gains to include five dimensions: knowledge acquisition, skill acquisition, performance, retention, and transfer. Knowledge and skill acquisition define traditional elements of learning and are elements of learner potential. Performance is a measure of actual ability to apply knowledge and skill to a specific task. Retention is the ability of the learner to recall knowledge so it might be applied. Transfer is the ability to carry learning from training contexts to operational contexts (e.g., from practice to the game as a sports analogy). These concepts may also be applied to teams.

However, “the use of fully automated, computer-based tutoring technologies to provide instruction for teams is as embryonic as the problem space is complex” [5]. In order to determine optimal strategies and tactics for team learning, performance, retention, and transfer, it is essential to assess the collective states of the team through the use of shared mental models.

2 Shared Mental Models

Shared mental models must be able to represent team objectives, individual roles, and the actions of both individual team members and the team as a whole which are needed to achieve those objectives (Fletcher & Sottilare, 2013). Standards (measures) are needed to determine levels of expectation for learning, performance, retention, and transfer and to assess the effectiveness of the strategies, and tactics implemented by the adaptive system (e.g., intelligent tutoring system – ITS).

These models represent various aspects of the team’s perception, decision-making, problem-solving, and interaction. Sottilare, Holden, Brawner & Goldberg [6] suggested the following shared or team models:

  • performance (measures of accomplishment during execution of an action, task or function);

  • competency (measures of past accomplishments as a guide to probability of success for future performance);

  • cognitive states (measures of learning and retention plus engagement and workload as a moderators of learning);

  • affective states (assessment of emotion as moderators of cognition/learning);

  • trust (measures of confidence in other team members to carry out assigned tasks);

  • communication (measures of interaction which may also indicate levels of trust and competency of team members).

As Fig. 3 suggests a learning effect model could be extended for teams and then specifically adapted to focus on shared mental models, but which models are most important to team learning and performance, and which are more than just the sum of the states of individual team members?

Fig. 3.
figure 3figure 3

Notional learning effect model for teams (Fletcher & Sottilare, 2013)

Full size image

3 Shared Mental Models in GIFT

In this section, we discuss how shared models of cognition and affect might be applied in a tutoring architecture like the Generalized Intelligent Framework for Tutoring (GIFT; [3]) to effectively instruct teams. As mentioned previously, initial analysis of the literature resulted in definitions for performance, competency, cognitive states (e.g., attention, workload, and engagement), affective states (personality, mood, and emotions), trust, and communication.

Additional collaborative research between the US Army Research Laboratory, the University of Central Florida and Iowa State is beginning to uncover an interdependent model of team tutoring strategies based on categories of team behaviors, mutual support, team cognition, and team affect. Each of these categories was analyzed with respect to their influence on team performance, team learning, team satisfaction, and team viability with the goal of implementing viable models as strategies within GIFT.

GIFT is a largely open-source adaptive tutoring architecture whose goals include the reduction of time and skill to author ITSs, automation of instructional management processes during one-to-one and one-to-many tutoring sessions, and the development of tools and methods to evaluate the effectiveness of adaptive tutoring technologies. GIFT development has been fundamentally based on the learning effect model (Fig. 2).

The approach we used is based on a meta-analysis of the team performance and ITS literature and uses structural equation modeling to determine the significance of the relationship of critical team tutoring model variables. The primary search terms used in the meta-analysis were: performance, competency, trust, cognition, affect, communication, intelligent tutoring, human-computer interaction, virtual human, mood, emotion, skill, knowledge, ability, responsibilities, roles, distributed, virtual, after action review, feedback, leadership, cohesion, personality and effectiveness. Each of these primary terms was paired with: team, unit, group, squad, and crew resulting in the selection of over 20,000 articles for inclusion in the meta-analysis.

Preliminary findings have identified key relationships between team performance, team learning, team satisfaction, and team viability and the following behaviors: communication, coordination, reflexivity (self-directed action), conflict management, and leadership. There are also significant relationships between team variables (performance, learning, satisfaction and viability) and affective variables which include trust, collective efficacy, cohesion, and psychological safety. Team variables also have significant relationships with cognitive variables which include team or shared mental models, transactive memory systems, and situational awareness.

Shared mental models characterize the team’s objectives and behaviors for both individual team members and the collective group. Shared mental models also represent the actions needed to achieve goals and objectives [5]. These models may represent, but are not limited to team communication and coordination, team efficacy, and situational awareness.

Transactive memory systems support a team’s ability to curate or manage (store and retrieve) knowledge and consists of knowledge stored in each individual’s memory combined with metadata or labeling information each team members areas of expertise [7].

Team situational awareness involves the perception, comprehension, and projection of activity in the vicinity of the team and an understanding of how these activities might impact team goals and objectives, both immediately and in the near future [8].

4 Discussion and Next Steps

The relationship between team variables, their influence on each other, and their influence on learning, retention, performance, and transfer are not fully understood and merit additional investigation. However, we have identified moderators which influence shared mental models and thereby influence learning. These moderators have been broadly grouped into categories which include:

  • team structure, and

  • team culture.

Team structure includes variables such as interdependence of team roles, the distribution of team members, the autonomy of team members, the size of the team, the length of the project, and the project’s perceived importance.

Team culture influences shared mental models through the values of its members, the collective values of the team, and norms established. The stability of the team and its history of success also influence the confidence and efficacy of the team in tackling new tasks.

Next steps will be to construct team performance and learning models with associated techniques or policies to allow GIFT’s intelligent agents to adapt (observe, assess, and act) to address sub-optimal learning states. Identifying shared states or mental models is step one of a multi-step process. Measures or identifying traits of these shared states may be indicated by observation of behaviors demonstrated by individual learners or collective groups of learners. The most difficult question to be answered is what to do once a sub-optimal learning state has been identified. In other words, what actions are available to the adaptive system and which of these actions provides the highest reward relative to learning and performance outcomes given the state(s) of the team.

The adaptive system must be able to correctly identify the state, analyze trends, formulate a plan for action (e.g., a strategy), and finally execute the plan by implementing a tactic or action per the learning effect model for individuals (Fig. 2) and the learning effect model for teams (Fig. 3). In order to successfully implement a team tutoring structure within GIFT, it must be able to support and uphold policies which are based on sound instructional design principles for both individuals and teams, and which account for individual differences related with learning, performance, retention, and transfer [9]. Just as GIFT’s instructional management model for individual training is based on Merrill’s Component Display Theory [10], we also anticipate that team tutoring constructs using shared mental models will also support presentation of domain rules and examples of success to the learners, followed by assessment of recall of knowledge related to rules and examples, and finally, immersion in a practice environment to apply knowledge and develop team skills.

Some of these shared states have more obvious action pairs than others. For example, if a task is highly interdependent and the team is relatively new, then communication takes on a higher value. It is simple enough for the ITS to recognize a lack of communication and prompt the team members to keep others informed of their actions. It is more difficult for the ITS to identify specific shared cognitive (e.g., situational awareness) or affective states (e.g., trust). To this end, we are developing a team modeling testbed (Fig. 4) to support experimentation to validate shared mental models developed from the meta-analysis. This testbed will be used to develop an understanding of the influence of key shared mental models on learning, performance, retention, and transfer, but will also be used to understand the influence of shared mental models on other shared mental models.

Fig. 4.
figure 4figure 4

GIFT effectiveness evaluation testbed methodology

Full size image