Keywords

1 Introduction

Universities can be thought of as huge information spaces and indeed one of the problems with things like embarking on the student voyage (e.g. Fresher’s Week) is the amount of information the traveller has to deal with. All users face some aspect of this problem. In this paper we will deal with how we can personalise this choice mechanism. The first part of the paper represents a generalisation of work on personalisation for special needs [1,2,3,4,5,6] to employ an Ontology-Based Community of Agents for Personalisation of Services for students in general. The second part looks at how we can use same mechanisms to personalise group project undertakings and assessment. What this paper brings out to the fore is the AI Agent Based Deliberation mechanisms that underpin this retrieval and presentation process. The central aim of this work is to deliver a personalised service to students. One that works for individual needs but is flexible for individual desires.

The problem with the amount of information available to students is the classic “woods for the trees” dilemma. Potentially there is too much information out there – what we have to do is find the information that is needed and weed out the flotsam and jetsam of the sea of information. One way to do this is to offer better ways of personalising this information space so that users see only what is best suited to their needs, desires, and profile. In order to do this we can use AI as an editorial underpinning. Semantic Technology allows us to organise information in a smart way. At the heart of semantic technology is ontology based knowledge representation, and to utilise this we require a representation at a knowledge level [7]. However, merely representing your information in the right way is not enough - we need ways of operationalising this information. Then we can use a small society of agents to rationally operate and reason about this information. This paper will demonstrate how this can be achieved and give an example of it in use.

In the second half of this paper we will discuss how the above can be taken forward to achieve hybrid group working. We will discuss some of the important design issues and how we can bring this together into a proposed architecture that would allow mixed group working within the context of the formal academic assessment.

2 Knowledge Navigation

Clearly one thing that computers are good at is crunching data. The data/information versus knowledge/wisdom debate is played out elsewhere (e.g. [8]). Semantic Technology represents a new viewpoint for this discourse and focuses on a higher level of dialog of interface between users and technology. In this section we will discuss some knowledge ordering principles before going on to discuss technical solutions in the following sections. We consider in turn semantic knowledge representation, AI and Agency, and Individual perspectives of knowledge.

2.1 Semantic Knowledge Representation

The centre of this approach is the representation and use of knowledge and meaning. Into this we introduce the concept of knowledge engineering as a method of structuring and ordering this material. Ontologies provide ways of ordering, structuring, and storing knowledge. For knowledge engineers, they can then be used to drive problem solving. This historic approach naturally evolves into Semantic Technologies. The specific problem solving that we are concerned with here is how to customise and personalise information and services for general learner needs within a Domain Specific university context and the Ontologies developed reflect this.

2.2 Agency and AI

Having the knowledge is not enough; we need to do something with it. Agents (e.g. see [9]) provide autonomous ways to architect our AI that allows us to consider different aspects to our domain. What is actually an agent has a wide definition running from simple reflex devices as seen in animals and modellable by Finite State Machines, through to full cognitive architectures that can be an agency like the SOAR implemented in the QuakeBot [10, 11]. In the work presented here they are used both as architectural, structuring, elements in their own right and to provide beacons for knowledge navigation. They can thus be used both as order making devices within the semantic technology itself and also reflect important dialog players within a group context. In this way they have a dramatic effect on the team dynamics in the manor of playing a character – similar to Laird’s use of agents above.

2.3 Personalisation and the Learning Space

In the context of providing an environment for learning, the enhancements that technology allow lie in the flexibility it can provide. Such flexibility can be in terms of the where, when and what of learning (see [12]). Computer based learning environments – whether a traditional virtual learning environment (VLE), or a fully immersive simulation of a learning space – can offer flexible and adaptive support for learning and assessment, from selecting and providing tailored content through to adaptive tasks and tests that respond to the apparent skills and capacity of the student user [13]. With flexibility comes the opportunity for the user to personalise. They may want to do this as a navigational device to deal with large volumes of data. This might involve varying the level of detail of view, compressing information, abstracting information or defining their own visualisations of the large domain data [14]. Brayshaw [15] extended this so that agents could be used as a basis for constructing customised views of a large search space which was the trace of a parallel program. At other times their need for personalisation may be driven by specific preferences to reflect taste. Other students may have specific needs like a disability (or disabilities) and need to tailor their services accordingly (e.g. [5]).

3 Using Ontologies and Agents to Personalise an Individual Student’s Experience

On the web – given the vast array of information - users are more likely to interact with information that is personally tailored to their needs rather than general information that may not be of interest to them. Similarly, when learning online and searching information, time might be of the essence, especially when learners are trying to meet certain deadlines. Hence, in the e-learning domain, learners will benefit from personalised services as it will save time and will also be particularly helpful for learners with disabilities. In order to accomplish personalisation, some vital considerations include focusing on the following as depicted in Fig. 1, with the following components.

Fig. 1.
figure 1

Using ontologies and agents to personalise services for a single user.

Full size image

3.1 Users

The users have various characteristics and needs. They could be users with special needs due to a disability, or they could have other needs brought about by their age or to represent learning styles. For users with disability, special accessibility considerations need to be made to ensure that they are fully included [16]. However, given that an ontology captures their needs, the method herein ensures that their needs are adequately met. In the e-learning domain, learners have specific goals which could be readily achieved by capturing their needs and preferences. When ontological design and development captures these needs and accurately represent the learner, it would facilitate personalisation of learning.

3.2 Client Interface

The users first interact with the ontology through an intelligent service interface through which they can manipulate the ontology such as directly making changes to it through updating or deleting information which is held about them. Indirectly, more information could be collected in the ontology based on user behaviour such as their interests over time which could also be inferred from their browsing patterns. If a disability-aware e-learning system for instance intelligently produces accessible formats of learning materials but the client interface is inaccessible, this could prevent most users with disabilities from accessing the content. Thus the client interface needs to also meet accessibility and usability standards in order to better respond to the needs of the user.

3.3 Ontologies

The semantic web offers a fantastic opportunity for collaborative provision of learner needs due to its ability to provide information to users in a meaningful way. The Web Ontology Language (OWL) can be used to produce ontologies that will capture vital information needed for provision of service. This information is collected about the users which includes their needs and preferences and the services that are available, which due to the explosion of information in this information age, is very vast; personalised services can be offered based on this information. Thus, a user profile ontology could be created to capture vital information about the user which could be updated as the user characteristics change probably due to age, an improvement in their situations (for those with disabilities) or a degeneration of their situation (such as acquiring other disabilities and thus having multiple disabilities; for those with disabilities).

An Agent based inference mechanism ensures that both the user and their requests are checked against existing services and the ontology to determine their existing needs and preferences and then transform the information into formats that meet the needs of the user. For a student who is completely blind for instance, audio and/or text-based formats of learning materials could be generated and presented to the learner.

3.4 Services

Users may need access to various services, again using an agent based model, which need to be personalised. Such services for instance could be e-learning, m-learning, e-commerce, etc. Due to the fact that most designers and developers of such services usually develop them without considering the needs of people with disabilities, some of these services might not be fully accessible to some users (such as those with disabilities). A learner for instance might want personalised course information from an e-learning service or personalised health information from an e-health service.

4 A Framework for Working in Hybrid Groups

We now demonstrate how to use agents and the technology described in Sect. 3 to personalise an individual’s learning experience in a group working context. Learning can be a lonely experience, if it always has to be done in isolation. Working in groups has a long established didactic standing (e.g. [17]). There are some very pragmatic reasons motivating working in groups:

  • We wish to simulate work as they will experience it when they leave education. For example, in Computer Science to prepare for working in a team of software developers.

  • In such a team one person is not going to be able to write the whole of an app so team working is an inherent part of the process.

  • Specialised expertise exists so groups can be more powerful than individuals.

  • The power and importance of peer support and the encouragement that this may bring.

However working in groups has its downside for example:

  • It is unfair when people get a very bad group and end up having to do all the work.

  • It is unfair when people get a very good group and poor colleagues are carried by the collecting momentum.

Working in a group can be harder than working solo. There are personality issues, ego, politics, fallouts, relationships, and group dynamics going on. If you are very technically competent it can be very frustrating working in a mixed ability group. The eventual mark a student gets may not reflect their individual efforts or ability, or indeed their ability to work in a group, but may be the product of a particular social adventure. There are approaches to manage this scenario: with peer assessment of team work [18]. However, the motivation for the work reported here is to investigate how we could enrich the benefits of group working, by providing homogeneous groups that are all similar, allowing the candidate to interact, thus enabling the derivation of an individual mark. To interact like this we need other agents within the group. When we interact on the internet (e.g. via Facebook or Twitter) the assumption is that the agents we are talking to are other people – although this is an assumption that with the growth of Chatbots is not always the case. Here we will argue that if the degrees of freedom in the dialog is relatively constrained – say within the context of a technical design task or evaluation – then we can use software agents, and the same semantic technology as before, to participate in this process.

To achieve this we are going to turn to AI, and need to select an AI to use. For the purposes here we can take a liberal definition and define AI as anything that passes the Turing Test [19]. To be a partner in a group exercise one has to fulfil the role of a group member. Now the actual roles of these members may differ (e.g. [20]), so that the type of AI we might need to functionally implement may differ [21]. Considering a functional definition of AI from the Games context, it may vary from Finite State Machines approach to a full utilitarian AI (e.g. [22]). In the context of Game AI, as a minimum we require an interaction with a non-playing character (NPC) that is plausible and can convey the necessary narrative of the game. To do this it is not always necessary to have a full Knowledge based AI and we can instead utilise a look up table or Finite State Machine approach. This is how many chat bots or vreps actually work; they are not a fully functioning AI but are based on an Eliza like application [23]. Indeed, many of the chat bots that compete for the annual Loebner Prize [24] fall into this ilk. At other times a fully functional and reasoned AI is called for. Thus in this model, AI can be thought of as constituting a range of functionalities, depending on context, as illustrated in Fig. 2.

Fig. 2.
figure 2

A range of agent based architectures

Full size image

Here we adopt a black-box approach to the implementation of the AI and are concerned only how it resolves the function in the group. We propose three basic Agent Building Blocks. Reflex Devices are implemented as state machines. These are state agents and a state definition language is provided for them. Knowledge Agents have their own inference engine that provides forward and backward chaining productions, object-like permanent memory, truth maintenance, and uncertain reasoning systems. Axiological agents rather than just applying rules to a situation aim to reflect on the value of an action to an agent and purposely choose what to do next based on that judgement. This type of reflection is important in group dynamics.

4.1 Degrees of Freedom in Dialogs

So what are the reasons for a distinction between the types of agents required? Critical to this is the degree of freedom in the dialog. If the dialog itself is well constrained e.g. of a technical nature, then there are limited degrees of freedom about what can be asked and what responses a rationale correspondent can make. For example if we are in the context of configuration design there are a limited set of design choices that are available to the designer, the configurations, and the dialog is essentially one of enumerating these choices [25]. If we are in the context of teaching how to build a PC or design a local network we can start the dialog from a clear fixed point – for example from some requirements capture exercise which may be as basic as a questionnaire or hypertext dialog (which is another interface to the FSM mechanism above). Once we have our initial starting point then we can map out our dialog from here. This can be represented as essentially a decision tree and implemented as simple state machines.

However if we want a more intelligent collaboration then we need to consider our choice points in the dialog construction. To this end we propose two methods of doing this. One is essentially using a rule based system. For each choice point in the dialog a knowledge based inference can decide what to do next. The second method is a Utilitarian Agent mechanism. Each Software Agent can have their own agenda. In this manner from a pedagogical perspective they can be engineered to follow a particular role in the group ([26]). More specifically an agent can have characteristic beliefs, desire, and intentions that inform any particular dialog choice point. Equipping an agent with their desire and beliefs allows them to take their own attitudinal stance to dialog. We thus propose to enable agents to become character agents.

How does this affect working in groups? The above allows us to potentially construct hybrid groups of people and agents. It allows us to invest groups with particular characters. Hence we can have one individual student who is being assessed but in the proximity of other contributing agents. Knowledge-based agents can inform according to their insight. Utilitarian agents can act on more axiological grounds.

The key here is limiting how smart the AI has to be. We have noted that if the dialog choice can be cut down to the point where a state-machine can decide on what to do next then things are much simpler. If we take as an example one of the seminal programming language tutoring systems [27] this constrains the language and dialog to the core. A clear task was defined – to write a LISP program – but the names of all the functions and variables were prescribed by a fixed vocabulary. Whilst this at first sounds like a limiting constraint it places the task within the confines of current AI. Sacrificing vocabulary is a trade-off for greater interaction with AIs.

4.2 An Example

Let us take an example task. Say we are teaching an undergraduate HCI course. The assignment that we wish to set is a group project on Heuristic Evaluation where we wish to place our students in a group with a technical expert/specialist, a management expert, an implementer, and a developer. In the simplest form the student works through a dialog with each of their co-workers. The dialog can result in either a state transition based output or an inference based one. The output is an expert response to a final report. Based on their deliverables the student then has to edit their outputs into a coherent final report. The student thus has to reflect, synthesize, and enhance the contributions of their fellow workers. What they have to work on reflects on how they have interacted and worked with their fellow group workers. Furthermore their final deliverable is the sum of their interaction and their own contribution in the process of the group work. In this way we can give individual marks based on common groups. What each student had to work with is a common base. What they end up with is as a result of their interaction with common experts and their cut and interpretation of the group’s interaction.

5 Conclusions and Future Work

In the work presented here we have discussed how we can use Agents and Semantic Technology to personalise individual student services. Secondly using the same approach we have shown a brief introduction to automating group assignments and assessment. With current trends in ubicomp [28] and the development of the MOOCs movement (e.g. EdX [29], Coursera [30], Canvas [31], or FutureLearn [32]), and criticisms thereof [33], how we deal with large numbers of students within a single cohort becomes a big issue. It is clearly desirable to give individual feedback where possible. At the same time we need to educate and prepare our students for the real world. Developing true scale software deliverables involves many person years of development effort. As such they will need to work in groups in order to achieve the above. As educationalists we therefore need to provide training for this type of working. However there is always frustration with group working in that we know individuals can carry a group and that the final mark derived may not always reflect an individual’s contribution. By providing a common surface we here aim to let a single user interact with other agents and they together produce a group output. That we provide the same surface to multiple users means that an individual mark may be derived. In this paper the task has been heavily constrained and the degrees of freedom of dialog restricted. This is a realistic constraint within many educational contexts. For example if we wish to teach someone how to build a jet engine then there is a limit to the degrees of freedom in the task. Components fit in a certain way – there is a set way of engineering the task. In software engineering there are clearly more options although we may wish to steer our students in certain ways. Thus the choice of dialog options may be larger.

Where the degrees of freedom in dialog are limited then simple agents can meet our needs. A Finite-State Machine may resolve the issue. However if more reflection is required we provide a full knowledge based inference system and a utilitarian agent package.

Where we are going with this work is to address more discursive domains where the constraints on task are not so limited. Part of this wider range of functionalities could be to implement other characters e.g. the full range Belbin [24] proposed is Team Roles. Thus we could personalise the agent group further to give our learners scenarios that reflect on specific group make ups.

A Semantic Approach cannot only change the content of learning packages but can also change the culture of learning. The chalk and talk of a traditional lecture theatre centred campus is not going to satisfy an increasingly sophisticated clientele who are used to a rich media online world. Users interact with media in a flexible way and to be relevant in the future we have to change the gestalt of learning and the university experience. We can only do that by looking for a root and branch change to the user experience. What we have looked at here is how to use AI and Semantic Technologies to start to make this happen.