1 Introduction

Online examination or assessment is an essential component of online learning environment. In traditional online learning, examination is an embedded and integral part of the learning environment. Learning and examination are performed remotely and largely rely upon remote authentication protocols for security and invigilation [1].

The UK quality assurance code for practice suggests that online examination are vital for evaluating student skills against the learning goals [2]. The outcome of online examination is used to award grades, and ultimately a certificate or degree. This makes online examination a high stake process. Online examinations are conducted into both supervised and unsupervised locations. The supervised online examinations are invigilated face-to-face or proctored remotely. However, the unsupervised examinations are taken remotely without any physical invigilation. In an ideal scenario, a legitimate student accesses remote online examinations using secure authentication and completing the examination process according to the requisite examination protocols.

However, cheating and academic dishonesty is reported in both face-to-face and remote online examinations [3]. Researchers suggest that online examination provides more opportunities for academic dishonesty [4–6]. Lanier [7] conducted a study on 1,262 college students and found that cheating in online examination is significantly higher than face-to-face examination.

In response to authentication threats. We proposed a “Challenge Questions” approach for authentication of online examinees. A Profile Based Authentication Framework (PBAF) was developed, which utilizes “Login-ID and Password” and “Challenge Questions” for authentication of online students. Challenge questions have been widely used for credential recovery by corporate email service provider, and banks for identity verification. The traditional challenge questions approach implements personal and security questions as authentication token. In the conventional challenge questions approach, users are required to register their answers to challenge questions at the outset for authentication at a later stage. In our proposed method, answers to challenge questions are recorded during the learning process and used for authentication in order to access online examination.

In this study we have evaluated the use of image-based and activity-based learning journey questions in a “six week” online course. The usability analysis of image-based and activity-based learning journey questions is reported in the results section.

2 Background

Authentication is important to prove that a user is, who he claims to be. Threats to online examination due to lack of physical interaction in a remote setting. The authors proposed and developed a Profile Based Authentication Framework (PBAF) [8]. The PBAF approach utilizes challenge questions and login identifier and password for authentication of online students. The method was empirically evaluated for guessing and collusion attacks [9, 10]. However, our previous studies reported usability and security issues with the pre-defined text-based and image-based challenge questions.

The pre-defined challenge questions had inherent usability issues reported by a number of earlier studies [11–13]. In the context of collusion, where a student may share access credentials with third party via email or telephone, pre-defined questions may be stored for sharing. In order to discourage sharing of challenge questions with a third party, we implemented non-invasive activity-based learning journey questions in this study. The activity-based questions integrates the learning and examination process and verify student identity based on the learning footprints. However, the focus of this paper is usability of the image-based and activity-based learning journey questions.

The study is part of an ongoing research to evaluate use of challenge questions for authentication of students in online examinations. In response to our previous studies [9, 10, 14] which reported the usability challenges using pre-defined text-based challenge questions, we implemented image-based and activity-based learning journey challenge questions for authentication purposes. This study aims to:

  • Implement the activity-based learning journey challenge questions for authentication in online examination using the PBAF approach.

  • Evaluate usability of image-based and activity-based learning journey questions.

3 Profile Based Authentication

The PBAF is a knowledge-based authentication approach [8], which utilizes challenge questions and login-identifier and password features. Login-identifier and password based authentication is used for the initial login to access the learning resources, whereas challenge questions authenticates online examinees. The authors designed and evaluated different types of challenges questions: pre-defined text-based questions, image-based questions, and activity-based learning journey questions.

Figure 1(a) shows the PBAF approach, which utilizes pre-defined text-based and image-based questions for authentication. Students are required to record answers to pre-defined questions. Answers to questions are stored into individual’s profile. Challenge questions are extracted from individual’s profile during authentication process. Challenge and profile questions are the same entities used in the learning and authentication contexts as shown in Fig. 1(a).

Fig. 1.
figure 1

Profile based authentication design

Full size image

Figure 1(b) shows the PBAF approach, which utilizes non-invasive programmatically generated activity-based learning journey questions. In typical online course, the traditional learning activities are a combination of lessons, assignments, forum discussion and quizzes etc. Student interaction with the learning activities trigger creation of challenge questions in the background and stored in the profile. As shown in Fig. 1(b), question generator creates activity-based learning journey questions and stored in a student profile. Challenge questions are extracted from individual’s profile for authentication in online examination process. Different types of challenge questions are described below:

3.1 Text-Based Questions

The text-based challenge question is a widely used question type implemented by a number of email service providers for authentication purposes [15]. Questions are based on individual’s personal and professional information. The text-based questions are further classified into fixed and open type questions [11]. The fixed-type questions are presented to users from a pool of pre-defined questions. The open-type text questions are users’ driven and users have full control over choosing question’s description and answer. Answers to text-based questions are received in free text form. The text-based challenge questions faces a number of challenges such as memorability, clarity, syntactic variation, which may cause security and usability issues [16].

3.2 Recall Image- Based Questions

The recall image-based authentication uses images of objects, nature and abstracts etc. to verify identity of users. Users are presented with previously chosen images to recall and identify their selection in order to authenticate [17]. The recall image-based authentication was implemented as recall image-based multiple choice challenge questions in the PBAF approach. Research suggests that image-based questions have advantage over the text-based questions in terms of memorability.

3.3 Cued-Recall Image-Based Questions

The cued recall image-based authentication relies upon individual’s recall ability, however, it is aided with a cue to help recall image selection [18]. The cued recall can either be a text-based information stored by the user [19] or automated retrieval cues [17]. Cued-recall image-based authentication can be implemented as cued-recall based multiple choice image questions in the PBAF approach, however, this study does not cover cued-recall based authentication.

3.4 Recognition Image-Based Questions

The recognition image-based authentication relies upon individual’s recognition ability and authenticate on the basis if an individual has seen or chosen a image before [18]. The correct image is presented with a set of distraction images and user is challenged to recognize a previously viewed or selected image. The recognition based authentication was implemented as recognition based multiple choice questions in the PBAF approach.

3.5 Activity-Based Learning Journey Questions

The activity-based questions are generated programmatically based on individual’s learning activity during the learning process [20]. In traditional online courses, learning is a set of pre-defined activities including lessons, forums, quizzes, assignments, chatting activities to name a few. Learners are anticipated to perform multiple learning activities during the learning process. The question generator creates activity-based questions, which is trigged on a student’s interaction with a learning activity. The activity-based learning journey challenge questions is an innovative approach to collect information about a student and utilize it for authentication. Characteristics of activity-based learning journey questions are:

  • Security: Answers to activity-based learning journey questions should be difficult to guess.

  • Usability: Activity-based learning journey questions should be easy to recall with better usability.

  • Non-Invasiveness: Unlike traditional challenge questions, where users are required to register answers at the outset, the activity-based learning journey questions are created in the background without interrupting users for registration of answers.

  • Adaptability: The activity-based questions are short lived and status of automatically created questions are checked and refreshed instantly.

4 Research Methodology

This study was organized to evaluate image-based and activity-based learning journey questions, in a 6 week online learning course involving online students. The description of online course, question generator and usability results is presented below.

  • Online Course: For the purpose of this study a 6 week “PHP and MySQL” course was developed and instructed online using MOODLE platform. Students were required to choose an image of their choice from multiple-choice images during the learning process. Their selection was recorded in their profiles for authentication. A questions generator was developed and integrated in the PBAF method to create activity-based questions and their answers based on individual’s learning activity, which is described below.

  • Question Generator: To improve usability and security of authentication system in online learning, we used non-invasive activity-based learning journey questions generated in the background on student’s interaction with the learning activities. The question generator algorithm was employed to create questions and add them to the student’s profile as shown in Fig. 1(b). Questions were created on student’s interaction with: forum, discussion, adding a post in forum, lessons, resources and quiz.

Individual student’s learning activities were logged and stored into the database. Question generator extracted and correlated logged activities fields with pre-defined question statements according to their semantics. As a result, meaningful and contextual activity-based challenge questions were created and stored into individual student’s profile. Each activity-based learning journey challenge question was embedded with distracting multiple choice options including the correct answer.

Table 1 column 1 shows activity-based learning journey questions created during the online course. The course was embedded with 5 forums, multiple discussions, 5 lessons, and more than 30 learning resources. In the questions shown in Table 1, original activities in the 6 week “PHP & MySQL” course are replaced with Forum, Lesson, Discussion and Examination for summarizing the results.

Table 1. Authentication result of activity-based challenge questions
Full size table

5 Usability Results

A total of 83 students participated in the initial registration phase. All 83 students submitted answers to at least 3 image-based questions. Of the total 83 students, 72 performed at least one learning activity and the question generator created and stored activity-based learning journey questions in their profiles.

5.1 Efficiency

The efficiency of questions was measured on student response time to questions. The activity-based learning questions were created in the background by question generator during the learning process and response time was not needed. The response time for both activity-based and image-based questions were recorded during the authentication process. An independent sample t-test was performed to compare the mean response time of activity-based and image-based questions. There was no significant difference in the response time of image-based (M = 12.41, N = 18, SD = 6.04) and activity-based questions (M = 18.25, N = 16, SD = 14.51) conditions t (32) = -1.563, p = 0.128 (p > 0.05).

Efficiency of Activity-Based Questions. Table 1 shows activity-based learning journey questions answered during the authentication process. A total of 3464 activity-based questions were generated during the 6 week course. Access to learning was performed recurrently and students were encouraged to participate in learning and examination activities. The activity-based questions were short lived. If a student performed a learning activity multiple times, any correlated questions generated previously were de-activated and new questions created with up to date information. It was aimed to record and present the most recent activity-based questions during the authentication process for better memorability and to avoid ambiguity. Unlike pre-defined text and image-based questions, activity-based learning journey questions increased the efficiency by generating questions in the background without interrupting students for recording answers. The comparison of response-time between image-based questions and “0” response for all activity-based questions show a significant difference (p < 0.01).

Efficiency of Image-Based Questions. Table 2 shows results of recall image-based questions recorded during the authentication process. The participants were asked to select their choice from 5 multiple choice image options. Data in column 1 show image question type and column 4 shows the response time. The mean response time of image-based questions was recorded as 12.42 s. There was no significant difference in the response time between image-based questions and activity-based learning journey questions (p > 0.01) during the authentication process.

5.2 Accuracy

The accuracy was measured on the number of matched and unmatched answers to challenge questions during the authentication process. Of all students, 58 participated in at least one weekly quiz and returned answers to a total of 1347 challenge questions. The accuracy results are described below. There was no significant difference in the accuracy of activity-based and image-based questions (p > 0.05).

Table 2. Authentication results of image-based questions
Full size table

Accuracy of Activity Based Questions. For better accuracy, we implemented multiple choice answers to activity-based learning journey questions during the authentication process. Results of learning activity based questions from authentication process are presented in Table 1 with the question statement under column 1, and the number of multiple choice options in column 2. The number of activity questions disabled or expired during before being presented to students are shown in column 3. The number of matched, unmatched answers and response time are shown in columns 4, 5 and 6. A total of 354 learning activity based questions were presented for authentication with 240 (68 %) accuracy. The activity questions were analyzed into two categories i.e. activity-based and activity date questions. There was a significant difference in the number matched answers of activity-based (M = 75.10, N = 10, SD = 24.47) and activity-date based questions (M = 42, N = 6, SD = 29.10) conditions t (14) = 2.38, p < 0.01. Students were presented with multiple choice answers, however 114 (32 %) of the total activity-based questions was incorrect and penalized.

Accuracy of Image-Based Questions. Table 2 shows summary of authentication of image-based questions submitted during the weekly quizzes. Results of the image-based questions show 336 (73 %) matched answers during authentication. Each question was presented with 5 multiple choice answers for student to recall their previous selection which was registered during the learning process. However, students were unable to recall matched answer to 123 (27 %) questions and penalized for memorability. Unlike free text answers, multiple choice questions address any issues related with spelling mistakes, spaces, case variation or syntactic variations. This shows that memorability is a common problem with both pre-defined text-based and image-based questions, when students are required to record answers to a large number of questions.

6 Conclusion

Online examination is a high stake process and therefore secure and usable approaches are important for remote authentication. The usability of challenge questions have been an ongoing issue reported by many studies. In this study, we implemented activity-based learning journey questions with predefined image-based questions for authentication.

The use of activity-based questions significantly increased the efficiency by eliminating the process of recording answers to challenge questions. Unlike activity-based questions, students were required to register their answers to image-based questions in an additional step during the learning process. There was no significant difference in the response time between image-based and activity-based learning journey questions. The findings show no significant difference in the accuracy of image-based and activity-based learning journey questions.

The initial usability findings are encouraging and future work will be carried out to analyze the security of activity-based learning journey questions against authentication attacks.