Skip to main content
Log in

Evaluation of topic-based adaptation and student modeling in QuizGuide

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

This paper presents an in-depth analysis of a nonconventional topic-based personalization approach for adaptive educational systems (AES) that we have explored for a number of years in the context of university programming courses. With this approach both student modeling and adaptation are based on coarse-grained knowledge units that we called topics. Our motivation for the topic-based personalization was to enhance AES transparency for both teachers and students by utilizing typical topic-based course structures as the foundation for designing all aspects of an AES from the domain model to the end-user interface. We illustrate the details of the topic-based personalization technology, with the help of the Web-based educational service QuizGuide—the first system to implement it. QuizGuide applies the topic-based personalization to guide students to the right learning material in the context of an undergraduate C programming course. While having a number of architectural and practical advantages, the suggested coarse-grained personalization approach deviates from the common practices toward knowledge modeling in AES. Therefore, we believe that several aspects of QuizGuide required a detailed evaluation—from modeling accuracy to the effectiveness of adaptation. The paper discusses how this new student modeling approach can be evaluated, and presents our attempts to evaluate it from multiple different prospects. The evaluation of QuizGuide across several consecutive semesters demonstrates that, although topics do not always support precise user modeling, they can provide a basis for successful personalization in AESs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. In the reminder of this paper, whenever possible, we apply independent-samples t-test to compare means of two populations. However, whenever our data do not satisfy the assumptions of parametric statistics (such as here), we use the Mann-Whitney test as a t-test substitute.

  2. Some researchers assert that the learning curve should be approximated with the exponential instead of power function (e.g., Heathcote et al. (2000), Murre and Chessa (2011).

  3. The reviewed drawbacks of the original topic-based student modeling formula caused us to replace it with a more cognitively grounded formula in the newer version of topic-based adaptation service Yudelson (2010). However, in this paper, we analyze the exact formula that was used in our QuizGuide studies.

  4. Figure 8 explains why .5 is the maximum level of student per-topic knowledge that results in a change in system adaptation: at this level the topic is annotated with the 3-arrows knowledge icon; any additional (positive) results will not influence the change of the topic annotation.

  5. This is a simplification, as students also take into account the goal-based annotation when choosing topics. However, in order to assess the roles of topic-based and goal-based adaptation separately, we have to simplify this analysis.

  6. There is a ongoing debate in the literature on whether Likert scales produce ordinal or interval data. We have chosen to use a more conservative approach, i.e. treat students’ answers as ordinal data. Based on this assumption and on the fact that students’ responses to all eight questions were not normally distributed around medians, we decided to use one-sample sign tests for this analysis.

References

  • Ahn, J.-W., Brusilovsky, P., He, D., Grady, J., Li, Q.: Personalized Web Exploration with Task Models. In: Proceedings of the 17th international conference on World Wide Web, WWW ’08, Beijing, China, April 21–25, 2008, ACM, pp. 1–10 (2008)

  • Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)

    Article  Google Scholar 

  • Apted, T., Kay, J., Lum, A.: Supporting metadata creation with an ontology built from an extensible dictionary. In: De Bra, P., Nejdl, W. (eds.) Proceedings of 3rd International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2004), pp. 4–13. Springer, Berlin, 23–26 August 2004

  • Aroyo, L., Dicheva, D.A.: Concept-based approach to support learning in a Web-based support environment. In: Moore, J.D., Redfield, C.L., Johnson, W.L. (eds.) Proceedings of AI-ED’2001, pp. 1–12. IOS Press, Amsterdam (2001)

  • Baker, R.S., Corbett, A.T., Aleven, V.: More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. In: Woolf, B., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) Proceedings of the 9th International Conference on Intelligent Tutoring Systems, Montreal, Canada, pp. 406–415. Springer, Berlin, 23–27 June 2008

  • Baker, R.S.J.d., Corbett, A.T., Gowda, S.M., Wagner, A.Z., MacLaren, B.A., Kauffman, L.R., Mitchell, A.P., Giguere, S.: Contextual slip and prediction of student performance after use of an intelligent tutor. In: De Bra, P., Kobsa, A., Chin, D. (eds.) Proceedings of 18th International Conference on User Modeling, Adaptation, and Personalization (UMAP 2009), pp. 52–63. Springer, Big Island, HI, 22–24 June 2010

  • Baker, R.S.J.d., Pardos, Z.A., Gowda, S.M., Nooraei, B.B., Heffernan, N.T.: Ensembling predictions of student knowledge within intelligent tutoring systems. In: Proceedings of 19th International Conference on User Modeling, Adaptation, and Personalization, UMAP 2011, Girona, Spain, pp. 13–24. Springer, 11–15 July 2011

  • Beck, J., Chang, K.-M., Mostow, J., Corbett, A.: Does help help? Introducing the Bayesian evaluation and assessment methodology. In: Woolf, B., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) Proceedings of the 9th International Conference on Intelligent Tutoring Systems, Montreal, Canada, pp. 383–394. Springer, Berlin, 23–27 June 2008

  • Brown, J.S., Burton, R.R.: Diagnostic models for procedural bugs in basic mathematical skills. Cogn. Sci. 2, 155–192 (1978)

    Article  Google Scholar 

  • Brusilovsky, P.: Adaptive hypermedia. User Model. User Adapt. Interact. 11(1/2), 87–110 (2001)

    Article  MATH  Google Scholar 

  • Brusilovsky, P.: Developing adaptive educational hypermedia systems: from design models to authoring tools. In: Murray, T., Blessing, S., Ainsworth, S. (eds.) Authoring Tools for Advanced Technology Learning Environments: Toward cost-effective adaptive, interactive, and intelligent educational software, pp. 377–409. Kluwer, Dordrecht (2003)

    Chapter  Google Scholar 

  • Brusilovsky, P.: Knowledgetree: a distributed architecture for adaptive e-learning. In: Proceedings of 13th International World Wide Web Conference, WWW 2004, pp. 104–113. ACM Press, New York, NY, 17–22 May 2004

  • Brusilovsky, P.: Adaptive navigation support. In: Brusilovsky, P., Kobsa, A., Neidl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science, vol. 4321, pp. 263–290. Springer, Berlin (2007)

  • Brusilovsky, P.: Adaptive hypermedia for education and training. In: Durlach, P., Lesgold, A. (eds.) Adaptive Technologies for Training and Education, pp. 46–68. Cambridge University Press, Cambridge (2012)

    Chapter  Google Scholar 

  • Brusilovsky, P., Cooper, D.W.: Domain, Task, and user models for an adaptive hypermedia performance support system. In: Gil, Y., Leake, D.B. (eds.) Proceedings of 2002 International Conference on Intelligent User Interfaces, pp. 23–30. ACM Press, San Francisco, CA 13–16 January 2002

  • Brusilovsky, P., Eklund, J.: A study of user-model based link annotation in educational hypermedia. J. Univ. Comput. Sci. 4(4), 429–448 (1998)

    Google Scholar 

  • Brusilovsky, P., Eklund, J., Schwarz, E.: Web-based education for all: a tool for developing adaptive courseware. In: Ashman, H., Thistewaite, P. (eds.) Proceedings of 7th International World Wide Web Conference, pp. 291–300. Elsevier Science B. V., Brisbane, Australia, 14–18 April 1998

  • Brusilovsky, P., Karagiannidis, C., Sampson, D.: Layered evaluation of adaptive learning systems. Int. J. Contin. Eng. Educ. Lifelong Learn. 14(4/5), 402–421 (2004)

    Article  Google Scholar 

  • Brusilovsky, P., Knapp, J., Gamper, J.: Supporting teachers as content authors in intelligent educational systems. Int. J. Knowl. Learn. 2(3/4), 191–215 (2006)

    Article  Google Scholar 

  • Brusilovsky, P., Millán, E.: User models for adaptive hypermedia and adaptive educational systems. In: Brusilovsky, P., Kobsa, A., Neidl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science, vol. 4321, pp. 3–53. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Brusilovsky, P., Sosnovsky, S.: Engaging students to work with self-assessment questions: a study of two approaches. In: Proceedings of 10th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE’2005, pp. 251–255. ACM Press, Monte de Caparica, Portugal, 27–29 June 2005a

  • Brusilovsky, P., Sosnovsky, S.: Individualized exercises for self-assessment of programming knowledge: an evaluation of QuizPACK. ACM Journal on Educational Resources in Computing 5(3): Article no. 6 (2005b)

  • Brusilovsky, P., Sosnovsky, S., Lee, D., Yudelson, M., Zadorozhny, V., Zhou, X.: Learning SQL programming with interactive tools: from integration to personalization. ACM Transactions on Computing Education 9 (4). Article No. 19, 1–15 (2010)

  • Brusilovsky, P., Sosnovsky, S., Shcherbinina, O.: User modeling in a distributed e-learning architecture. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) Proceedings of 10th International User Modeling Conference, pp. 387–391. Springer, Berlin, 24–29 July 2005a

  • Brusilovsky, P., Sosnovsky, S., Yudelson, M., Chavan, G.: Interactive authoring support for adaptive educational systems. In: Looi, C.-K., McCalla, G., Bredeweg, B., Breuker, J. (eds.) Proceedings of 12th International Conference on Artificial Intelligence in Education, AIED’2005, pp. 96–103. IOS Press, Amsterdam, 18–22 July 2005b

  • Bull, S.: Supporting learning with open learner models. In: Proceedings of 4th Hellenic Conference on Information and Communication Technologies in Education, pp. 47–61. Athens, Greece, September 29–October 3, 2004

  • Carbonell, J.R.: AI in CAI: an artificial intelligence approach to computer aided instruction. IEEE Trans. Man-Mach. Syst. MMS 11(4), 190–202 (1970)

    Article  Google Scholar 

  • Carmona, C., Bueno, D., Guzmán, E., Conejo, R.: SIGUE: making web courses adaptive. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) Proceedings of Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2002), May 29–31, 2002, Springer, Málaga, Spain, pp. 376–379 (2002)

  • Chin, D.: Empirical evaluations of user models and user-adapted systems. User Model. User Adapt. Interact. 11(1–2), 181–194 (2001)

    Article  MATH  Google Scholar 

  • Conati, C., Gertner, A., Vanlehn, K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adapt. Interact. 12(4), 371–417 (2002)

    Article  MATH  Google Scholar 

  • Conejo, R., Guzman, E., Millán, E.: SIETTE: a web-based tool for adaptive teaching. Int. J. Artif. Intell. Educ. 14(1), 29–61 (2004)

    Google Scholar 

  • Corbett, A., McLaughlin, M., Scarpinatto, C.: Modeling student knowledge: cognitive tutors in high school and college. User Model. User Adapt. Interact. 10(2–3), 81–108 (2000)

    Article  Google Scholar 

  • Corbett, A.T., Anderson, J.R.: Student modeling and mastery learning in a computer-based programming tutor. In: Frasson, C., Gauthier, G., McCalla, G. (eds.) Proceedings of Second International Conference on Intelligent Tutoring Systems, ITS’92, pp. 413–420. Springer, Berlin, 10–12 June 1992

  • Corbett, A.T., Anderson, J.R.: Knowledge tracing: modelling the acquisition of procedural knowledge. User Model. User Adapt. Interact. 4(4), 253–278 (1995)

    Article  Google Scholar 

  • Corbett, A.T., Anderson, J.R., Carver, V.H., Brancolini, S.A.: Individual differences and predictive validity in student modeling. In: Ram, A., Eiselt, K. (eds.) Proceedings of the 16th Annual Conference of the Cognitive Science Society. pp. 457–464. Lawrence Erlbaum, Edinburgh, 23–27 August 1993a

  • Corbett, A.T., Anderson, J.R., O’Brien, A.T.: The predictive validity of student modeling in the ACT programming tutor. In: Brna, P., Ohlsson, S., Pain, H. (eds.) Proceedings of AI-ED’93, World Conference on Artificial Intelligence in Education, pp. 457–464. AACE, Charlottesville, Edinburgh, 23–27 August 1993b

  • Cristea, A., Aroyo, L.: Adaptive authoring of adaptive educational hypermedia. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) Proceedings of Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2002), pp. 122–132. Springer, Berlin, 29–31 May 2002

  • Davidovic, A., Warren, J., Trichina, E.: Learning benefits of structural example-based adaptive tutoring systems. IEEE Trans. Educ. 46(2), 241–251 (2003)

    Article  Google Scholar 

  • De Bra, P.: Pros and cons of adaptive hypermedia in web-based education. J. CyberPsychol. Behav. 3(1), 71–77 (2000)

    Article  Google Scholar 

  • De Bra, P., Aerts, A., Berden, B., de Lange, B., Rousseau, B., Santic, T., Smits, D., Stash, N.: AHA! The adaptive hypermedia architecture. In: Proceedings of the 14th ACM Conference on Hypertext and Hypermedia, pp. 81–84. Nottingham, UK, ACM (2003)

  • De Bra, P., Ruiter, J.-P.: AHA! Adaptive hypermedia for all. In: Fowler, W., Hasebrook, J. (eds.) Proceedings of WebNet’2001, World Conference of the WWW and Internet, pp. 262–268. AACE, Orlando, FL, 23–27 October 2001

  • De Bra, P., Smits, D., van der Sluijs, K., Cristea, A., Foss, J., Glahn, C., Steiner, C.: GRAPPLE: learning management systems meet adaptive learning environments. In: Peña-Ayala, A. (ed.) Intelligent and Adaptive Educational-Learning Systems, vol. 17, pp. 133–160. Springer, Berlin (2013)

    Chapter  Google Scholar 

  • Eliot, C., Neiman, D., Lamar, M.: Medtec: A Web-based intelligent tutor for basic anatomy. In: Lobodzinski, S., Tomek, I. (eds.) Proceedings of WebNet’97, World Conference of the WWW, Internet and Intranet, pp. 161–165. AACE, Toronto, Canada, 1–5 November 1997

  • Falakmasir, M.H., Pardos, Z.A., Gordon, G.J., Brusilovsky, P.: A spectral learning approach to knowledge tracing. In: D’Mello, S.K., Calvo, R.A., Olney, A. (eds.) Proceedings of the 6th International Conference on Educational Data Mining (EDM 2010), pp. 28–34. Memphis, TN, USA, 6–9 July 2013

  • Farzan, R., Brusilovsky, P.: AnnotatEd: a social navigation and annotation service for web-based educational resources. New Rev. Hypermedia Multimed 14(1), 3–32 (2008)

    Article  Google Scholar 

  • Fogarty, J., Baker, R., Hudson, S.: Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction. Proc. Grap. Interface 2005, 129–136 (2005)

    Google Scholar 

  • Gena, C. Weibelzahl, S.: Usability engineering for the adaptive web. In: Brusilovsky, P., Kobsa, A., Neidl, W. (eds.): The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science, Vol. 4321, pp. 720–762. Springer, Berlin (2007)

  • Goldstein, I.P.: The genetic graph: a representation for the evolutionof procedural knowledge. Int. J. Man-Mach. Stud. 11(1), 51–77 (1979)

    Article  Google Scholar 

  • Goldstein, I.P., Carr, B.: The computer as coach: an athletic paradigm for intelligent education. In: Proceedings of 1977 Annual ACM Conference, Seatle, pp. 227–233. October (1977)

  • González-Brenes, J.P., Huang, Y., Brusilovsky, P.: General features in knowledge tracing to model multiple subskills, temporal item response theory, and expert knowledge. In: Stamper, J., Pardos, Z., Mavrikis, M., McLaren, B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), London, UK, pp. 84–91. 4–7 July 2014

  • Hatala, M., Gasevic, D., Siadaty, M., Jovanovic, J., Torniai, C.: Can educators develop ontologies using ontology extraction tools: an end-user study. In: Cress, U., Dimitrova, V., Specht, M. (eds.) Proceedings of 4th European Conference on Technology Enhanced Learning (ECTEL 2009), Nice, France, pp. 127–139. Springer, Berlin, 29 September–2 October 2009

  • Heathcote, A., Brown, S., Mewhort, D.J.K.: The power law repealed: the case for an exponential law of practice. Psychon. Bull. Rev. 7(2), 185–207 (2000)

    Article  Google Scholar 

  • Henze, N., Naceur, K., Nejdl, W., Wolpers, M.: Adaptive hyperbooks for constructivist teaching. Künstliche Intell. 4, 26–31 (1999)

    Google Scholar 

  • Henze, N., Nejdl, W.: Student modeling for KBS Hyperbook system using Bayesian networks, Technical report, Report, University of Hannover (1999)

  • Hothi, J., Hall, W., Sly, T.: A study comparing the use of shaded text and adaptive navigation support in adaptive hypermedia. In: Brusilovsky, P., Stock, O., Strapparava, C. (eds.) Proceedings of Adaptive Hypermedia and Adaptive Web-based systems, pp. 335–342. Springer, Berlin, 28–30 August 2000

  • Hovland, C.I., Lumsdaine, A.A., Sheffield, F.D.: A baseline for measurement of percentage change. In: Hovland, C.I., Lumsdaine, A.A., Sheffield, F.D. (eds.) Experiments on Mass Communication. Wiley, New York (1949)

    Google Scholar 

  • Hsiao, I.-H., Sosnovsky, S., Brusilovsky, P.: Guiding students to the right questions: adaptive navigation support in an E-Learning system for Java programming. J. Comput. Assist. Learn. 26(4), 270–283 (2010)

    Article  Google Scholar 

  • Jameson, A.: Numerical uncertainty management in user and student modeling: an overview of systems and issues. User Model. User Adapt. Interact. 5(3–4), 193–251 (1996)

    Article  Google Scholar 

  • Jameson, A.: Modeling both the context and the user. Pers. Technol. 5(1), 29–33 (2001)

    Google Scholar 

  • Karagiannidis, C., Sampson, D.G.: Layered evaluation of adaptive applications and servers. In: Brusilovsky, P., Stock, O., Strapparava, C. (eds.) Proceedings of Adaptive Hypermedia and Adaptive Web-based systens, pp. 343–346. Springer, Berlin, 28–30 August 2000

  • Käser, T., Koedinger, K.R., Gross, M.: Different parameters - same prediction: An analysis of learning curves. In: Stamper, J., Pardos, Z., Mavrikis, M., McLaren, B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), London, UK, pp. 52–59. 4–7 July 2014

  • Kavcic, A.: Fuzzy user modeling for adaptation in educational hypermedia. IEEE Trans. Syst. Man Cybern. 34(4), 439–449 (2004)

    Article  Google Scholar 

  • Keane, M., O’Brien, M., Smyth, B.: Are people biased in their use of search engines? Commun. ACM 51(2), 49–52 (2008)

    Article  Google Scholar 

  • Knutov, E., De Bra, P., Pechenizkiy, M.: AH 12 years later: a comprehensive survey of adaptive hypermedia methods and techniques. New Rev. Hypermed. Multimed. 15(1), 5–38 (2009)

    Article  Google Scholar 

  • Koedinger, K., Mathan, S.: Distinguishing qualitatively different kinds of learning using log files and learning curves. In: Mostow, J., Tedesco, P. (eds.) Proceedings of TS 2004 Log Analysis Workshop, pp. 39–46. Maceio, Brazi (2004)

    Google Scholar 

  • Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. Int. J. Artif. Intell. Educ. 8, 30–43 (1997)

    Google Scholar 

  • Koedinger, K.R., McLaughlin, E.A., Stamper, J.: Automated Student Model Improvement. In: Yacef, K., Zaïane, O., Hershkovitz, H., Yudelson, M., Stamper, J. (eds.) Proceedings of the 5th International Conference on Educational Data Mining, pp. 17–24. Chania, Greece (2012)

  • Lawless, S., Hederman, L., Wade, V.: Enhancing Access to Open Corpus Educational Content: Learning in the Wild. In: Proceedings of the 19th ACM Conference on Hypertext & Hypermedia, Pittsburgh, Pennsylvania, USA, pp. 167–174 (2008)

  • Martin, B., Koedinger, K.R., Mitrovic, A., Mathan, S.: On using learning curves to evaluate ITS. In: Looi, C.-K., McCalla, G., Bredeweg, B., Breuker, J. (eds.) Proceedings of 12th International Conference on Artificial Intelligence in Education, AIED’2005, pp. 419–426. IOS Press, Amsterdam, July 18–22, 2005

  • Martin, B., Mitrovic, A., Koedinger, K., Mathan, S.: Evaluating and improving adaptive educational systems with learning curves. User Model. User Adapt. Interact. 21(3), 249–283 (2011)

    Article  Google Scholar 

  • Mitrovic, A.: An intelligent SQL tutor on the web. Int. J. Artif. Intell. Educ. 13(2–4), 173–197 (2003)

    Google Scholar 

  • Mitrovic, A., Ohlsson, S.: Evaluation of a constraint-based tutor for a database language. Int. J. Artif. Intell. Educ. 10(304), 238–256 (1999)

    Google Scholar 

  • Murre, J., Chessa, A.: Power laws from individual differences in learning and forgetting: mathematical analyses. Psychon. Bull. Rev. 18(3), 592–597 (2011)

    Article  Google Scholar 

  • Newell, A., Rosenbloom, P.S.: Mechanisms of skill acquisition and the law of practice. In: Anderson, J.R. (ed.) Cognitive Skills and their Acquisition, pp. 1–51. Lawrence Erlbaum Associates Inc, Hillsdale, NJ (1981)

    Google Scholar 

  • Oberlander, J., O’Donell, M., Mellish, C., Knott, A.: Conversation in the museum: experiments in dynamic hypermedia with the intelligent labeling explorer. New Rev. Multimed. Hypermed. 4, 11–32 (1998)

    Article  Google Scholar 

  • Ohlsson, S.: Constraint-based student modeling. J. Artif. Intell. Educ. 3(4), 429–447 (1992)

    Google Scholar 

  • Olston, C., Chi, E.H.: ScentTrails: integrating browsing and searching on the Web. ACM Trans. Comput. Hum. Interact. 10(3), 177–197 (2003)

    Article  Google Scholar 

  • Papanikolaou, K.A., Grigoriadou, M., Kornilakis, H., Magoulas, G.D.: Personalising the interaction in a web-based educational hypermedia system: the case of inspire. User Model. User Adapt. Interact. 13(3), 213–267 (2003)

    Article  Google Scholar 

  • Paramythis, A., Weibelzahl, S.: A decomposition model for the layered evaluation of interactive adaptive systems. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) Proceedings of 10th International User Modeling Conference, pp. 438–442. Springer, Edinburgh, UK, 24–29 July 2005

  • Pardos, Z.A., Heffernan, N.: KT-IDEM: introducing item difficulty to the knowledge tracing model. In: Konstan, J., Conejo, R., Marzo, J., Oliver, N. (eds.) Proceedings of 19th International Conference on User Modeling, Adaptation, and Personalization, UMAP 2011, pp. 243–254. Springer, Girona, Spain, 11–15 July 2011

  • Pavlik Jr, P.I., Cen, H., Koedinger, K.R.: Performance factors analysis-a new alternative to knowledge tracing. In: Proceedings of 14th International Conference on Artificial Intelligence in Education (AIED 2009), pp. 531–538. Brighton, UK, 6–10 July 2009

  • Pirolli, P., Kairam, S.: A knowledge-tracing model of learning from a social tagging system. User Model. User Adapt. Interact. 23(2–3), 139–168 (2013)

    Article  Google Scholar 

  • Prentzas, J., Hatzilygeroudis, I., Garofalakis, J.: A Web-based intelligent tutoring systems using hybrid rules as its representation basis. In: Cerri, S.A., Gouardères, G., Paraguaçu, F. (eds.) Proceedings of 6th International Conference on Intelligent Tutoring Systems (ITS’2002), pp. 119–128. Springer, Berlin, 2–7 June 2002

  • Ritter, S.: PAT Online: A Model-tracing tutor on the World-wide Web. In: P. Brusilovsky, K. Nakabayashi and S. Ritter (eds.) Proceedings of Workshop ”Intelligent Educational Systems on the World Wide Web” at AI-ED’97, 8th World Conference on Artificial Intelligence in Education, Kobe, Japan, 18 August 1997, ISIR, pp. 11–17, also available at http://www.contrib.andrew.cmu.edu/~plb/AIED97_workshop/Ritter/Ritter.html (1997)

  • Schneider-Hufschmidt, M., Kühme, T., Malinowski, U. (eds.): Adaptive user Interfaces: Principles and Practice. Human factors in information technology. North-Holland, Amsterdam (1993)

    Google Scholar 

  • Schultz, S., Arroyo, I.: Tracing Knowledge and Engagement in Parallel in an Intelligent Tutoring System. In: Stamper, J., Pardos, Z., Mavrikis, M., McLaren, B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), pp. 312–315. London, UK, 4–7 July 2014

  • Self, J.: Student models in computer-aided instruction. Int. J. Man-Mach. Stud. 6, 261–276 (1974)

    Article  Google Scholar 

  • Self, J.: Bypassing the intractable problem of student modelling. In: Frasson, C., Gauthier, G. (eds.) Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education, pp. 107–123. Ablex Publishing, Norwood (1990)

    Google Scholar 

  • Shute, V.J.: SMART: Student modeling approach for responce tutoring. User Model. User Adapt. Interact. 5(1), 1–44 (1995)

    Article  Google Scholar 

  • Smith, A.S.G., Blandford, A.: MLTutor: an application of machine learning algorithms for an adaptive Web-based information system. Int. J. Artif. Intell. Educ. 13(2–4), 235–261 (2003)

    Google Scholar 

  • Sosnovsky, S., Brusilovsky, P.: Layered evaluation of topic-based adaptation to student knowledge. In: Proceedings of Fourth Workshop on the Evaluation of Adaptive Systems at 10th International User Modeling Conference, UM 2005, pp. 47–56. 26 July 2005

  • Sosnovsky, S., Brusilovsky, P., Hsiao, I.-H.: Adaptation ”in the Wild”: Ontology-based personalization of open-corpus learning material. In: Proceedings of 7th European Conference on Technology Enhanced Learning (EC-TEL 2012), pp. 425–431. Saarbrücken, Germany (2012)

  • Sosnovsky, S., Brusilovsky, P., Lee, D.H., Zadorozhny, V., Zhou, X.: Re-assessing the value of adaptive navigation support in e-learning. In: Nejdl, W., Kay, J., Pu, P., Herder, E. (eds.) Proceedings of 5th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2008), pp. 193–203. Springer, Hannover, Germany, 29 July–1 August, 2008

  • Sosnovsky, S., Brusilovsky, P., Yudelson, M.: Supporting adaptive hypermedia authors with automated content indexing. In: Proceedings of Second International Workshop on Authoring of Adaptive and Adaptable Educational Hypermedia at the Third International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH’2004), Eindhoven, the Netherlands (2004)

  • Specht, M., Kravcik, M., Pesin, L., Klemke, R.: Learner’s Lounge: Information Brokering for the Adaptive Learning Environment. In: Driscoll, M., Reeves, T.C. (eds.) Proceedings of World Conference on E-Learning, E-Learn 2002, pp. 2510–2512. AACE, Montreal, Canada, 15–19 October 2002

  • Specht, M., Oppermann, R.: ACE—adaptive courseware environment. New Rev. Hypermed. Multimed. 4, 141–161 (1998)

    Article  Google Scholar 

  • Specht, M., Weber, G., Heitmeyer, S., Schöch, V.: AST: Adaptive WWW-Courseware for statistics. In: Brusilovsky, P., Fink, J., Kay, J. (eds.) Proceedings of Workshop ”Adaptive Systems and User Modeling on the World Wide Web” at 6th International Conference on User Modeling, UM97, Chia Laguna, Sardinia, Italy, June 2, 1997, pp. 91–95, also available at http://www.contrib.andrew.cmu.edu/~plb/UM97_workshop/Specht.html (1997)

  • Srivastava, J., Cooley, R., Deshpande, M., Tan, P.: Web usage mining: discovery and applications of usage patterns from Web data. SIGKDD Explor. Newsl. 1(2), 12–23 (2000)

    Article  Google Scholar 

  • Stern, M.K., Woolf, B.P.: Curriculum sequencing in a Web-based tutor. In: Goettl, B.P., Halff, H.M., Redfield, C.L., Shute, V.J. (eds.) Proceedings of 4th International Conference, ITS-98, pp. 574–583. Springer, Berlin, 16–19 August 1998

  • Triantafillou, E., Pomportis, A., Demetriadis, S., Georgiadou, E.: The value of adaptivity based on cognitive style: an empirical study. Br. J. Educ. Technol. 35(1), 95–106 (2004)

    Article  Google Scholar 

  • VanLehn, K.: Student models. In: Polson, M.C., Richardson, J.J. (eds.) Foundations of Intelligent Tutoring Systems, pp. 55–78. Lawrence Erlbaum Associates, Hillsdale (1988)

    Google Scholar 

  • Vassileva, J.: DCG + GTE: dynamic courseware generation with teaching expertise. Instr. Sci. 26(3/4), 317–332 (1998)

    Article  Google Scholar 

  • Virvou, M., Moundridou, M.: Student and instructor models: Two kinds of user models and their interaction in an ITS authoring tools. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) Proceedings of 8th International Conference on User Modeling, UM 2001, pp. 158–167. Springer, Berlin, 13–17 July 2001

  • Wang, J.Z., Taylor, W.: Concept forest: a new ontology-assisted text document similarity measurement method. In: Lin, T.Y. et al. (eds.) Proceedings of the 2007 international conference on Web Intelligence, WI ’07, pp. 395–401. IEEE, Silicon Valey, CA, USA, 2–5 November 2007

  • Weber, G., Kuhl, H.-C., Weibelzahl, S.: Developing adaptive internet based courses with the authoring system NetCoach. In: Reich, S., Tzagarakis, M.M., De Bra, P.M.E. (eds.) Hypermedia: Openness, Structural Awareness, and aptivity, pp. 226–238. Springer, Berlin (2002)

    Chapter  Google Scholar 

  • Yudelson, M.: Providing service-based personalization in an adaptive hypermedia University of Pittsburgh (2010)

  • Yudelson, M., Fancsali, S., Ritter, S., Berman, S., Nixon, T., Joshi, A.: Better data beats big data. In: Stamper, J., Pardos, Z., Mavrikis, M., McLaren, B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), pp. 205–208. London, UK, 4–7 July 2014

  • Yudelson, M., Koedinger, K., Gordon, G.: Individualized Bayesian knowledge tracing models. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) Proceedings of Artificial Intelligence in Education 2013, pp. 171–180. Springer, Berlin/Heidelberg, Germany (2013)

Download references

Acknowledgments

This research is supported by the National Science Foundation under Grants No. 0447083 and 0633494 to the second author.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sergey Sosnovsky.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sosnovsky, S., Brusilovsky, P. Evaluation of topic-based adaptation and student modeling in QuizGuide. User Model User-Adap Inter 25, 371–424 (2015). https://doi.org/10.1007/s11257-015-9164-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-015-9164-4

Keywords

Navigation