A knowledge-tracing model of learning from a social tagging system

Original Paper

Abstract

We propose a user model to support personalized learning paths through online material. Our approach is a variant of student modeling using the computer tutoring concept of knowledge tracing. Knowledge tracing involves representing the knowledge required to master a domain, and, from traces of online user behavior, diagnosing user knowledge states as a profile over those elements. The user model is induced from documents tagged by an expert in a social tagging system. Tags identified with “expertise” in a domain can be used to identify a corpus of domain documents. That corpus can be fed to an automated process that distills a topic model representation characteristic of the domain. As a learner navigates and reads online material, inferences can be made about the degree to which topics in the target domain have been learned. We validate this knowledge tracing approach against data from a social tagging study. As part of this evaluation, we match the predictions of the knowledge-tracing model to individual participant responses made to individual question items used to test domain knowledge.

Keywords

Cognitive models User models Latent Dirichlet allocation LDA Topic models SparTag.us Social tagging Social web 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abel, F., Baldoni, M., Baroglio, C., Henze, N., Krause, D., Patti, V.: Context-based ranking in folksonomies. Paper presented at the proceedings of the 20th ACM conference on hypertext and hypermedia (HT ’09), pp. 209–218 (2009)Google Scholar
  2. Abel, F., Herder, E., Houben, G.-J., Henze, N., Krause, D.: Cross-system User Modeling and Personalization on the Social Web. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9131-2
  3. Aleven V., McLaren B.M., Sewall J., Koedinger K.R.: A new paradigm for intelligent tutoring systems: example-tracing tutors. J. Artif. Intell. Educ 19(2), 105–154 (2009)Google Scholar
  4. Allen, I.E., Seaman, J.: Learning on demand: online education in the United States, 2009: Sloan Consortium (2010)Google Scholar
  5. Anderson J.R.: Learning to program in LISP. Cogn. Sci. 8, 87–129 (1984)CrossRefGoogle Scholar
  6. Anderson J.R.: The Adaptive Character of Thought. Lawrence Erlbaum Associates, Hillsdale (1990)Google Scholar
  7. Anderson, J.R.: Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale (1993)Google Scholar
  8. Anderson J.R., Bothell D., Byrne M.D., Douglass S., Lebiere C., Qin Y.: An integrated theory of mind. Psychol Rev 11(4), 1036–1060 (2004)CrossRefGoogle Scholar
  9. Anderson J.R., Boyle C.F., Corbett A., Lewis M.W.: Cognitive modelling and intelligent tutoring. Artif. Intell. 42, 7–49 (1990)CrossRefGoogle Scholar
  10. Balog, K., Azzopardi, L., de Rijke, M.: Formal models for expert finding in enterprise corpora. Paper presented at the 29th annual ACM SIGIR conference on research and development in information retrieval (SIGIR ’06) (2006)Google Scholar
  11. Bernstein, M., Suh, B., Hong, L., Chen, J., Kairam, S., Chi, E.H.: Interactive topic-based browsing of social status streams. Paper presented at the proceedings of the 23rd symposium on user interface software and technology (UIST ’10) (2010)Google Scholar
  12. Blei D.M., Ng A.Y., Jordan M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)MATHGoogle Scholar
  13. Bloom B.: The 2 sigma problem: the search for methods of instruction as effective as one-to-one tutoring. Educ. Res. 13(6), 4–16 (1984)Google Scholar
  14. Brin S., Page L.: The anatomy of a large-scale hypertextual web search engine. Comput. Netw. ISDN Syst. 30(1–7), 107–117 (1998)CrossRefGoogle Scholar
  15. Brusilovsky P., Peylo C.: Adaptive and intelligent web-based educational systems. Int. J. Artif. Intell. Educ. 13, 156–169 (2003)Google Scholar
  16. Budura, A., Bourges-Waldegg, D., Riordan, J.: Deriving expertise profiles from tags. Paper presented at the international conference on computational science and engineering (CSE ’09) (2009)Google Scholar
  17. Campbell, C.S., Maglio, P.P., Cozzi, A., Dom, B.: Expertise identification using email communications. Paper presented at the proceedings of the 2003 ACM CIKM international conference on information and knowledge management (CIKM ’03) (2003)Google Scholar
  18. Carbonell J.R.: AI in CAI: an artificial-intelligence approach to computer-assisted instruction. IEEE Trans. Man Mach. Syst. 11(4), 190–202 (1970)CrossRefGoogle Scholar
  19. Champ, H.: (2009, August 13, 2010). 4,000,000,000. http://blog.flickr.net/en/2009/10/12/4000000000
  20. Conati C., Gertner A., Vanlehn K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adapt. Interact. 12(4), 371–417 (2002)MATHCrossRefGoogle Scholar
  21. Corbett, A.T.: Cognitive computer tutors: solving the two-sigma problem. Paper presented at the user modeling 2001: 8th international conference, Berlin (2001)Google Scholar
  22. Corbett A.T., Anderson J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adapt. Interact. 4(4), 253–278 (1995)CrossRefGoogle Scholar
  23. Corbett, A. T., Anderson, J. R., O’Brien, A. T.: Student modeling in the ACT Programming Tutor. In: Nichols P. D., Chipman S. F., Brennan R. L. (eds.) Cognitively diagnostic assessment, pp. 19–41. Lawrence Erlbaum Associates, Hillsdale, NJ (1995)Google Scholar
  24. Delicious.: (2008, August 13, 2010). Oh Happy Day. http://blog.delicious.com/blog/2008/07/oh-happy-day.html
  25. Foltz, P.W., Laham, D., Landauer, T.K.: Automated essay scoring: application to educational technology. Paper presented at the world conference on education, multimedia, hypermedia, and telecommunications, Seattle, WA (1999)Google Scholar
  26. Fox, S., Fallows, D.: Internet health resources. Retrieved December, 2003, from http://www.pewinternet.org/reports/pdfs/PIP_Health_Report_July_2003.pdf (2003, August)
  27. Fox, S., Jones, S.: The social life of health information. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2009/8-The-Social-Life-of-Health-Information.aspx (2009, June 11)
  28. Furnas G.W., Landauer T.K., Gomez L.M., Dumais S.T.: The vocabulary problem in human-system communication. Commun. ACM 30, 964–971 (1987)CrossRefGoogle Scholar
  29. Gayo-Avello, D., Brenes D.J.: Overcoming spammers in twitter—-a tale of five algorithms. Paper presented at the Congreso Español de Recuperación de Información (CERI 2010) (2010)Google Scholar
  30. Gena, C., Cena, F., Vernero, F., Grillo, P.: The evaluation of a social adaptive web site for cultural events. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9129-9
  31. Golder S.A., Huberman B.A.: The structure of collaborative tagging systems. J. Inf. Sci. 32, 198–208 (2006)CrossRefGoogle Scholar
  32. Griffiths T.L., Steyvers M., Tenenbaum J.B.: Topics in semantic representation. Psychol. Rev. 114(2), 211–244 (2007)CrossRefGoogle Scholar
  33. Han S.-G., Lee S.-G., Jo G.-S.: Case-based tutoring systems for procedural problem solving on the www. Expert Syst. Appl. Int. J. 29(3), 573–582 (2005)CrossRefGoogle Scholar
  34. Hartley J.R., Sleeman D.H.: Towards more intelligent teaching systems. Int. J. Man Mach. Stud. 5(2), 215–236 (1973)CrossRefGoogle Scholar
  35. Hofman, T.: Probabilistic latent semantic indexing. Paper presented at the twenty-second international SIGIR conference on research and development in information retrieval (SIGIR-99) (1999)Google Scholar
  36. Hong, L., Chi, E. H., Budiu, R., Pirolli, P., Nelson, L.: SparTag.us: a low cost tagging system for foraging of web content. Paper presented at the working conference on advanced visual interfaces (AVI ’08) (2008)Google Scholar
  37. Horrigan, J.: The Internet as a resource for news and information about science. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2006/The-Internet-as-a-Resource-for-News-and-Information-about-Science.aspx (2006, November 20)
  38. Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: Information retrieval in folksonomies: search and ranking. Paper presented at the 3rd European semantic web conference (ESWC ’06) (2006)Google Scholar
  39. John, A., Seligmann, D.: Collaborative tagging and expertise in the enterprise. Paper presented at the collaborative web tagging workshop in conjunction with 15th international conference on world wide web (WWW ’06) (2006)Google Scholar
  40. Junker, B.: Some statistical models and computational methods that may be useful for cognitively-relevant assessment. National Research Council (1999)Google Scholar
  41. Kakkonen, T., Myller, N., Timonen, J., Sutinen, E.: Automatic essay grading with probabilistic latent semantic analysis. Paper presented at the proceedings of the second workshop on building educational applications Using NLP (2005)Google Scholar
  42. Kammerer, Y., Nairn, R., Pirolli, P., Chi, E. H.: Signpost from the masses: learning effects in an exploratory social tag search browser. Paper presented at the 27th conference on human factors in computing systems (CHI ’09) (2009)Google Scholar
  43. Kim, H.-N., El Saddik, A.: Exploring social tagging for personalized community recommendations. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9130-3
  44. Kleinberg J.: Authoritative sources in a hyperlinked environment. J. ACM 46(5), 604–632 (1999)MathSciNetMATHCrossRefGoogle Scholar
  45. Kline T.J.B.: Psychological Testing: A Practical Approach to Design and Evaluation. Sage Publications, Thousand Oaks (2005)Google Scholar
  46. Koedinger K.R., Anderson J.R., Hadley W.H., Mark M.A.: Intelligent tutoring goes to the big city. Int. J. Artif. Intell. Educ. 8, 30–43 (1997)Google Scholar
  47. Landauer T.K., Dumais S.T.: A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104, 211–240 (1997)CrossRefGoogle Scholar
  48. Lenhart, A.: Adults and social network websites. Pew Internet & American Life Project. Retrieved June 27, 2009, from http://www.pewinternet.org/Reports/2009/Adults-and-Social-Network-Websites.aspx (2009, January 14)
  49. Loizou, S. K., Dimitrova, V.: Adaptive notifications to support knowledge sharing in close-knit virtual communities. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9127-y
  50. Mimno, D., McCallum, A.: Expertise modeling for matching papers with reviewers. Paper presented at the 13th international conference on knowledge discovery and data mining (KDD ’07) (2007)Google Scholar
  51. Murray T.: Authoring intelligence tutoring systems: an analysis of the state of the art. Int. J. Artif. Intell. Educ. 10, 98–129 (1999)Google Scholar
  52. Murray T.: An overview of intelligent tutoring system authoring tools: updated analysis of the state of the art. In: Murray, T., Blessing, S., Aisworth, S. (eds) Authoring Tools for Advanced Technology Environments, pp. 493–546. Kluwer, Dordrecht (2003)Google Scholar
  53. Nelson, L., Held, C., Pirolli, P., Hong, L., Schiano, D., Chi, E. H.: With a little help from my friends: examining the impact of social annotations in sensemaking tasks. Paper presented at the proceedings of the 27th international conference on human factors in computing systems (2009)Google Scholar
  54. Noll, M. G., Au Yeung, C.-M., Gibbins, N., Meinel, C., Shadbolt, N.: Telling experts from spammers: expertise ranking in folksonomies. Paper presented at the 32nd annual ACM SIGIR conference on research and development in information retrieval (SIGIR ’09) (2009)Google Scholar
  55. Noll, M. G., Yeung, C.-m. A., Gibbins, N., Meinel, C., Shadbolt, N.: Telling experts from spammers: expertise ranking in folksonomies. Paper presented at the proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval (2009)Google Scholar
  56. PACT.: Pittsburgh advanced cognitive tutor center home. Retrieved August 13, 2010, from http://pact.cs.cmu.edu (2005)
  57. Petkova, D., Croft, B.W.: Hierarchical language models for expert finding in enterprise corpora. Paper presented at the 18th IEEE conference on tools with artificial intelligence (ICTAI ’06) (2006)Google Scholar
  58. Pirolli P., Wilson M.: A theory of the measurement of knowledge content, access, and learning. Psychol. Rev. 105, 58–82 (1998)CrossRefGoogle Scholar
  59. Robu V., Halpin H., Shepherd H.: Emergence of consensus and shared vocabularies in collaborative tagging systems. ACM Trans. Web 3(4), 1–34 (2009)CrossRefGoogle Scholar
  60. Rosen-Zvi, M., Griffiths, T.L., Steyvers, M., Smyth, P.: The author-topic model for authors and documents. Paper presented at the 20th conference on uncertainty in artificial intelligence (2004)Google Scholar
  61. Shapira, B., Rokach, L., Freilichman, S.: Utilizing facebook single and cross domain data for recommendation systems. User Model. User Adapt. Interact. (2012). doi:10.1007/s11257-012-9128-x
  62. Shute V.J., Psotka J.: Intelligent tutoring systems: past, present, and future. In D.H. Jonassen (ed.) Handbook of Research on Educational Communications and Technology, pp. 570–600. Simon & Schuster Macmillan, New York (1996)Google Scholar
  63. Steyvers M., Griffiths, T.L., Dennis, S.: Probabilistic inference in human semantic memory. TRENDS Cogn. Sci. 10(7), 327–334 (2006)Google Scholar
  64. Weng, J., Lim, E.-P., Jiang, J., He, Q.: TwitterRank: finding topic-sensitive influential twitterers. Paper presented at the proceedings of the 3rd ACM international conference on web search and data mining (WSDM ’10) (2010)Google Scholar
  65. Yin, Z., Li, R., Mei, Q., Han, J.: Exploring social tagging graph for web object classification. Paper presented at the 15th ACM SIGKDD conference on knowledge discovery and data mining (KDD ’09) (2009)Google Scholar
  66. YouTube. YouTube Statistics. Retrieved January 16, 2012, from http://www.youtube.com/t/press_statistics
  67. Zhang, J., Ackerman, M.S.: Searching for expertise in social networks: a simulation of potential strategies. Paper presented at the 2005 international ACM SIGGROUP conference on supporting group work (Group ’05) (2005)Google Scholar
  68. Zhang, J., Ackerman, M. S., Adamic, L.: Expertise networks in online communities: structure and algorithms. Paper presented at the 16th international world wide web conference (WWW ’07) (2007)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  1. 1.Palo Alto Research CenterPalo AltoUSA
  2. 2.Department of Computer ScienceStanford UniversityStanfordUSA

Personalised recommendations