Automating the Measurement of Critical Thinking for Individuals Participating in Discussion Forums

  • Stephen CorichEmail author
  • Kinshuk
  • Lynn Jeffrey


The use of discussion forums in an educational setting is increasing. However the extent to which discussion forums contribute to learning is subject to debate. A review of the literature reveals some evidence of researchers attempting to develop models that measure indicators of critical thinking for group participation within a discussion forum; however, there is little evidence of research aimed at measuring the critical thinking of individuals. This paper builds on previous studies presented by the authors which have reported on the use of a computerized tool designed to measure evidence of critical thinking among participants in a discussion forum. The paper presents an attempt to validate the computerized tool as it is used to measure evidence of critical thinking for individual participants. The validation process involved comparing the results obtained using the tool against the results obtained from administering a recognized critical thinking skills assessment.


Critical thinking Discussion forums Content analysis 


  1. Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tami, R., et al. (2008, December). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102–1134.CrossRefGoogle Scholar
  2. Bailin, S., & Siegel, H. (2003). Critical thinking. In N. Blake, P. Smeyers, R. Smith, & P. Standish (Eds.), The Blackwell guide to the philosophy of education (pp. 181–193). Oxford: Blackwell.Google Scholar
  3. Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  4. Corich, S. P., Kinshuk, & Hunt, L. M. (2004). Assessing discussion forum participation: In search of quality. International Journal of Instructional Technology and Distance Learning, 1(12), 1–12.Google Scholar
  5. Corich, S. P., Kinshuk, & Hunt, L. M. (2006). Measuring critical thinking within discussion forums using a computerised content analysis tool. Proceedings of the fifth international conference on networked learning 2006, Lancaster University, UK, April 10–12, Lancaster, UK. Retrieved on August 1, 2007, from
  6. Corich, S. P., Kinshuk, & Jefferey, L. (2007). Changing focus from group to individual: using an automated tool to measure evidence of critical thinking in discussion forums. CELDA 2007. Proceedings of the IADIS international conference on cognition and exploratory learning in digital age, 7–9 December, Algarve, Portugal.Google Scholar
  7. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston, MA: D.C. Heath.Google Scholar
  8. Doherty, J. J., Hansen, M. A., & Kaya, K. K. (1999). Teaching information skills in the information age: The Need for Critical Thinking. Library Philosophy and Practice, 1, 2. Retrived on April 8, 2010, from
  9. Ennis, R. H. (2004). Applying soundness standards to qualified reasoning. Informal Logic, 24(1), 23–39.Google Scholar
  10. Facione, P. A. (1990). California critical thinking skills test manual. Millbrae, CA: California Academic Press.Google Scholar
  11. Facione, P. A. (2010). Critical thinking: What it is and why it counts. Millbrae, CA: California Academic Press.Google Scholar
  12. Facione, P. A., Facione, N. C., & Giancarlo, C. A. (1997). The motivation to think inworking and learning. In E. A. Jones (Ed.), Preparing competent college graduates: Setting new and higher expectations for student learning—New directions for higher education (Vol. 96, pp. 67–79). San Francisco: Jossey-Bass.Google Scholar
  13. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical thinking in a text-based environment. Computer Conferencing in higher education. Internet in Higher Education, 2(2), 87–105.CrossRefGoogle Scholar
  14. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23.CrossRefGoogle Scholar
  15. Goodell, J., & Yusko, B (2005). Overcoming barriers to student participation in online discussions. Contemporary Issues in Technology and Teacher Education, 5(1), 77–92.Google Scholar
  16. Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of a global on-line debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 395–429.CrossRefGoogle Scholar
  17. Hara, N., Bonk, C. J., & Angeli, C. (1998). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28(2), 115–152.CrossRefGoogle Scholar
  18. Halpern, D. F. (1996). Thought and knowledge: An introduction to critical thinking (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  19. Henri, F. (1991). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden papers (pp. 116–136). Berlin: Springer.Google Scholar
  20. Kanuka, H. (2002). A principled approach to facilitating distance education: The Internet, higher education and higher levels of learning. Journal of Distance Education, 17(2), 70–86.Google Scholar
  21. Kanuka, H., & Anderson, T. (1998). On-line interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57–74.Google Scholar
  22. Klein, J. D., Spector, J. M., Grabowski, B., & de la Teja, I. (2004). Instructor competencies: Standards for face-to-face, online and blended settings. Greenwich, CT: Information Age Publishing.Google Scholar
  23. Mayer, R. E. (1997). Incorporating problem solving into secondary school curricula. In G. E. Phye (Ed.), Handbook of academic learning: Construction of knowledge—The educational psychology series (pp. 473–492). San Diego, CA: Academic.Google Scholar
  24. McKlin, T., Harmon, S. W., Evans, W., & Jone, M. G. (2002). Cognitive presence in web-based learning: A content analysis of students’ online discussions. American Journal of Distance Education, 15(1), 7–23.Google Scholar
  25. Meyer, K. (2003). Face-to-face versus threaded discussions: The role of time and higher-order thinking. Journal of Asynchronous Learning Networks, 7(3), 2003. Retrieved August 1, 2007, from
  26. Meyer, K. A. (2004, April). Evaluating online discussions: Four different frames of analysis. Journal of American Learning Networks, 8(2), 101–114.Google Scholar
  27. Myers, M. T. (2001). Preparing students for an uncertain future. Liberal Education, 87(3), 22–25.Google Scholar
  28. Newman, G., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face computer supported group learning. Interpersonal Computing and Technology, 3(2), 56–77.Google Scholar
  29. New Zealand Ministry of Education. (2007). The New Zealand curriculum. Wellington, New Zealand: Learning Media Limited.Google Scholar
  30. Paul, R. Elder, L., & Bartell, E. (1997) California teacher preparation for instruction in critical thinking: Research findings and policy recommendations. Sacramento, CA: California Commission on Teacher Credentialing. Retrieved October 12, 2010, from Scholar
  31. Perkins, C., & Murphy, E. (2006). Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, 9(1), 298–307.Google Scholar
  32. Raths, L. E., Jonas, A., Rothstein, A., & Wassermann, S. (1967). Teaching for thinking, theory and application. Columbus, OH: Charles E. Merrill.Google Scholar
  33. Romiszowski, A. J., & Mason, R. (2004). Computer-mediated communication. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (a Project of the AECT). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  34. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12(1), 8–22.Google Scholar
  35. U.S. Department of Education. (1990). National goals for education. Washington, DC: Government Printing Office.Google Scholar
  36. White, M. D., & Marsh, E. E. (2006) Content analysis: A flexible methodology. Library Trends, 55(1), 22–45.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Eastern Institute of Technology Hawke’s BayNapierNew Zealand
  2. 2.University of AthabascaAthabascaCanada
  3. 3.Massey UniversityWellingtonNew Zealand

Personalised recommendations