Education and Information Technologies

, Volume 24, Issue 1, pp 929–951 | Cite as

A valid and reliable tool for examining computational thinking skills

  • Mustafa YağcıEmail author


The aim of this study was to develop a scale which can be used to measure the computational thinking skills (CTS) of high school students. Validity and reliability testing of the scale was performed with the participation of 785 students. Exploratory and confirmatory factor analysis showed that the five-point Likert scale had a construct consisting of four factors Problem-solving, Cooperative Learning & Critical Thinking, Creative Thinking, and Algorithmic Thinking expressed by 42 items. The factor loadings of the scale varied from .475 to .853. The confirmatory factor analysis performed to reveal the factorial validity of the scale showed that the Chi-square value (χ2 = 2679.07; sd = 815, p = 0.00) was significant. The fitness index values were found to be RMSA = .0075; SRMR = .081; NNFI = .91; CFI = .92; GFI = .90; and AGFI = .88. The Cronbach’s Alpha internal consistency coefficient was .969 for the overall scale. In addition, the stability of the scale was examined to obtain information about its reliability and the test-re-test method was used. It was concluded as a result of the analysis that the scale was a valid and reliable measurement tool which can be used to measure the CTS of high school students.


Computational thinking Problem solving Algorithmic thinking Teaching & learning strategies 



  1. Amabile, T. M. (1985). Motivation and creativity: Effects of motivational orientation on creative writers. Journal of Personality and Social Psychology, 48(2), 393–399.CrossRefGoogle Scholar
  2. Anderson, C. W., & Smith, E. L. (1984). Children's preconceptions and content-area textbooks in G.G. Duffy. Comprehension Instruction Perspectives and Suggestions. New York: Longman.Google Scholar
  3. Ater-Kranov, A., Bryant, R., Orr, G., Wallace, S., & Zhang, M. (2010). Developing a community definition and teaching modules for computational thinking: Accomplishments and challenges. Proceedings of the 2010 ACM Conference on Information Technology Education, 143–148.Google Scholar
  4. Atmatzidou, S., & Demetriadis, S. (2016). Advancing students’ computational thinking skills through educational robotics : A study on age and gender relevant differences. Robotics and Autonomous Systems, 75, 661–670. Scholar
  5. Balcı, A. (2009). Research in Social Sciences: Methods, Techniques and Principles. Ankara: Pegem A.Google Scholar
  6. Barr, D., Harrison, J., & Conery, L. (2011). Computational thinking: A digital age skill for everyone. Retrieved from:
  7. Bati, H. (2017). Computational thinking and integrative education (STEAM) in science education. Pamukkale University Journal of Education, 41, 91–103. Scholar
  8. Berkeley, (2017). The beauty and joy of computing. Retrieved from:
  9. Boden, M. A. (1998) What is creativity, creativity in human evolution and prehistory (1 ed.). Steven J. Mithen, Routledge.Google Scholar
  10. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. 2012 annual meeting of the american educational research association (Vancouver: Canada). Retrieved from:
  11. Bundy, A. (2007). Computational thinking is pervasive. Retrieved from:
  12. Büyüköztürk, Ş. (2002). Data analysis handbook. Ankara: Pegem Publication.Google Scholar
  13. Büyüköztürk, Ş. (2008). Manual of data analysis for social sciences. Ankara: Pegem A Publications.Google Scholar
  14. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2015). Scientific research methods. Ankara: Pegem A Publications.Google Scholar
  15. Carmines, E. G., & Zeller, R. A. (1982). Reliability and validity assessment (5th ed.). Beverly Hills: Sage Publications Inc..Google Scholar
  16. Chaffee, J. (1994). Thinking critically. Boston: Houghton Mifflin.Google Scholar
  17. Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175. Scholar
  18. (2017). Hour of code. Retrieved from:
  19. Cooper, S., Dann, W., & Pausch, R. (2000). Developing algorithmic thinking with Alice. In Proceedings of Isecon, 17, 506–539.Google Scholar
  20. Craft, A. (2003). Creative thinking in the early years of education. Early Years, 23(2), 143–154.CrossRefGoogle Scholar
  21. Cropley, A. (2001). Creativity in education and Learning Great Britain: Clays.Google Scholar
  22. Curzon, P. (2015). Computational thinking: Searching to speak. Retrieved from:
  23. Czerkawski, B. C., & Lyman III, E. W. (2015). Exploring issues about computational thinking in higher education. Tech Trends, 59(2), 57–65. Scholar
  24. Demir, Ö., & Seferoğlu, S. S. (2017). New concepts, different uses: An evaluation related to computational thinking. Readings of Educational Technology, 41(August), 468–483.Google Scholar
  25. Department for Education England (2013). National curriculum in England:Computing programmes of study key stages 1 and 2. Ref: DFE-00171e2013. Retrieved from:
  26. Ennis, R. H. (1962). A concept of critical thinking. Havard Educational Review, 22, 88–111.Google Scholar
  27. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.CrossRefGoogle Scholar
  28. Facione, P. A., & Facione, N. C. (1993). Test Manual: the California Critical Thinking Skills Test, Form A and Form B. Millbrae: The California Academic Press.Google Scholar
  29. Facione, N. C., Facione, P. A., & Sanchez, C. A. (1994). Critical thinking disposition as a measure of competent clinical judgement: The development of the California thinking disposition inventory. Journal of Nursing Education, 33, 345–350.Google Scholar
  30. Fisher, R. (2004). Creativity across the curriculum. In R. Fisher & M. Williams (Eds.), In unlocking reativity (pp. 160–172). Britain: David Fulton.Google Scholar
  31. Gelbal, S. (1991). Problem solving. Hacettepe University Journal of Education, 6, 167–173.Google Scholar
  32. Gorsuch, R. L. (1983). Factor analysis (2. ed.). Hillsdale: Lawrence Erlbaum Associates.Google Scholar
  33. Guilford, J. P. (1967). Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1), 3–14.CrossRefGoogle Scholar
  34. Halpern, D. F. (2013). Thought and knowledge: An introduction to critical thinking. New York: Psychology Press.CrossRefGoogle Scholar
  35. Harvard (2017). CS50: Introduction to computer science. Retrieved from:
  36. Heppner, P. P., & Krauskopf, C. J. (1987). The integration of personal problem solving processes within counseling counsel. Psychologist, 15, 371–447.Google Scholar
  37. Heppner, P. P., & Reeder, L. (1983). The relationship between problem solving self-appraisal and psychological adjustment. Cognitive Therapy and Research, 9(4), 415–427.CrossRefGoogle Scholar
  38. Heppner, P. P., Baumgardner, A. H., & Jakson, J. (1985). Depression and attributional style: Are they related? Cognitive Therapy and Research, 9, 105–113.CrossRefGoogle Scholar
  39. Hu, C. (2011). Computational thinking: What it might meanand what we might do about it. In M. Goldweber (Ed.), 16th annual joint conference on Inno-vation and Technology in Computer Science Education (pp. 223–227). New York: ACM.Google Scholar
  40. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.CrossRefGoogle Scholar
  41. ISTE (2015). Computational thinking leadership toolkit (First Edition). Retrieved from:
  42. ISTE (2016). ISTE standards for students. Retrieved from:
  43. ISTE-CSTA (2011). Computational thinking: Teacher resources (2. edition) (Grant No: CNS-1030054). Retrieved from:
  44. Jane, L. (2001). The relationship between social problem solving and bullying behaviour among male and female adult prisoners. Aggresive Behavior, 27, 297–312.CrossRefGoogle Scholar
  45. Johnson, D.W., & Johnson, R.T. (1994). Leading the cooperative school. (Second edition), Edina, Minnesota: Interaction Book Company.Google Scholar
  46. Johnson, D. W., & Johnson, R. T. (1995). Creative controversy: Intellectual challenge in the classroom. Edina: Interaction Book Company.Google Scholar
  47. Johnson, D. W., Johnson, R. T., & Holubec, E. J. (1994). The nuts & bolts of cooperative learning. In Edina. Minnesota: Interaction Book Company.Google Scholar
  48. Jöreskog, K., & Sörbom, D. (1993). Lisrel 8: Structural equation modeling with the Simplis command language. USA: Scientific Software International.Google Scholar
  49. Kalaycı, Ş. (2009). SPSS applied multivariate statistical techniques. Ankara: Asil Publication.Google Scholar
  50. Katai, Z. (2015). The challenge of promoting algorithmic thinking of both sciences- and humanities-oriented learners. Journal of Computer Assisted Learning, 31(4), 287–299. Scholar
  51. Kazimoglu, C., Kiernan, M., Bacon, L., & MacKinnon, L. (2012). Learning programming at the computational thinking level via digital game-play. Procedia Computer Science, 9, 522–531. CrossRefGoogle Scholar
  52. Kline, R. B. (2005). Structural equation modeling (2nd ed.). New York: The Guilford Press.zbMATHGoogle Scholar
  53. Knuth, D.E. (1980). Algorithms in modern mathematics and computer Science Stanford Department of Computer Science Report No. STAN-CS-80-786, 1980.Google Scholar
  54. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Education, 72, 558–569. Scholar
  55. Lau, J. Y. (2011). An introduction to critical thinking and creativity: Think more, think better. Wiley.Google Scholar
  56. Lee, T. Y., Mauriello, M. L., Ahn, J., & Bederson, B. B. (2014). CTArcade: Computational thinking with games in school age children. International Journal of Child-Computer Interaction, 2(1), 26–33. Scholar
  57. Lu, J. J., & Fletcher, G. H. L. (2009). Thinking about computational thinking. ACM Special Interest Group on Computer Science Education Conference (SIGCSE 2009). Chattanooga: ACM Press.Google Scholar
  58. Mayer, R. E. (1992). Thinking, problem solving, Cognition. New York: W H Freeman and Company.Google Scholar
  59. Mayer, R. E. (1998). Cognitive , metacognitive , and motivational aspects of problem solving. Instructional Science, 26(1–2), 49–63. Scholar
  60. Mayer, R. E. (1999). Fifity years of creativiy research. Ed by Robert J. Handbook of creativity, Sternberg, Cambridge University Press.Google Scholar
  61. National Research Council. (1999). Being fluent with infor-mation technology. Washington, DC: National AcademyPress.Google Scholar
  62. Newell, A., & Simon, H. A. (1972). Human problem solving (Vol. 104, No. 9). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  63. Papert, S. (1996). An exploration in the space of mathematics educations. International Journal of Computers for Mathematical Learning, 1(1), 95–123.Google Scholar
  64. Paul, R. W. (1984). Critical thinking fundamental to education for a free society. Educational Leadership, 1, 5–14.Google Scholar
  65. Paul, R. (1990). Critical thinking. In Rohnert Park. California: Sonoma State University.Google Scholar
  66. Piirto, J. (2004). Understanding creativity. Great potential press, Inc. In USA.Google Scholar
  67. Rouquette, M. L. (2007). Creativity . Ankara: Dost.Google Scholar
  68. Şahiner, A., & Kert, S. B. (2016). Examining studies related with the concept of computational thinking between the years of 2006-2015. European Journal of Science and Technology, 5(9), 38–43.Google Scholar
  69. Seçer, İ. (2013). Practical data analysis and reporting with SPSS and LISREL. Ankara: Anı Publication.Google Scholar
  70. Seferoğlu, S., & Akbıyık, C. (2006). Critical thinking and teaching. Hacettepe Universitesi Journal of Education, 30, 193–200. Scholar
  71. Simpson, E., & Courtney, M. (2002). Critical thinking in nursing education: Literature review. International Journal of Nursing Practice, 8, 89–98.CrossRefGoogle Scholar
  72. Şimşek, Ö. F. (2007). Introduction to Structural Equation Modeling; Basic Principles and LISREL Applications. Ankara: Ekinoks Press.Google Scholar
  73. Slavin, R. E. (1980). Cooperative learning: Theory research and Practice. Rentice Hall: Englewood Cliffs.Google Scholar
  74. Slavin, R. E. (1987). Cooperative learning in student teams: What research says to the teacher. Washington, D.C.: National Education Association.Google Scholar
  75. Sönmez, V., & Alacapınar, G. F. (2014). Sampled scientific research methods. Ankara: Anı Publication.Google Scholar
  76. Sternberg, R. J., & Lubart, T. I. (1999). The concept of creativity: Prospects and paradigms. In R. J. Sternberg (Ed.), Handbook of creativity (pp. 3–15). Cambridge: Cambridge University Press.Google Scholar
  77. Tavşancıl, E. (2002). Attitude Measurement and Data Analysis with SPSS. Ankara: Nobel Publications.Google Scholar
  78. Thornton, S. (1998). Kids solve the problem. Istanbul: Gendaş Publishing.Google Scholar
  79. Turgut, M. F., & Baykul, Y. (2011). Measurement and evaluation in education. Ankara: PegemA Publication.Google Scholar
  80. Vee, A. (2013). Understanding computer programming as a literacy. Literacy in Composition Studies, 1(2), 42e64.CrossRefGoogle Scholar
  81. Veenman, S., Benthum, N., Bootsma, D., Dieren, J., & Kemp, N. (2002). Cooperative learning and teacher education. Teaching and Teacher Education, 18, 87–103.CrossRefGoogle Scholar
  82. Veneziano, L., & Hooper, J. (1997). A method for quantifying content validity of health-related questionnaires. American Journal of Health Behavior, 21(1), 67–70.Google Scholar
  83. Watson, G., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. San Antonio: The Psychological Corporation.Google Scholar
  84. Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.CrossRefGoogle Scholar
  85. Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society, 366, 3717–3725.MathSciNetCrossRefzbMATHGoogle Scholar
  86. Wing, J. M. (2011). Research notebook: Computational thinking What and why? The Link Magazine, Spring. Carnegie Mellon University, Pittsburgh. Retrieved from:
  87. Wing, J. M. (2016). Computational thinking. 10 years later. Microsoft Research blog. Retrieved from:
  88. Wolfram, S. (2016). How to teach computational thinking. Blog Stephen Wolfram. Retrieved from:

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Engineering and ArchitectureAhi Evran UniversityKirsehirTurkey

Personalised recommendations