Abstract
The participants of the Bebras competitions solve tasks that involve informatics concepts and require computational thinking (CT) skills for general problem solving. The recent popularity of Bebras is closely related to the increasing interest in development of the CT skills among the school students of all ages. Despite the increase in interest, we still lack a consensual, unambiguous definition and/or categorization of CT skills. In this paper, we provide an empirical evidence for the ambiguity of the current categorization of CT skills by assessing the agreement among experts when annotating five tasks of the Slovene Bebras competition in 2019. The empirical data include the annotations of the selected tasks by six experts, where each of them was required to annotate a given task with one to three categories of CT skills required to solve it. The categorization of the CT skills used in the experiment include the five categories of the well-known categorization Dagiené, Sentance and Stupuriene: algorithmic thinking, decomposition, generalization, evaluation and abstract thinking. To narrow down the broad scope of the first category of algorithmic thinking, we introduced a new, sixth category of modelling and simulation. Despite this specialization of the categorization scheme, the measurement of the Fleiss’ Kappa statistics on the empirical data, shows a weak agreement among the experts, especially for the general category of abstract thinking. This result confirms the lack of consensus among experts about the delineation between different categories of CT skills. A possible explanation of this results is that reaching a consensus about the definition and categorization of the CT skills within the heterogeneous group of experts involving teachers, programmers and computer scientists, is a challenging task. The result also calls for further effort in reaching such a consensus that would lead to deeper understanding and better teaching of the CT skills.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bebras – International Challenge on Informatics and Computational Thinking. https://www.bebras.org/, Accessed 22 May 2020
Banerjee, M., Capozzoli, M., McSweeney, L., Sinha, D.: Beyond kappa: a review of interrater agreement measures. Can. J. Stat. 27(1), 3–23 (1999)
Bollin, A., Demarle-Meusel, H., Kesselbacher, M., Mößlacher, C., Rohrer, M., Sylle, J.: The bebras contest in Austria–do personality, self-concept and general interests play an influential role? International Conference on Informatics in Schools: Situation. Evolution, and Perspectives, pp. 283–294. Springer, Cham (2018)
Barendsen, E., et al.: Concepts in K-9 computer science education. In: Proceedings of the 2015 ITiCSE on Working Group Reports, pp. 85–116 (2015)
Budinská, L., Mayerová, K., Veselovská, M.: Bebras task analysis in category little beavers in Slovakia. In: Dagiene, V., Hellas, A. (eds.) ISSEP 2017. LNCS, vol. 10696, pp. 91–101. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71483-7_8
Budinská, L., Mayerová, K., Šimandl, V.: Differences between 9–10 years old pupils’ results from Slovak and Czech bebras contest. In: Pozdniakov, S.N., Dagienė, V. (eds.) ISSEP 2018. LNCS, vol. 11169, pp. 307–318. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02750-6_24
Dagienė, V., Futschek, G.: Bebras international contest on informatics and computer literacy: criteria for good tasks. In: Mittermeir, R.T., Sysło, M.M. (eds.) ISSEP 2008. LNCS, vol. 5090, pp. 19–30. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69924-8_2
Dagienė, V., Sentance, S., Stupurienė, G.: Developing a two-dimensional categorization system for educational tasks in informatics. Informatica 28(1), 23–44 (2017)
Denning, P.J., Tedre, M.: Computational Thinking. The MIT Press, Cambridge (2019)
Grover, S., Pea, R.: Computational thinking in K–12: a review of the state of the field. Educ. Res. 42(1), 38–43 (2013)
Izu, C., Mirolo, C., Settle, A., Mannila, L., Stupuriene, G.: Exploring bebras tasks content and performance: a multinational study. Inf. Educ. 16(1), 39–59 (2017)
Kabatova, M., Kalaš, I., Tomcsanyiova, M.: Programming in Slovak primary schools. Olympiads Inf. 10, 125–159 (2016)
Kalas, I., Tomcsanyiova, M.: Students’ attitude to programming in modern informatics. In: Proceedings of the 9th WCCE 2009, Education and Technology for a Better World (2009)
Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33, 363–374 (1977)
NSW Government: coding and computational thinking: what is the evidence? https://education.nsw.gov.au/our-priorities/innovate-for-the-future/education-for-a-changing-world/media/documents/future-frontiers-education-for-an-ai-world/Coding-and-Computational-Report_A.pdf. Accessed 5 Nov 2019
Opmanis, M., Dagiene, V., Truu, A.: Task types at “Beaver” contests. In: Information Technologies at School, pp. 509–519 (2006)
Papert, S.: Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, New York (1980)
Tekmovanja Bober, ACM. Naloge in rešitve (2015/16; 16/17; 17/18; 18/19). https://tekmovanja.acm.si/?q=bober/naloge-re%C5%A1itve. Accessed 22 May 2020
Ternik, Ž.: Analiza rezultatov tekmovanja Bober skozi prizmo razumevanja konceptov računalništva ter računalniškega mišljenja = Analysis of the results of the Beaver contest based on the understanding of the computer science concepts and on computational thinking: MSc thesis, Ljubljana (2019)
Vaníček, J.: What makes situational informatics tasks difficult? In: Brodnik, A., Tort, F. (eds.) ISSEP 2016. LNCS, vol. 9973, pp. 90–101. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46747-4_8
Wing, J.: Computational thinking. Commun. ACM 49(3), 33–36 (2006)
Acknowledgements
We would like to acknowledge the financial support of the Slovenian Research Agency, via the grants P5-0093, V5-1930 and N2-0056, as well as the University of Rijeka, via the grant uniri-drustv-18-20.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ternik, Ž., Todorovski, L., Nančovska Šerbec, I. (2020). Assessing the Agreement in the Bebras Tasks Categorisation. In: Kori, K., Laanpere, M. (eds) Informatics in Schools. Engaging Learners in Computational Thinking. ISSEP 2020. Lecture Notes in Computer Science(), vol 12518. Springer, Cham. https://doi.org/10.1007/978-3-030-63212-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-63212-0_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63211-3
Online ISBN: 978-3-030-63212-0
eBook Packages: Computer ScienceComputer Science (R0)