Abstract
This paper describes an investigation into the reliability of an automatic assessment method of the learner-build concept map by comparing it with two well-known manual methods. We have previously proposed the Kit-Build (KB) concept map framework where a learner builds a concept map by using only a provided set of components, known as the set “kit”. In this framework, instant and automatic assessment of a learner-build concept map has been realized. We call this assessment method the “Kit-Build method” (KB method). The framework and assessment method have already been practically used in classrooms in various schools. As an investigation of the reliability of this method, we have conducted an experiment to compare the assessment results of the method with the assessment results of two other manual assessment methods. In this experiment, 22 university students attended as subjects and four as raters. It was found that the scores of the KB method had a very strong correlation with the scores of the other manual methods. The results suggest that automatic assessment of the Kit-Build concept map can attain almost the same level of reliability as well-known manual assessment methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Novak, J.D., Musonda, D.: A twelve-year longitudinal study of science concept learning. Am. Educ. Res. J. 28(1), 117–153 (1991)
Ausubel, D.P., Novak, J.D., Hanesian, H.: Educational Psychology: A Cognitive View, 2nd edn. Holt, Rinehart and Winston, New York (1978)
Novak, J.D., Cañas, A.J.: Technical Report IHMC CmapTools. Institute for Human and Machine Cognition, Ocala (2008)
Hu, M.L.M., Wu, M.H.: The effect of concept mapping on students’ cognitive load. World Trans. Eng. Technol. Educ. 10(2), 134–137 (2012)
Hirashima, T., Yamasaki, K., Fukuda, H., Funaoi, H.: Kit-Build concept map for automatic diagnosis. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS, vol. 6738, pp. 466–468. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21869-9_71
Hirashima, T., Yamasaki, K., Fukuda, H., Funaoi, H.: Framework of Kit-Build concept map for automatic diagnosis and its preliminary use. Res. Pract. Technol. Enhanced Learn. 10(1), 1–21 (2015)
Sugihara, K., Osada, T., Nakata, S., Funaoi, H., Hirashima, T.: Experimental evaluation of Kit-Build concept map for science classes in an elementary school. In: Proceedings of Computers in Education 2012, pp. 17–24. National Institute of Education, Singapore (2012)
Yoshida, K., Sugihara, K., Nino, Y., Shida, M., Hirashima, T.: Practical use of Kit-Build concept map system for formative assessment of learners’ comprehension in a lecture. In: Proceedings of Computers in Education 2013, pp. 906–915. Asia-Pacific Society for Computers in Education, Bali (2013)
Nomura, T., Hayashi, Y., Suzuki, T., Hirashima, T.: Knowledge propagation in practical use of Kit-Build concept map system in classroom group work for knowledge sharing. In: Proceeding of International Conference on Computers in Education Workshop 2014, pp. 463–472. ICCE 2014 Organizing Committee, Nara (2014)
Alkhateeb, M., Hayashi, Y., Hirashima, T.: Comparison between Kit-Build and Scratch-Build concept mapping methods in supporting EFL reading comprehension. J. Inf. Syst. Educ. 14(1), 13–27 (2015)
Novak, J.D., Gowin, D.B.: Learning How to Learn. Cambridge University Press, New York (1984)
McClure, J.R., Bell, P.E.: Effects of an Environmental Education Related STS Approach Instruction on Cognitive Structures of Pre-service Science Teachers. State University, Pennsylvania (1990)
McClure, J.R., Sonak, B., Suen, H.K.: Concept map assessment of classroom learning: reliability, validity, and logistical practicality. J. Res. Sci. Teach. 36(4), 475–492 (1999)
Klaus, R.: Sugar (2013). http://www.english-online.at/biology/sugar/sugar-carbohy-drate-that-gives-us-energy.htm. Accessed 01 Aug 2016
The Institute for Human and Machine Cognition. (2014). Cmap Cloud & CmapTools in the Cloud. http://cmap.ihmc.us/cmap-cloud/. Accessed Aug 2016
Brennan, R.L.: Elements of Generalizability Theory. American College Testing Program (1983)
Webb, N.M., Shavelson, R.J.: Generalizability theory: overview. In: Everitt, B.S., Howell D.C. (eds.) Encyclopedia of Statistics in Behavioral Science, vol. 2, pp. 717–719. Wiley, Chichester (2005)
Evans, J.D.: Straightforward Statistics for the Behavioral Sciences. Brooks/Cole, Boston (1996)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wunnasri, W., Pailai, J., Hayashi, Y., Hirashima, T. (2017). Reliability Investigation of Automatic Assessment of Learner-Build Concept Map with Kit-Build Method by Comparing with Manual Methods. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-61425-0_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61424-3
Online ISBN: 978-3-319-61425-0
eBook Packages: Computer ScienceComputer Science (R0)