Abstract
For achievement tests, the guess score is often used as a baseline for the lowest possible grade for score to grade transformations and setting the cut scores. For test item types such as multiple-response, matching and drag-and-drop, determining the guess score requires more elaborate calculations than the more straightforward calculation of the guess score for True-False and multiple-choice test item formats. For various variants of multiple-response and matching types with respect to dichotomous and polytomous scoring, methods for determining the guess score are presented and illustrated with practical applications. The implications for theory and practice are discussed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Berk, R.A.: A consumer’s guide to setting performance standards on criterion-referenced tests. Rev. Educ. Res. 56, 137–172 (1986)
Brennan, R.L.: Educational Measurement. Rowman & Littlefield Publishers, Lanham (2006)
Cizek, G.J.: Setting Performance Standards: Concepts, Methods, and Perspectives. Routledge, Abingdon (2001)
Cohen-Schotanus, J., Van der Vleuten, C.P.M.: A standard setting method with the best performing students as point of reference: practical and affordable. Med. Teach. 32, 154–160 (2010)
Burton, R.F.: Quantifying the effects of chance in multiple choice and true/false tests: question selection and guessing of answers. Assess. Eval. High. Educ. 26, 41–50 (2001)
Burton, R.F.: On guessing corrections. Med. Educ. 38, 113 (2004)
Ebel, R.L., Frisbie, D.A.: Essentials of Educational Measurement. Prentice Hall, Upper Saddle River (1991)
Gronlund, N.E.: Assessment of Student Achievement. Allyn & Bacon, Boston (1998)
Van Berkel, H.J.M., Bax, A.: Toetsen in het Hoger Onderwijs [Testing in Higher Education]. Bohn Stafleu Van Loghum, Houten/Diegem (2006)
Frary, R.B.: Formula scoring of multiple-choice tests (correction for guessing). In: ITEMS: The Instructional Topics in Educational Measurement Series (1988)
Roberts, D.: Let’s talk about the “correction for guessing” formula. Penn State University (2001)
Draaijer, S., Hartog, R., Hofstee, J.: Guidelines for the design of digital closed questions for assessment and learning in higher education. E-J. Instr. Sci. Technol. 10 (2007)
Hartog, R., Draaijer, S., Rietveld, L.C.: Practical aspects of task allocation in design and development of digital closed questions in higher education. Pract. Assess. Res. Eval. 13, 2–15 (2008)
Jordan, S.: E-assessment: past, present and future. New Dir. 9, 87–106 (2013)
Draaijer, S., Hartog, R.: Design patterns for digital item types in higher education. E-J. Instr. Sci. Technol. 10 (2007)
Mackenzie, D., O’Hare, D.: Empirical prediction of the measurement scale and base level “guess factor” for advanced computer-based assessments. In: Danson, M. (ed.) Sixth International Computer Assisted Assessment (CAA) Conference Proceedings. Loughborough University (2002)
Jordan, H.: Random guess score (2009). http://www.open.ac.uk/blogs/SallyJordan/wp-content/uploads/2011/05/RGS.pdf
Brown, G.A., Bull, J., Pendlebury, M.: Assessing Student Learning in Higher Education. Routledge, Abingdon (1997)
Attali, Y., Bar-Hillel, M.: Guess where: the position of correct answers in multiple-choice test items as a psychometric variable. J. Educ. Meas. 40, 109–128 (2003)
Kupsch, B., Horn, E.: Writing matching questions. UW–Madison School of Nursing
Vos, H., De Graaf, A.: De quotiëntregel [The quotient rule]. Examas. 1, 18–21 (2008)
Vos, H., Kloppenburg, M., Tomson, O.: Optimale uniforme scoringsregels voor innovatieve vraagvormen [Optimal uniform scoring rules for innovative test item formats]. Examas. 3, 21–24 (2010)
Eggen, T.J., Lampe, T.T.: Comparison of the reliability of scoring methods of multiple-response items, matching items, and sequencing items. CADMO (2012)
Acknowledgment
We would like to thank Dick Neeleman of the Vrije Universiteit Amsterdam for his contribution to this article.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Draaijer, S., Jordan, S., Ogden, H. (2018). Calculating the Random Guess Score of Multiple-Response and Matching Test Items. In: Ras, E., Guerrero Roldán, A. (eds) Technology Enhanced Assessment. TEA 2017. Communications in Computer and Information Science, vol 829. Springer, Cham. https://doi.org/10.1007/978-3-319-97807-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-97807-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97806-2
Online ISBN: 978-3-319-97807-9
eBook Packages: Computer ScienceComputer Science (R0)