Abstract
This article explores the effect on assessment of ‘translating’ paper and pencil test items into their computer equivalents. Computer versions of a set of mathematics questions derived from the paper-based end of key stage 2 and 3 assessments in England were administered to age appropriate pupil samples, and the outcomes compared. Although in most cases the change to the different medium seems to make little difference, for some items the affordances of the computer profoundly affect how the question is attempted, and therefore what is being assessed when the item is used in a test. These differences are considered in terms of validity and legitimacy, that is whether the means used to answer a question in a particular medium are appropriate to the assessment intention. The conclusion is not only that translating paper and pencil items into the computer format sometimes undermines their validity as assessments, it is also that some paper and pencil items are less valid as assessments than their computer equivalents would be.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beevers, C. E., Youngson, M. A., McGuire, G. R., Wild, D. G., & Fiddes, D. J. (1999). Issues of partial credit in mathematical assessment by computer. Alt-J (Association for Learning Technology Journal), 7, 26–32.
Brown, J., Stillman, G., & Herbert, S. (2004). Can the notion of affordances be of use in the design of a technology enriched mathematics curriculum. In I. Putt, R. Faragher, & M. McLean (Eds.), Proceedings of the 27th annual conference of the mathematics education research group of Australasia, vol. 1, pp.119–126.
Gaver, W. W. (1991). Technology affordances. In Proceedings of the Special Interest Group on Computer–Human Interaction (SOGHCI) conference on human factors in computing systems. New Orleans, pp. 79–84.
Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin.
Greeno, J. G. (1998). The situativity of knowing, learning and research. American Psychologist, 53(1), 5–26.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory. Principles and application. Dordrecht: Kluwer.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
McGuire, G. R., & Youngson, M. A. (2002). Assessing ICT Assessment in Mathematics. Maths CAA Series, Maths, Stats and OR network. http://ltsn.mathstore.ac.uk/articles/maths-caa-series/mar2002/index.htm Accessed 11/11/2005.
McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and evolving a concept. In Proceedings of the Graphics Interface 2000. Canadian Human–Computer Communications Society (pp. 179–186), Toronto.
Raikes, N., & Harding, R. (2003). The horseless carriage stage: Replacing conventional measures. Assessment in Education, 10(3), 267–277.
Russell, M., Goldberg, A., & O’Connor, K. (2003). Computer-based testing and validity: a look back into the future. Assessment in Education, 10(3), 278–293.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4, 295–312.
Threlfall, J., & Pool, P. (2004). How might the assessment of mathematics through dynamic interactive computer items be different from that in conventional tests?. Paper presented at the 10th International Conference on Mathematics Education (ICME 10), Copenhagen, Denmark, July 4–11.
Watson, A. (2003). Affordances, constraints and attunements in mathematical activity. In Proceedings of the British Society for Research into Learning Mathematics, 23(2), pp. 103–108.
Acknowledgement
The work reported in this article was derived from a project funded by the Qualifications and Curriculum Authority.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Threlfall, J., Pool, P., Homer, M. et al. Implicit aspects of paper and pencil mathematics assessment that come to light through the use of the computer. Educ Stud Math 66, 335–348 (2007). https://doi.org/10.1007/s10649-006-9078-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10649-006-9078-5