Implicit aspects of paper and pencil mathematics assessment that come to light through the use of the computer

  • John Threlfall
  • Peter Pool
  • Matthew Homer
  • Bronwen Swinnerton


This article explores the effect on assessment of ‘translating’ paper and pencil test items into their computer equivalents. Computer versions of a set of mathematics questions derived from the paper-based end of key stage 2 and 3 assessments in England were administered to age appropriate pupil samples, and the outcomes compared. Although in most cases the change to the different medium seems to make little difference, for some items the affordances of the computer profoundly affect how the question is attempted, and therefore what is being assessed when the item is used in a test. These differences are considered in terms of validity and legitimacy, that is whether the means used to answer a question in a particular medium are appropriate to the assessment intention. The conclusion is not only that translating paper and pencil items into the computer format sometimes undermines their validity as assessments, it is also that some paper and pencil items are less valid as assessments than their computer equivalents would be.

Key words

Assessment Computer-based Key stage tests Mathematics assessment Paper and pencil Validity 



The work reported in this article was derived from a project funded by the Qualifications and Curriculum Authority.


  1. Beevers, C. E., Youngson, M. A., McGuire, G. R., Wild, D. G., & Fiddes, D. J. (1999). Issues of partial credit in mathematical assessment by computer. Alt-J (Association for Learning Technology Journal), 7, 26–32.Google Scholar
  2. Brown, J., Stillman, G., & Herbert, S. (2004). Can the notion of affordances be of use in the design of a technology enriched mathematics curriculum. In I. Putt, R. Faragher, & M. McLean (Eds.), Proceedings of the 27th annual conference of the mathematics education research group of Australasia, vol. 1, pp.119–126.Google Scholar
  3. Gaver, W. W. (1991). Technology affordances. In Proceedings of the Special Interest Group on Computer–Human Interaction (SOGHCI) conference on human factors in computing systems. New Orleans, pp. 79–84.Google Scholar
  4. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin.Google Scholar
  5. Greeno, J. G. (1998). The situativity of knowing, learning and research. American Psychologist, 53(1), 5–26.CrossRefGoogle Scholar
  6. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory. Principles and application. Dordrecht: Kluwer.Google Scholar
  7. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.Google Scholar
  8. McGuire, G. R., & Youngson, M. A. (2002). Assessing ICT Assessment in Mathematics. Maths CAA Series, Maths, Stats and OR network. Accessed 11/11/2005.
  9. McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and evolving a concept. In Proceedings of the Graphics Interface 2000. Canadian Human–Computer Communications Society (pp. 179–186), Toronto.Google Scholar
  10. Raikes, N., & Harding, R. (2003). The horseless carriage stage: Replacing conventional measures. Assessment in Education, 10(3), 267–277.CrossRefGoogle Scholar
  11. Russell, M., Goldberg, A., & O’Connor, K. (2003). Computer-based testing and validity: a look back into the future. Assessment in Education, 10(3), 278–293.CrossRefGoogle Scholar
  12. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4, 295–312.CrossRefGoogle Scholar
  13. Threlfall, J., & Pool, P. (2004). How might the assessment of mathematics through dynamic interactive computer items be different from that in conventional tests?. Paper presented at the 10th International Conference on Mathematics Education (ICME 10), Copenhagen, Denmark, July 4–11.Google Scholar
  14. Watson, A. (2003). Affordances, constraints and attunements in mathematical activity. In Proceedings of the British Society for Research into Learning Mathematics, 23(2), pp. 103–108.Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2007

Authors and Affiliations

  • John Threlfall
    • 1
  • Peter Pool
    • 1
  • Matthew Homer
    • 1
  • Bronwen Swinnerton
    • 1
  1. 1.Assessment and Evaluation Unit, School of EducationUniversity of LeedsLeedsUK

Personalised recommendations