Abstract
In 2012, onscreen marking replaced paper-based marking throughout the entire public examinations system in Hong Kong. To chart and validate this major change, a series of studies, mainly quantitative, were conducted. To complement the quantitative studies, therefore, this chapter reports on an interview-based qualitative study into the responses of markers to OSM and PBM in the English Language examination. It investigates the in-depth responses not only of those who had experience of OSM but of those who formerly had marked only on paper. Using semi-structured interviews to probe participants’ views, issues arose that had not been revealed in the quantitative studies. In addition to revealing abundant insights, new issues were raised in a number of areas, especially reading onscreen; training and standardisation; attitudes towards marking at centres and marking at home; and the accuracy and reliability of marking on screen.
This study originally appeared as: Falvey, Peter and Coniam, David (2010). A qualitative study of the response of markers towards onscreen and paper-based marking. Melbourne Papers in Language Testing, 15(1), 1–26. Reprinted with kind permission from the Language Testing Research Centre, University of Melbourne, Australia.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Borko, H., & Putman, R. T. (1996). Learning to teach. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 673–708). New York: Macmillan.
Coniam, D. (2009a). A comparison of onscreen and paper-based marking in the Hong Kong public examination system. Educational Research and Evaluation, 15(3), 243–263.
Coniam, D. (2010). Examining negative attitudes towards onscreen marking in Hong Kong. Education Journal, 37(1–2), 71–87.
Hannan, A. (2007). Interviews in education research. Faculty of Education, University of Plymouth. Retrieved January 11, 2011, from http://www.edu.plymouth.ac.uk/resined/interviews/inthome.htm.
Harrell, M. C., & Bradley, M. A. (2007). Semi-structured interviews and focus groups. RAND, National Defense Research Institute. Retrieved January 1, 2011, from http://www.rand.org/pubs/technical_reports/2009/RAND_TR718.pdf.
Miles, M., & Huberman, M. (1984). Qualitative data analysis: A source book for new methods. Thousand Oaks: Sage Publications.
Nespor, J. (1987). The role of beliefs in the practice of teaching. Journal of Curriculum Studies, 19, 317–328.
Silverman, D. (2000). Doing qualitative research: A practical handbook. London: Sage.
Wall, D. (2005). The impact of high-stakes testing on classroom teaching: A case study using insights from testing and innovation theory. In Studies in language testing volume 22. Cambridge: ESOL and Cambridge University Press.
Author information
Authors and Affiliations
Appendices
Appendix 1: Interviewees’ Profiles
No. | Names | New/exp’d marker | Age | Sex M/F | Teaching level S1–3/S4–5/S6–7 | Tchg exp’nce (years) | Panel chair | ||
|---|---|---|---|---|---|---|---|---|---|
1. | N1 | N | 26–30 | F | – | Y | – | 2–5 | N |
2. | N2 | N | 26–30 | F | Y | Y | – | 2–5 | N |
3. | N3 | N | 31–40 | F | Y | Y | – | 11–20 | N |
4. | N4 | N | 31–40 | F | Y | Y | – | 11–20 | N |
5. | N5 | N | 31–40 | F | Y | Y | Y | 11–20 | N |
6. | E6 | E | 31–40 | F | – | Y | Y | 11–20 | N |
7. | E7 | E | 31–40 | M | Y | Y | Y | 11–20 | Y |
8. | E8 | E | 41–50 | M | – | Y | Y | >20 | N |
9. | E9 | E | 31–40 | F | Y | Y | – | 11–20 | N |
10. | E10 | E | 41–50 | F | – | Y | – | 11–20 | N |
11. | E11 | E | 31–40 | F | Y | Y | – | 11–20 | N |
12. | E12 | E | 31–40 | F | – | Y | Y | 6–10 | N |
13. | E13 | E | 31–40 | F | Y | Y | – | 6–10r | N |
14. | E14 | E | 26–30 | F | Y | Y | – | 6–10 | N |
15. | N15 | E | 31–40 | M | Y | Y | – | 6–10 | Y |
16. | E16 | E | 31–40 | F | Y | Y | Y | 6–10 | N |
17. | E17 | E | 41–50 | F | Y | Y | Y | 11–20 | N |
Total: | N – 5 | M – 3 | 2 | ||||||
E – 12 | |||||||||
F – 14 | |||||||||
Appendix 2: Checklist of Guideline Questions for the Semi-structured Interviews
The checklist begins with a repetition of the research hypotheses:
-
Hypothesis 1: Markers will be sufficiently competent technologically to accept the new OSM medium
-
Hypothesis 2: Markers will not be negative in terms of the OSM medium, having no preference for either marking medium
A. Attitudes Towards Using Computers
-
How computer savvy are you?
-
How often do you use a computer at work or at leisure?
-
How do you find the technical operation of the OSM medium?
-
Is the system difficult or easy to manipulate?
-
How comfortable are you reading off the screen?
-
How do you find the ergonomic design (comfortableness and ease of operation) of the computer facilities at the marking centers?
B. Attitudes Towards OSM
-
How do you find OSM?
-
What advantages to OSM are there when compared with PBM; e.g. reliability and consistency between markers?
-
How useful do you find the control scripts and qualifying scripts.
-
How far do the control and qualifying scripts help to ensure quality and eliminate the potential for disparity?
-
Can you tell us about any improvements you would like to see in OSM?
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Coniam, D., Falvey, P. (2016). The Second Validation Study: A Qualitative Study of the Response of Markers Towards Onscreen and Paper-Based Marking. In: Validating Technological Innovation. Springer, Singapore. https://doi.org/10.1007/978-981-10-0434-6_6
Download citation
DOI: https://doi.org/10.1007/978-981-10-0434-6_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-0432-2
Online ISBN: 978-981-10-0434-6
eBook Packages: EducationEducation (R0)