Abstract
This chapter details my research involvement with the Hong Kong Examinations and Assessment Authority (HKEAA) leading up to the wholesale implementation of onscreen marking (OSM) in Hong Kong – where, from 2012 onwards, paper-based marking (PBM) has been superseded by OSM in Hong Kong public examinations. This chapter outlines a series of validation studies conducted into OSM with the support of the HKEAA. The context for the uptake of OSM worldwide is first briefly laid out, after which – to put the issues into perspective – an overview of the OSM process in Hong Kong is provided. Aspects of a number of studies related to the implementation of English are then outlined. This chapter closes with some observations about OSM and its implementation in Hong Kong, as well as further avenues for investigation into the effectiveness and applicability of OSM.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
While it would have been preferable that the distribution of new/experienced markers in the sample matched the distribution in the live HKCEE Writing examination, it was not possible to achieve this for two reasons. One, more experienced markers had better marking statistics and were thus more ‘eligible’ to participate. Two, fewer new markers expressed an interest in participating.
- 2.
A key statistical method of analysis, increasingly accepted over the past decade in the area of language performance tests is Multifaceted Rasch Analysis (MFRA), since it allows for situational factors such as marker severity, prompt difficulty, etc., to be modelled and compensated for (Weir 2005, p. 199; Bond and Fox 2007).
References
Adams, C. (2005, September 4–9). How does assessment differ when e-marking replaces paper-based marking? Paper presented at the 31st annual international Association for Educational Assessment Conference “Assessment and the future of schooling and learning”, Abuja, Nigeria.
Attali, Y., & Burstein, J. (2005). Automated essay scoring with e-rater v.2. Journal of Technology, Learning, and Assessment, 4(3). Retrieved August 6, 2013, from http://origin-www.ets.org/Media/Products/e-rater/erater_IAEA.pdf
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: Fundamental measurement in the human sciences (2nd ed.). Mahwah: Lawrence Erlbaum.
Coniam, D. (1991). Essay marking: A comparison of criterion-referenced and norm-referenced marking. Institute of Language in Education Journal, 7, 154–164.
Coniam, D. (2002). IT use in the English language classroom in Hong Kong: How far is Government policy being achieved? Education Journal, 30(2), 21–39.
Coniam, D. (2009a). A comparison of onscreen and paper-based marking in the Hong Kong public examination system. Educational Research and Evaluation, 15(3), 243–263.
Coniam, D. (2009b). Discrepancy essays: Natural phenomenon or problem to be solved? Melbourne Papers in Language Testing, 14(2), 1–31.
Coniam, D. (2010a). Validating onscreen marking in Hong Kong. Asia Pacific Education Review, 11(3), 423–431.
Coniam, D. (2010b). Examining negative attitudes towards onscreen marking in Hong Kong. Education Journal, 37(1–2), 71–87.
Coniam, D. (2010c). A qualitative examination of the attitudes of Liberal Studies markers towards onscreen marking. British Journal of Educational Technology, 42(6), 1042–1054.
Coniam, D. (2013). The increasing acceptance of onscreen marking – The ‘tablet computer’ effect. Journal of Educational Technology & Society, 16(3), 1–10.
Dörnyei, Z. (2003). Questionnaires in second language research: Construction, administration, and processing. Mahwah: Lawrence Erlbaum.
Education Manpower Bureau [EMB]. (1998). Information technology for learning in a new era: Five-year strategy 1998/99 to 2002/03. Hong Kong: Printing Department.
Falvey, P., & Coniam, D. (2010). A qualitative study of the response of raters towards onscreen and paper-based marking. Melbourne Papers in Language Testing, 15(1), 1–26.
Fowles, D. (2008, September 7–12). Does marking images of essays on screen retain marker confidence and reliability? Paper presented at the International Association for Educational Assessment Annual Conference, Cambridge, UK.
Hatch, E., & Lazaraton, A. (1991). The research manual. Boston: Heinle and Heinle.
Hong Kong Examinations and Assessment Authority (HKEAA). (2007). HKCEE English language examination report and question papers. Hong Kong: Hong Kong Examinations and Assessment Authority.
Johnson, M., Nádas, R., & Bell, J. F. (2010). Marking essays on screen: An investigation into the reliability of marking extended subjective texts. British Journal of Educational Technology, 41(5), 814–826.
King, R. (1994). Historical survey of English language testing in Hong Kong. In J. Boyle & P. Falvey (Eds.), English language testing in Hong Kong (pp. 3–29). Hong Kong: The Chinese University Press.
Linacre, J. M. (1997). Communicating examinee measures as expected ratings. Rasch Measurement Transactions, 11(1), 550–551. Retrieved from http://www.rasch.org/rmt/rmt111m.htm.
Linacre, J. M. (2004). A user’s guide to FACETS: Rasch-model computer programs. Chicago: Winsteps.com.
Luo, Y. H., & Liu, T. M. (2009). Review of online marking research. China Examinations, 11, 34–37 (in Chinese); (Retrieved January 5, 2011, from http://www.cqvip.com/qk/87265x/200911/32082753.html).
Ma, S. Y. (2004). A review and analysis of online marking. China Examinations, 7, 24–26 (in Chinese); (Retrieved January 5, 2011, from http://www.cqvip.com/qk/87265x/200407/10344905.html).
Weir, C. J. (2005, May 19–21). A socio-cognitive approach to test validation. Plenary presented at the ALTE Second International Conference, Berlin.
Yan, Z., & Coniam, D. (2013). Assessing the ease of use in the environment and markers’ acceptance of onscreen marking: A Rasch measurement perspective. Educational Research and Evaluation, 19(5), 461–483.
Zhang, Y., Powers, D., Wright, W., & Morgan, R. (2003). Applying the online scoring network (OSN) to advanced program placement program (AP) Tests (ETS Research Report RR-03-12).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Coniam, D. (2014). Perspectives into the Onscreen Marking of English in Hong Kong. In: Coniam, D. (eds) English Language Education and Assessment. Springer, Singapore. https://doi.org/10.1007/978-981-287-071-1_15
Download citation
DOI: https://doi.org/10.1007/978-981-287-071-1_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-287-070-4
Online ISBN: 978-981-287-071-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)