Skip to main content
SpringerLink
Log in

Learning by Assessing in an EFL Writing Class

  • Conference paper
  • First Online:
Pacific Rim Objective Measurement Symposium (PROMS) 2012 Conference Proceeding

Abstract

This study used many-faceted Rasch measurement to research peer assessment in EFL writing classes, following previous research which reported acceptance of the pedagogical benefits by students of attention paid to a rubric during peer assessment. Pre and post treatment writing was compared on two rubrics, one targeting specific instructional items, the second intended to measure general academic writing. Students used the instructional rubric to conduct peer assessment, but were not exposed to the secondary rubric. Substantively and statistically significant gains were observed on the instructional rubric but not on the secondary rubric, providing evidence of learning by assessing. Response patterns suggested holistic rating by peer raters, resulting in effective rank ordering of overall performances but an inability to provide formative feedback, supporting the view that the mechanism of learning was awareness arising from learning by assessing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

    Google Scholar 

  • Blanchard, K., & Root, C. (2010). Ready to write 3: From paragraph to essay (3rd ed.). White Plains: Pearson Longman.

    Google Scholar 

  • Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model (2nd ed.). London: Lawrence Erlbaum Associates.

    Google Scholar 

  • Brown, J. D., & Hudson, T. D. (2002). Criterion-referenced language testing. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Brown, J. D., Hudson, T. D., Norris, J. M., & Bonk, W. J. (2002). An investigation of second language task-based performance assessments. Honolulu: University of Hawaii.

    Google Scholar 

  • Chapelle, C. A. (2008). The TOEFL validity argument. In C. A. Chapelle, M. K. Enright, & J. M. Jamieson (Eds.), Building a validity argument for the Test of English as a Foreign Language (pp. 319–352). New York: Routledge.

    Google Scholar 

  • Cheng, W., & Warren, M. (2005). Peer assessment of language proficiency. Language Testing, 22(1), 93–121. doi:10.1191/0265532205lt298oa.

    Article  Google Scholar 

  • Cho, Y., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39(5), 629–643. doi:10.1007/s11251-010-9146-1.

    Article  Google Scholar 

  • Cho, K., & MacArthur, C. (2010). Student revision with peer and expert reviewing. Learning and Instruction, 20(4), 328–338.

    Article  Google Scholar 

  • Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73–84. doi:10.1037/a0021950.

    Article  Google Scholar 

  • Cho, K., & Schunn, C. D. (2010). Developing writing skills through students giving instructional explanations. In M. K. Stein & L. Kucan (Eds.), Instructional explanations in the disciplines. New York: Springer.

    Google Scholar 

  • Crinon, J., & Marin, B. (2010). The role of peer feedback in learning to write explanatory texts: Why the tutors learn the most. Language Awareness, 19(2), 111–128. doi:10.1080/09658411003746604.

    Article  Google Scholar 

  • Diab, R., & Balaa, L. (2011). Developing detailed rubrics for assessing critique writing: Impact on EFL university students’ performance and attitudes. TESOL Journal, 2(1), 52–72. doi:10.5054/tj.2011.244132.

    Article  Google Scholar 

  • Engelhard, G. (2009). Using item response theory and model-data fit to conceptualize differential item and person functioning for students with disabilities. Educational and Psychological Measurement, 69(4), 585–602. doi:10.1177/0013164408323240.

    Article  Google Scholar 

  • ETS. (2008). The TOEFL® Test – Test of English as a Foreign Language™. Retrieved from http://tinyurl.com/zocgc. Accessed 28 Mar 2008.

  • ETS. (2012). About the GRE® revised General Test. Retrieved from http://www.ets.org/gre/revised_general/about. Accessed 19 Jan 2012.

  • Farrokhi, F., Esfandiari, R., & Schaefer, E. (2012). A many-facet Rasch measurement of differential rater severity/leniency in three types of assessment. JALT Journal, 34(1), 79–101.

    Google Scholar 

  • Fukuzawa, M. (2010). Validity of peer assessment of speech performance. Annual Review of English Language Education in Japan, 21, 181–190.

    Google Scholar 

  • Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9(6), 495–522. doi:10.1002/acp.2350090604.

    Article  Google Scholar 

  • Henning, G. (1987). A guide to language testing. Boston: Heinle & Heinle.

    Google Scholar 

  • Hirai, A., Ito, N., & O’ki, T. (2011). Applicability of peer assessment for classroom oral performance. JLTA Journal, 14, 41–59.

    Google Scholar 

  • Holster, T. A., & Pellowe, W. R. (2011). Using a mobile audience response system for classroom peer assessment. Paper presented at the JALT CALL 2011 conference, Kurume University, Kurume.

    Google Scholar 

  • Hughes, A. (2003). Testing for language teachers. Cambridge: Cambridge University Press.

    Google Scholar 

  • Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536. doi:10.1111/j.1467-8535.2009.00968.x.

    Article  Google Scholar 

  • Linacre, J. M. (1994). Many-facet Rasch measurement (2nd ed.). Chicago: MESA Press.

    Google Scholar 

  • McNamara, T. F. (1996). Measuring second language performance. Harlow: Pearson Education.

    Google Scholar 

  • Mendonça, C. O., & Johnson, K. E. (1994). Peer review negotiations: Revision activities in ESL writing instruction. TESOL Quarterly, 28(4), 745–769.

    Article  Google Scholar 

  • Min, H. T. (2006). The effects of trained peer review on EFL students’ revision types and writing quality. Journal of Second Language Writing, 15(2), 118–141. doi:10.1016/j.jslw.2006.01.003.

    Article  Google Scholar 

  • Mok, J. (2011). A case study of students’ perceptions of peer assessment in Hong Kong. ELT Journal, 65(3), 230–239. doi:10.1093/elt/ccq062.

    Article  Google Scholar 

  • Pellowe, W. R. (2002). Keitai-assisted language learning (KALL). Paper presented at the 28th JALT international conference, Granship Conference Center, Shizuoka.

    Google Scholar 

  • Pellowe, W. R. (2010a). MOARS (Version 0.8.3) [Audience response system]. Retrieved from http://moars.com

  • Pellowe, W. R. (2010b). Quiz and survey system for mobile devices. Paper presented at the 36th JALT international conference, WINC, Aichi.

    Google Scholar 

  • Roskams, T. (1999). Chinese EFL students’ attitudes to peer feedback and peer assessment in an extended pairwork setting. RELC Journal, 30(1), 79–123. doi:10.1177/003368829903000105.

    Article  Google Scholar 

  • Saito, H. (2008). EFL classroom peer assessment: Training effects on rating and commenting. Language Testing, 25(4), 553–581. doi:10.1177/0265532208094276.

    Article  Google Scholar 

  • Saito, H., & Fujita, T. (2004). Characteristics and user acceptance of peer rating in EFL writing classrooms. Language Teaching Research, 8(1), 31–54. doi:10.1191/1362168804lr133oa.

    Article  Google Scholar 

  • Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129–158. doi:10.1093/applin/11.2.129.

    Article  Google Scholar 

  • Schuchert, S. A. (2004). The neurobiology of attention. In J. H. Schumann, S. E. Crowell, N. E. Jones, N. Lee, S. A. Schuchert, & L. A. Wood (Eds.), The neurobiology of learning (pp. 143–174). Mahway: Lawrence Erlbaum Associates.

    Google Scholar 

  • Schumann, J. H., & Wood, L. A. (2004). The neurobiology of motivation. In J. H. Schumann, S. E. Crowell, N. E. Jones, N. Lee, S. A. Schuchert, & L. A. Wood (Eds.), The neurobiology of learning (pp. 23–42). London: Lawrence Erlbaum Associates.

    Google Scholar 

  • Thompson, B. (1999). Statistical significance tests, effect size reporting and the vain pursuit of pseudo-objectivity. Theory & Psychology, 9(2), 191–196. doi:10.1177/095935439992007.

    Article  Google Scholar 

  • Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249–276. doi:10.3102/00346543068003249.

    Article  Google Scholar 

  • Tsui, A. B. M., & Ng, M. (2000). Do secondary L2 writers benefit from peer comments? Journal of Second Language Writing, 9(2), 147–170.

    Article  Google Scholar 

  • Weigle, S. C. (1994). Effects of training on raters of ESL compositions. Language Testing, 11(2), 197–223. doi:10.1177/026553229401100206.

    Article  Google Scholar 

  • Wong Mei Ha, H., & Storey, P. (2006). Knowing and doing in the ESL writing class. Language Awareness, 15(4), 283–300.

    Article  Google Scholar 

  • Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback in a Chinese EFL writing class. Journal of Second Language Writing, 15(3), 179–200.

    Article  Google Scholar 

  • Yarrow, F., & Topping, K. J. (2001). Collaborative writing: The effects of metacognitive prompting and structured peer interaction. The British Journal of Educational Psychology, 71, 261–282.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Academic English Program, Fukuoka Women’s University, Fukuoka, Japan

    Trevor A. Holster & J. Lake

  2. Department of Human Engineering, Kinki University, Higashiosaka, Japan

    William R. Pellowe

  3. Department of English, Fukuoka High School, Fukuoka, Japan

    Aaron Hahn

Authors
  1. Trevor A. Holster
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William R. Pellowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Lake
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aaron Hahn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Trevor A. Holster .

Editor information

Editors and Affiliations

  1. , Faculty of Foreign Studies, University of Jiaxing, No. 65 Yuexiu Nanlu, Jiaxing, 314001, Zhejiang, China, People's Republic

    Quan Zhang

  2. , Faculty of Foreign Studies, University of Jiaxing, No. 65 Yuexiu Nanlu, Jiaxing, Jiaxing, 314001, Zhejiang, China, People's Republic

    Hong Yang

Appendix

Appendix

7.1.1 Essay Rating Instructions and Rubric

7.1.1.1 Essay Revision

Read other students’ essays. Rate each essay from “A” to “D” on the following points by marking the bubbles on the grading sheet.

他の学生の発表を見て評価をします。以下の評価基準を参考にして、評価シートのA~Dを塗りつぶして下さい。

“A” = Excellent performance.

(素晴らしい。)

“B” = Good performance, but could be improved.

(良いが、改善出来る部分もある。)

“C” = Weak performance, should be improved.

(良いとは言えない。改善した方が良い。)

“D” = Very weak performance, must be improved.

(良くない。改善すべき。)

  1. 1.

    Thesis stment: How well does the introduction identify the focus of the essay using a thesis stment?

  2. 2.

    Introduction: How well does the introduction preview the main points of the essay?

  3. 3.

    Conclusion: How well does the conclusion summarize the main points of the essay?

  4. 4.

    Organization: Are the supporting paragraphs in a logical order?

  5. 5.

    Unity: Does each supporting paragraph have a clear topic sentence and focus?

  6. 6.

    Support: Do the supporting paragraphs support the essay focus with specific details?

  7. 7.

    Coherence: Are the supporting sentences in each paragraph organized in a logical way?

  8. 8.

    Cohesion: Did the writer use transition words to guide the reader from one idea to the next?

  9. 9.

    Relevance: Are all the supporting sentences relevant?

  10. 10.

    Formatting: Is the essay formatted correctly?

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Holster, T.A., Pellowe, W.R., Lake, J., Hahn, A. (2013). Learning by Assessing in an EFL Writing Class. In: Zhang, Q., Yang, H. (eds) Pacific Rim Objective Measurement Symposium (PROMS) 2012 Conference Proceeding. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37592-7_7

Download citation

Publish with us

Policies and ethics

Search

Navigation