Advertisement

Chinese Users’ Perceptions of the Use of Automated Scoring for a Speaking Practice Test

  • Xiaoming Xi
  • Jonathan Schmidgall
  • Yuan Wang

Abstract

The use of automated scoring in large-scale testing for high-stakes purposes has generated a lot of debate and controversy in the last decade, and researchers have described a variety of ways in which the use of automated scoring may impact validity (e.g., Clauser, Kane, & Swanson, 2002; Weigle, 2010; Xi, 2010). Validity frameworks that tie automated scoring into the overall validity argument for using an assessment (Williamson, Xi & Breyer, 2012; Xi, 2012) highlight the many ways in which the use of automated scoring may impact the validity argument for the entire assessment, including test takers’ interactions with assessment tasks; the accuracy and generalizability of assessment scores; test takers’ interpretations and uses of assessment scores; and the overall consequences for test takers, the educational system, and the broader society.

Keywords

Test Taker Validity Argument Computer Score Practice Test Test Task 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bennett, R. E., & Bejar, I. I. (1998). Validity and automated scoring: It’s not only the scoring. Educational Measurement: Issues and Practice, 17(4), 9–17.CrossRefGoogle Scholar
  2. Brent, E., & Townsend, M. (2006). Automated essay grading in the sociology classroom. In P. F. Ericsson & R. Haswell (Eds.), Machine scoring of student essays: Truth and consequences (pp. 177–198). Logan, Utah: Utah State University Press.Google Scholar
  3. Bridgeman, B., Powers, D., Stone, E., & Mollaun, P. (2012). TOEFL iBT Speaking test scores as indicators of oral communicative language proficiency. Language Testing, 29(1), 91–108.CrossRefGoogle Scholar
  4. Clauser, B. E., Kane, M. T., & Swanson, D. B. (2002). Validity issues for performance-based tests scored with computer-automated scoring systems. Applied Measurement in Education, 15(4), 413–432.CrossRefGoogle Scholar
  5. Deane, P. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18(1), 7–24.CrossRefGoogle Scholar
  6. Herrington, A., & Moran, C. (2001). What happens when machines read our students’ writing? College English, 63(4), 480–499.CrossRefGoogle Scholar
  7. Herrington, A., & Moran, C. (2006). Write placer plus in place: An exploratory case study. In P. F. Ericsson & R. Haswell (Eds.), Machine scoring of student essays: Truth and consequences (pp. 114–129). Logan, Utah: Utah State University Press.Google Scholar
  8. Kelly, P. A. (2001). Automated scoring of essays: Evaluating score validity (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3028998)Google Scholar
  9. Landauer, T. K., Laham, D., & Foltz, P. (2003). Automatic essay assessment. Assessment in Education, 10(3), 295–308.CrossRefGoogle Scholar
  10. Powers, D. E., Burstein, J. C., Chodorow, M., Fowles, M. E., & Kukich, K. (2001). Stumping e-rater: Challenging the validity of automated essay scoring (Graduate Record Examination Board Professional Report No. 98-08bP). Princeton, NJ: Educational Testing Service.Google Scholar
  11. Weigle, S. C. (2010). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing, 27(3), 335–353.CrossRefGoogle Scholar
  12. Williamson, D., Xi, X., & Breyer, J. (2012). A Framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31(1), 2–13.CrossRefGoogle Scholar
  13. Xi, X. (2010). Automated scoring and feedback systems x2014; Where are we and where are we heading? Language Testing, 27(3), 291–300.CrossRefGoogle Scholar
  14. Xi, X. (2012). Validity and the automated scoring of performance tests. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 438–451). New York: Routledge.Google Scholar
  15. Xi, X., Higgins, D., Zechner, K., & Williamson, D. M. (2008). Automated scoring of spontaneous speech using SpeechRater v1.0 (Research Report No. RR-08-62). Princeton, NJ: Educational Testing Service.Google Scholar
  16. Yang, Y., Buckendahl, C. W., Juszkiewicz, P. J., & Bhola, D. S. (2002). A review of strategies for validating computer-automated scoring. Applied Measurement in Education, 15(4), 391–412.CrossRefGoogle Scholar

Copyright information

© Xiaoming Xi, Jonathan Schmidgall and Yuan Wang 2016

Authors and Affiliations

  • Xiaoming Xi
  • Jonathan Schmidgall
  • Yuan Wang

There are no affiliations available

Personalised recommendations