Abstract
In order to track the development of young readers’ oral reading fluency (ORF) at scale, it is necessary to move away from hand-scoring responses to automating the assessment of ORF, while retaining the quality of the scores. We present a method for improving automated ORF scoring that utilizes an observed systematicity in machine error, namely, that cases with low estimated reading accuracy are harder to score correctly for fluency. We show that the method yields an improved performance, including on out-of-domain data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balogh, J., Bernstein, J., Cheng, J., Van Moere, A., Townshend, B., Suzuki, M.: Validation of automated scoring of oral reading. Educ. Psychol. Measur. 72(3), 435–452 (2012)
Barth, A.E., Tolar, T.D., Fletcher, J.M., Francis, D.: The effects of student and text characteristics on the oral reading fluency of middle-grade students. J. Educ. Psychol. 106(1), 162–180 (2014)
Bernstein, J., Cheng, J., Balogh, J., Rosenfeld, E.: Studies of a self-administered oral reading assessment. In: Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education, Stockholm, Sweden, pp. 180–184 (2017)
Biancarosa, G., Kennedy, P.C., Park, S., Otterstedt, J., Gearin, B., Yoon, H.: 8th edition of dynamic indicators of basic early literacy skills (DIBELS®): administration and scoring guide. University of Oregon, Technical report (2019)
Capes, T., et al.: Siri on-device deep learning-guided unit selection text-to-speech system. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2017, pp. 4011–4015 (2017)
Godde, E., Bailly, G., Bosse, M.L.: Reading prosody development: automatic assessment for a longitudinal study. In: Speech & Language Technology for Education (SLaTE) (2019)
Good, R., Kaminski, R.: Dynamic indicators of basic early literacy skills. Institute for the Development of Educational Achievement, Eugene, OR (2002)
Hasbrouck, J., Tindal, G.: An update to compiled ORF norms. Behavioral Research and Teaching, University of Oregon, Technical report (2017)
Loukina, A., et al.: Automated estimation of oral reading fluency during summer camp e-Book reading with MyTurnToRead. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2019, pp. 21–25 (2019). https://doi.org/10.21437/Interspeech.2019-2889
Loukina, A., Klebanov, B.B., Lange, P., Gyawali, B., Qian, Y.: Developing speech processing technologies for shared book reading with a computer. In: Proceedings of WOCCI 2017: 6th International Workshop on Child Computer Interaction, pp. 46–51 (2017). https://doi.org/10.21437/WOCCI.2017-8
Madnani, N., et al.: MyTurnToRead: an interleaved e-book reading tool for developing and struggling readers. In: Proceedings of the 57th Conference of the Association for Computational Linguistics: System Demonstrations, pp. 141–146. Association for Computational Linguistics, Florence (2019). https://www.aclweb.org/anthology/P19-3024
Napolitano, D., Sheehan, K., Mundkowsky, R.: Online readability and text complexity analysis with TextEvaluator. In: Proceedings of the North American Chapter of the Association for Computational Linguistics, pp. 96–100 (2015)
Ridgeway, G.: The state of boosting. Comput. Sci. Stat. 31, 172–181 (1999)
van Santen, J.P.: Assignment of segmental duration in text-to-speech synthesis. Comput. Speech Lang. 8(2), 95–128 (1994)
Sheehan, K.M., Kostin, I., Napolitano, D., Flor, M.: The TextEvaluator tool: helping teachers and test developers select texts for use in instruction and assessment. Elementary School J. 115(2), 184–209 (2014)
Tokuda, K., Hashimoto, K., Oura, K., Nankaku, Y.: Temporal modeling in neural network based statistical parametric speech synthesis. In: 9th ISCA Speech Synthesis Workshop, no. 2, pp. 106–111 (2016)
Yoshimura, T., Tokuda, K., Kobayashi, T., Masuko, T., Kitamura, T.: Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis. EUROSPEECH 1999, 2347–2350 (1999)
Zen, H., Tokuda, K., Black, A.W.: Statistical parametric speech synthesis. Speech Commun. 51(11), 1039–1064 (2009)
Acknowledgement
We thank J. R. Lockwood for expert advice and help with the analyses; T.O’Reilly, A. Misra, K. Zechner for their helpful comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Klebanov, B.B., Loukina, A. (2021). Exploiting Structured Error to Improve Automated Scoring of Oral Reading Fluency. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12749. Springer, Cham. https://doi.org/10.1007/978-3-030-78270-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-78270-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78269-6
Online ISBN: 978-3-030-78270-2
eBook Packages: Computer ScienceComputer Science (R0)