Abstract
Writing assessments make up an important part of the learning process as one masters the important linguistic skill of writing. However, this process has not been implemented effectively or on a large scale because the task of essay scoring is very time-consuming. The solution to this problem is AES, where machines are used to automatically score essays. In fact, the application of AES to English learning has been successful. Due to differences in linguistic characteristics, a redesign is needed before AES can be applied to Chinese learning. The purpose of this chapter is to introduce ACES, an automated system for scoring Chinese essays, and explain the basic framework, design principles, and scoring accuracy of the system. Unlike some end-to-end AES systems, ACES’ basic framework is designed to provide more interpretative features. The experimental results show that the performance of the ACES system is stable and reliable, and on par with other commercial English AES systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Attali, Y., & Burstein, J. (2006). Automated scoring with e-raterV.2. The Journal of Technology, Learning and Assessment, 4(3), 1–30.
Bai, S. T., & Shi, A. W. (2002). A comparative study of figures of speech between Chinese and English. Journal of Xinzhou Teachers University,18(1), 70–71.
Burstein, J., Kukich, K., Wolff, S., Lu, C., Chodorow M., Braden-Harder, L., & Harris, M. D. (1998). Automated scoring using a hybrid feature identification technique. In Proceedings of the 36th Annual Meeting of the Association of Computational Linguistics (pp. 206–210). Montreal, Canada.
Cai, J. G. (2006). Contrastive study of writing and rhetoric in English and Chinese. Shanghai, China: Fudan University Press.
Chang. T. H. (2015). The development of Chinese word segmentation tool for educational text. In Proceedings of the 7th International Conference on Information (pp. 179–182). Taipei, Taiwan.
Chang. T. H., Chen, H. C., & Yang, C. H. (2015). Introduction to a proofreading tool for Chinese spelling check task of SIGHAN-8. In Proceedings of the 8th SIGHAN Workshop on Chinese Language Processing (pp. 50–55). Beijing, China.
Chang, T. H., & Lee, C. H. (2003). Automatic Chinese unknown word extraction using small-corpus-based method. In Proceedings of IEEE International Conference on Natural language processing and knowledge engineering (pp. 459–464). Beijing, China.
Chang, T. H., & Lee, C. H. (2009). Automatic Chinese essay scoring using connections between concepts in paragraphs. In Proceedings of the International Conference on Asian Language Processing (pp. 265–268). Singapore.
Chang, T. H., Lee, C. H., & Tam, H. P. (2007a). On issues of feature extraction in Chinese automatic essay scoring system. In Proceedings of the 13th International Conference on Artificial Intelligence in Education (pp. 545–547). Los Angeles, CA.
Chang, T. H., Lee, C. H., & Tam, H. P. (2007b). On developing techniques for automated Chinese essay scoring: A case in ACES system. In Proceedings of the Forum for Educational Evaluation in East Asia (pp. 151–152). Taipei, Taiwan.
Chang, T. H., Lee, C. H., Tsai, P. Y., & Tam, H. P. (2009). Automated essay scoring using set of literary sememes. Information: An International Interdisciplinary Journal, 12(2), 351–357.
Chang, T. H., Liu, C. L., Su, S. Y., & Sung, Y. T. (2014). Integrating various features to grade students’ writings based on improved multivariate Bernoulli model. Information: An International Interdisciplinary Journal, 17(1), 45–52.
Chen, K. J., Luo, C. C., Chang, M. C., Chen, F. Y., Chen, C. J., Huang, C. R., et al. (2003). Sinica Treebank. In A. Abeillé (Ed.), Treebanks: Building and using parsed corpora (pp. 231–248). Dordrecht: Springer.
Dong, Z., & Dong, Q. (2003). HowNet—A hybrid language and knowledge resource. In Proceedings of International Conference on Natural Language Processing and Knowledge Engineering (pp. 820–824). Beijing, China.
Elliot, S. M. (2003). IntelliMetric: From here to validity. In M. D. Shermis & J. C. Burstein (Eds.), Automated essay scoring: A cross-disciplinary perspective (pp. 71–86). Mahwah, NJ: Lawrence Erlbaum Associates.
Jiao, C. Y. (2002). A syntactic comparison and transformation between English and Chinese. Journal of Yancheng Teachers College,22(2), 83–87.
Kaplan, R. B. (1966). Cultural thought patterns in intercultural education. Language Learning,16(1–2), 1–20.
Lafferty, J., McCallum, A., & Pereira, F. C. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (pp. 282–289). Williamstown, MA.
Lee, G. N. (1999). Contrastive studies of figures of speech in English and Chinese. Fuzhou, China: Fujian People’s Publishing House.
Lee, X. L., & Zeng, K. (2001). Heterogeneity and homogeneity of sentence structure in English and Chinese. Journal of Shenyang University,12(1), 52–55.
Liu, L. J. (1999). A contrastive study of discourse structure in English and Chinese. Modern Foreign Languages,86(4), 408–419.
Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge, MA: MIT press.
Peng, X., Ke, D., Chen, Z., & Xu, B. (2010). Automated Chinese essay scoring using vector space models. In Proceedings of the 4th International Universal Communication Symposium (pp. 149–153). Beijing, China.
Ramineni, C., Trapani, C. S., Williamson, D. M., Davey, T., & Bridgeman, B. (2012). Evaluation of the e‐rater® scoring engine for the GRE® issue and argument prompts (ETS RR–12-02). https://www.ets.org/Media/Research/pdf/RR-12-02.pdf. Accessed August 31, 2017.
Rudner, L. M., Garcia, V., & Welch, C. (2006). An evaluation of IntelliMetric™ essay scoring system. The Journal of Technology, Learning and Assessment,4(4), 1–22.
Rudner, L. M., & Liang, T. (2002). Automated essay scoring using Bayes’ theorem. The Journal of Technology, Learning, and Assessment,1(2), 1–21.
Scollon, R., Scollon, S. W., & Kirkpatrick, A. (2000). Contrastive discourse in Chinese and English: A critical appraisal. Beijing: Foreign Language Teaching and Research Press.
Sung, Y. T., Lin, W. C., Dyson, S. B., Chang, K. E., & Chen, Y. C. (2015). Leveling L2 texts through readability: Combining multilevel linguistic features with the CEFR. The Modern Language Journal, 99(2), 371–391.
Sung, Y. T., Chang, T. H., Lin, W. C., Hsieh, K. S., & Chang, K. E. (2016). CRIE: An automated analyzer for Chinese texts. Behavior Research Methods,48(4), 1238–1251.
Zeng, X. H. (1997). Enhancing English writing ability by comparing the difference of organization in paragraphs between English and Chinese. Journal of Nanchang Vocation-technical Teachers College,4, 75–77.
Acknowledgements
This study was partially supported by the Ministry of Science and Technology, under the grant 107-2511-H-003 -022 -MY3; 104-2511-S-003 -018 -MY3; 107-2511-H-992-001-MY3; 104-2511-S-151-001-MY3, and the University Sprout Project―Chinese Language and Technology Center of National Taiwan Normal University, sponsored by the Ministry of Education, Taiwan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Chang, TH., Sung, YT. (2019). Automated Chinese Essay Scoring Based on Multilevel Linguistic Features. In: Lu, X., Chen, B. (eds) Computational and Corpus Approaches to Chinese Language Learning. Chinese Language Learning Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-13-3570-9_13
Download citation
DOI: https://doi.org/10.1007/978-981-13-3570-9_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-3569-3
Online ISBN: 978-981-13-3570-9
eBook Packages: EducationEducation (R0)