Abstract
Holistic scoring of written texts is a most favored procedure to evaluate text quality in both the teaching and research of writing. However, the text properties that educators take into account to perform those evaluations have rarely been investigated. In this paper we examined the extent to which a series of linguistic markers obtained from written narrative texts contributed to explaining variation in the holistic scores assigned by independent raters. The written texts were produced by 80 participants divided into four age groups (9-, 12, 16-year olds, and adults), who were asked to write about the topic of a silent video showing conflicts at school. Linguistic markers were organized into three domains: syntactic complexity, cohesion, and vocabulary use. Our findings suggest that linguistic features are fundamental to perceptions of text quality in Spanish, though only a few text-based measures contributed significantly to the models for each age group. Educators took into account modality and genre constraints, and adjusted their criteria to the educational level of the writers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For the larger project of which text elicitation was part, participants also produced a spoken narrative on the same topic, as well as two other expository texts based on the theme of the video (spoken and written). The order of text production was counterbalanced for all age groups.
References
Aparici, M. (2010). El desarrollo de la conectividad discursiva en diferentes géneros y modalidades de producción. Doctoral dissertation, University of Barcelona.
Beaman, K. (1984). Coordination and subordination revisited: Syntactic complexity in spoken and written narrative discourse. Coherence in Spoken and Written Discourse, 12, 45–80.
Beers, S. F., & Nagy, W. (2011). Writing development in four genres from grades three to seven: Syntactic complexity and genre differentiation. Reading and Writing: An Interdisciplinary Journal, 24, 183–202.
Berman, R. (1997). Preschool knowledge of language: What five-year olds know about language structure and language use. In C. Pontecorvo (Ed.), Writing development: An interdisciplinary view (pp. 61–76). Amsterdam/Philadelphia: John Benjamins.
Berman, R. (2004). Between emergence and mastery: The long developmental route of language acquisition. In R. A. Berman (Ed.), Language development across childhood and adolescence (pp. 9–34). Amsterdam: John Benjamins.
Berman, R., & Nir, B. (2009). Cognitive and linguistic factors in evaluating text quality: Global versus local? In V. Evans & S. Pourcel (Eds.), New directions in cognitive linguistics (pp. 421–440). Amsterdam: John Benjamins.
Berman, R. A., & Slobin, D. I. (1994). Relating events in narrative: A crosslinguistic developmental study. Hillsdale: Lawrence Erlbaum Associates.
Berman, R. A., & Verhoeven, L. (2002). Cross-linguistic perspectives on the development of text-production abilities: Speech and writing. Written Language & Literacy, 5(1), 1–43.
Berninger, V. W., & Swanson, H. L. (1994). Modifying Hayes and Flower’s model of skilled writing to explain beginning and developing writing. Advances in Cognition and Educational Practice, 2, 57–81.
Berninger, V., Yates, C., Cartwright, A., Rutberg, J., Remy, E., & Abbott, R. (1992). Lower-level developmental skills in beginning writing. Reading and Writing, 4(3), 257–280.
Biber, D. (1995). Dimensions of register variation: A cross-linguistic comparison. Cambridge/New York: Cambridge University Press.
Blanche Benveniste, C. (1982). La escritura del lenguaje dominguero. In Ferreiro (Ed.), Nuevas perspectivas sobre los procesos de lectura y escritura (pp. 247–270). Buenos Aires: Siglo XXI.
Bourdin, B., & Fayol, M. (1994). Is written language production more difficult than oral language production? A working memory approach. International Journal of Psychology, 29(5), 591–620.
Bourdin, B., & Fayol, M. (2002). Even in adults, written production is still more costly than oral production. International Journal of Psychology, 37(4), 219–227.
Bybee, J. (2007). Review of Goldberg’s constructions at work: The nature of generalization in language. Journal of Child Language, 3, 14–16.
Charney, D. (1984). The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English, 18(1), 65–81.
Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing generating text in L1 and L2. Written Communication, 18(1), 80–98.
Dockrell, J. E., Lindsay, G., Connelly, V., & Mackie, C. (2007). Constraints in the production of written text in children with specific language impairments. Exceptional Children, 73(2), 147–164.
Dockrell, J. E., Connelly, V., Walter, K., & Critten, S. (2014). Assessing children’s writing products: The role of curriculum based measures. British Journal of Research in Education. doi:10.1002/berj.3162.
Freedman, S. W. (1979). How characteristics of student essays influence teachers’ evaluations. Journal of Educational Psychology, 71(3), 328–338.
Freedman, S. W., & Calfee, R. C. (1983). Holistic assessment of writing: Experimental design and cognitive theory. In P. Mosenthal, L. Amor, & S. A. Walmsley (Eds.), Research on writing: Principles and methods (pp. 75–98). New York: Longman.
Graham, S. (1990). The role of production factors in learning disabled students’ compositions. Journal of Educational Psychology, 82(4), 781.
Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Learning and Assessment, 8(6), 1–43.
Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29(3), 369–388.
Hunt, K. (1965). Grammatical structures written at three grade levels (Research Report No. 3). Illinois: National Council of Teachers of English.
Jarvis, S. (2002). Short texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19(1), 57–84.
Jones, D., & Christensen, C. A. (1999). Relationship between automaticity in handwriting and students’ ability to generate written text. Journal of Educational Psychology, 91(1), 44–49.
Karmiloff-Smith, A. (1986). From meta-processes to conscious access: Evidence from children’s metalinguistic and repair data. Cognition, 23(2), 95–147.
Karmiloff-Smith, A. (1992). Beyond modularity: A developmental perspective on cognitive science. Cambridge, MA: The MIT Press.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: The Guilford Press.
Lee, Y. W., Gentile, C., & Kantor, R. (2009). Toward automated multi-trait scoring of essays: Investigating links among holistic, analytic, and text feature scores. Applied Linguistics, 31(3), 391–417.
Llauradó, A., & Tolchinsky, L. (2013). The growth of the text-embedded lexicon in Catalan from childhood to adolescence. First Language, 33(6), 628–653.
MacArthur, C. A., & Graham, S. (1987). Learning disabled students’ composing under three methods of text production: Handwriting, word processing and dictation. The Journal of Special Education, 21(3), 22–42.
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk (3rd ed.). Mahwah: Lawrence Erlbaum Associates.
Malvern, D., Richards, B. J., Chipere, N., & Durán, P. (2004). Lexical diversity and language development. New York: Palgrave Macmillan.
McColly, W. (1970). What does educational research say about the judging of writing ability? Journal of Educational Research, 64, 147–156.
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27(1), 57–86.
Nir, B., & Berman, R. A. (2010). Complex syntax as a window on contrastive rhetoric. Journal of Pragmatics, 42(3), 744–765.
Nir-Sagiv, B. (2008). Clause packages as constructions in the development of narrative discourse. PhD dissertation, Tel Aviv University.
Olinghouse, N. G., & Wilson, J. (2013). The relationship between vocabulary and writing quality in three genres. Reading and Writing: An Interdisciplinary Journal, 26, 45–65.
Page, E. B. (1994). New computer grading of student prose, using modern concepts and software. Journal of Experimental Education, 62, 127–142.
Puranik, C. S., Lombardino, L. J., & Altmann, L. J. (2008). Assessing the microstructure of written language using a retelling paradigm. American Journal of Speech-Language Pathology, 17(2), 107–120.
Ravid, D., & Levie, R. (2010). Hebrew adjectives in later language text production. First Language, 30(1), 27–55.
Ravid, D., & Tolchinsky, L. (2002). Investigating literacy development and language acquisition. Reply to commentaries on ‘Developing linguistic literacy: A comprehensive model’. Journal of Child Language, 29(02), 489–494.
Scott, C. M. (2009). Language-based assessment of written expression. In G. A. Troia (Ed.), Instruction and assessment for struggling writers: Evidence-based practices (pp. 358–385). New York: Guilford Press.
Scott, C., & Stokes, S. L. (1995). Measures of syntax in school-age children and adolescents. Language, Speech, and Hearing Services in Schools, 26, 309–319.
Tolchinsky, L. (2004). The nature and scope of later language development. In R. A. Berman (Ed.), Language development across childhood and adolescence (Trends in language acquisition research 3) (pp. 233–247). Amsterdam: John Benjamins.
Tolchinsky, L., Aparici, M., & Salas, N. (2012, July 11–13). The development of syntactic complexity in Spanish narratives. Paper presented at the 13th international conference of the EARLI Special Interest Group on Writing. University of Porto, Porto.
Verhoeven, L., Aparici, M., Cahana-Amitay, D., van Hell, J., Kriz, S., & Viguié, A. (2002). Clause packaging in writing and speech: A cross-linguistic developmental study. Written Language and Literacy, 5, 135–162.
Wagner, R. K., Puranik, C. S., Foorman, B., Foster, E., Wilson, L. G., Tschinkel, E., & Kantor, P. T. (2011). Modeling the development of written language. Reading and Writing, 24(2), 203–220.
Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.
Acknowledgments
We are extremely grateful to our selfless text quality raters: Judith Aparici, Estela García Alcaraz, Xavier Sebastià, and Mercè Vidiella. We would also like to thank Montse Nofre for her help to obtain some of the measures.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Salas, N., Llauradó, A., Castillo, C., Taulé, M., Martí, M.A. (2016). Linguistic Correlates of Text Quality from Childhood to Adulthood. In: Perera, J., Aparici, M., Rosado, E., Salas, N. (eds) Written and Spoken Language Development across the Lifespan. Literacy Studies, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-319-21136-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-21136-7_18
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21135-0
Online ISBN: 978-3-319-21136-7
eBook Packages: Social SciencesSocial Sciences (R0)