Abstract
This paper reports on an essay-writing study using a technical system that has been developed to generate automated feedback on academic essays. The system operates through the combination of a linguistic analysis engine, which processes the text in the essay, and a web application that uses the output of the linguistic analysis engine to generate the feedback. In this paper we focus on one particular visual representation produced by the system, namely “rainbow diagrams”. Using the concept of a reverse rainbow, diagrams are produced which visually represent how concepts are interlinked between the essay introduction (violet nodes) and conclusion (red nodes), and how concepts are linked and developed across the whole essay – thus a measure of how cohesive the essay is as a whole. Using a bank of rainbow diagrams produced from real essays, we rated the diagrams as belonging to high-, medium- or low-scoring essays according to their structure, and compared this rating to the actual marks awarded for the essays. On the basis of this we can conclude that a significant relationship exists between an essay’s rainbow diagram structure and the mark awarded. This finding has vast implications, as it is relatively easy to show users what the diagram for a “good” essay looks like. Users can then compare this to their own work before submission so that they can make necessary changes and so improve their essay’s structure, without concerns over plagiarism. Thus the system is a valuable tool that can be utilised across academic disciplines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alden Rivers, B., Whitelock, D., Richardson, J.T.E., Field, D., Pulman, S.: Functional, frustrating and full of potential: learners’ experiences of a prototype for automated essay feedback. In: Kalz, M., Ras, E. (eds.) CAA 2014. CCIS, vol. 439, pp. 40–52. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08657-6_4
Alexander, P.A., Schallert, D.L., Hare, V.C.: Coming to terms: how researchers in learning and literacy talk about knowledge. Rev. Educ. Res. 61, 315–343 (1991). https://doi.org/10.2307/1170635
Banyard, P., Underwood, J., Twiner, A.: Do enhanced communication technologies inhibit or facilitate self-regulated learning? Eur. J. Ed. 41, 473–489 (2006). https://doi.org/10.1111/j.1465-3435.2006.00277.x
Butler, D.L., Winne, P.H.: Feedback and self-regulated learning: a theoretical synthesis. Rev. Educ. Res. 65, 245–281 (1995). https://doi.org/10.3102/00346543065003245
Chickering, A.W., Gamson, Z.F.: Seven principles for good practice in undergraduate education. Am. Assoc. High. Educ. Bull. 39(7), 3–7 (1987). http://www.aahea.org/aahea/articles/sevenprinciples1987.htm. Accessed 23 Jun 2015
Cohen, J.: Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Academic Press, New York (1988). https://doi.org/10.1016/B978-0-12-179060-8.50001-3
Crabtree, J., Roberts, S.: Fat Pipes, Connected People: Rethinking Broadband Britain. The Work Foundation, London (2003). http://www.theworkfoundation.com/DownloadPublication/Report/121_121_fat_pipes.pdf. Accessed 23 Jun 2015
D’Mello, S.K., Picard, R., Graesser, A.C.: Toward an affect-sensitive AutoTutor. IEEE Intell. Syst. 22(4), 53–61 (2007). https://doi.org/10.1109/MIS.2007.79
Evans, C.: Making sense of assessment feedback in higher education. Rev. Educ. Res. 83, 70–120 (2013). https://doi.org/10.3102/0034654312474350
Gibbs, G., Simpson, C.: Conditions under which assessment supports students’ learning. Learn. Teach. High. Educ. 1, 1–31 (2004). http://insight.glos.ac.uk/tli/resources/lathe/documents/issue%201/articles/simpson.pdf. Accessed 23 Jun 2015
Greene, J.A., Azevedo, R.: The measurement of learners’ self-regulated cognitive and metacognitive processes while using computer-based learning environments. Educ. Psychol. 45, 203–209 (2010). https://doi.org/10.1080/00461520.2010.515935
Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77, 81–112 (2007). https://doi.org/10.3102/003465430298487
Ifenthaler, D.: Intelligent model-based feedback: helping students to monitor their individual learning progress. In: Graf, S., Lin, F., Kinshuk, McGreal, R. (eds.) Intelligent and Adaptive Learning Systems: Technology Enhanced Support for Learners and Teachers, pp. 88–100. IGI Global, Hershey (2011)
Ifenthaler, D., Pirnay-Dummer, P.: Model-based tools for knowledge assessment. In: Spector, J.M., Merrill, M.D., Elen, J., Bishop, M.J. (eds.) Handbook of Research on Educational Communications and Technology, 4th edn, pp. 289–301. Springer, New York (2014). https://doi.org/10.1007/978-1-4614-3185-5_23
Moos, D.C., Azevedo, R.: The role of goal structure in undergraduates’ use of self-regulatory variables in two hypermedia learning tasks. J. Educ. Multimed. Hypermed. 15, 49–86 (2006)
Newman, M.E.J.: Mathematics of networks. In: Durlauf, S.N., Blume, L.W. (eds.) The New Palgrave Dictionary of Economics, vol. 5, 2nd edn, pp. 465–470. Palgrave Macmillan, Houndmills (2008)
Price, M., Handley, K., Millar, J.: Feedback: focusing attention on engagement. Stud. High. Educ. 36, 879–896 (2011). https://doi.org/10.1080/03075079.2010.483513
Sadler, D.R.: Formative assessment and the design of instructional systems. Instr. Sci. 18, 119–144 (1989). https://doi.org/10.1007/BF00117714
Steffens, K.: Self-regulated learning in technology-enhanced learning environments: lessons from a European review. Eur. J. Ed. 41, 353–380 (2006). https://doi.org/10.1111/j.1465-3435.2006.00271.x
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D., Pulman, S.: OpenEssayist: a supply and demand learning analytics tool for drafting academic essays. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 208–212. ACM (2015)
Acknowledgements
This work is supported by the Engineering and Physical Sciences Research Council (EPSRC, grant numbers EP/J005959/1 & EP/J005231/1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix 1
1.1 Marking Criteria for Essays
Criterion | Definition | Maximum marks |
|---|---|---|
1. Introduction | Introductory paragraph sets out argument | 10 |
2. Conclusion | Concluding paragraph rounds off discussion | 10 |
3. Argument | Argument is clear and well followed through | 10 |
4. Evidence | Evidence for argument in main body of text | 20 |
5. Paragraphs | All paragraphs seven sentences long or less | 5 |
6. Within word count | Word count between 500 and 1000 words | 5 |
7. References | Two or three references | 5 |
Four or more references | 10 | |
8. Definition | Provides a clear and explicit definition of risk or memory | 10 |
9. Written presentation | Extensive vocabulary, accurate grammar and spelling | 10 |
10. Practical implications | Understanding of practical issues, innovative proposals | 10 |
Maximum total marks | 100 |
Appendix 2
2.1 Rating Criteria for Rainbow Diagrams
Low-scoring diagrams | Medium-scoring diagrams | High-scoring diagram |
|---|---|---|
Not densely connected | Densely connected area but some outlying nodes | Densely connected |
Red nodes (conclusion) not central | Red (conclusion) and violet (introduction) not so closely connected | Red nodes (conclusion) central |
Few links between violet (introduction) and red (conclusion) nodes | Close links between violet (introduction) and red (conclusion) nodes |
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Whitelock, D., Twiner, A., Richardson, J.T.E., Field, D., Pulman, S. (2018). What Does a ‘Good’ Essay Look Like? Rainbow Diagrams Representing Essay Quality. In: Ras, E., Guerrero Roldán, A. (eds) Technology Enhanced Assessment. TEA 2017. Communications in Computer and Information Science, vol 829. Springer, Cham. https://doi.org/10.1007/978-3-319-97807-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-97807-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97806-2
Online ISBN: 978-3-319-97807-9
eBook Packages: Computer ScienceComputer Science (R0)