Advertisement

How do students process complex formative feedback in question-answering tasks? A think-aloud study

  • Ignacio MáñezEmail author
  • Eduardo Vidal-Abarca
  • Panayiota Kendeou
  • Tomás Martínez
Article

Abstract

The goal of this study was to examine how students process formative feedback that included corrective and elaborative information in online question-answering tasks. Skilled and less-skilled comprehenders in grade 8 read texts and answered comprehension questions. Prior to responding, students were asked to select the textual information relevant to answer each question. Students received formative feedback that included information about the answer correctness and an elaborative message, either automatically delivered or optionally accessed. Students’ actions, as well as verbal protocols during feedback processing, were recorded. The results showed that: (a) students paid more attention to the answer correctness than to elaborative feedback messages; (b) students paid more attention to feedback information after failure than after success; (c) students actively monitored the accuracy of their responses; and (d) differences between skilled and less-skilled comprehenders in processing feedback were very limited. Theoretical and practical implications of these results are discussed.

Keywords

Formative feedback Metacognitive processes Question-answering Reading comprehension Think-aloud 

Notes

Acknowledgments

This research was supported by the projects FPU014/04646 and EST15/00492, granted by the Spanish Ministry of Education, Culture and Sport to Ignacio Máñez, and the projects EDU2014-55662-R, granted by the Spanish Ministry of Economy and Competitiveness, and PROMETEO/2013/011, granted by Conselleria d’Educació, Cultura i Esport de la Generalitat Valenciana to the Psicotext research group at the University of Valencia.

Compliance with ethical standards

Conflict of interest

The authors (Ignacio Máñez, Eduardo Vidal-Abarca, Panayiota Kendeou, Tomás Martínez) declare that they have no conflict of interest.

References

  1. Ackerman, R., & Goldsmith, M. (2011). Metacognitive regulation of text learning: On screen versus on paper. Journal of Experimental Psychology: Applied, 17(1), 18–32.  https://doi.org/10.1037/a0022086.Google Scholar
  2. Ackerman, R., & Lauterman, T. (2012). Taking reading comprehension exams on screen or on paper? A metacognitive analysis of learning texts under time pressure. Computers in Human Behavior, 28(5), 1816–1828.  https://doi.org/10.1016/j.chb.2012.04.023.CrossRefGoogle Scholar
  3. Anmarkrud, Ø., McCrudden, M. T., Bråten, I., & Strømsø, H. I. (2013). Task-oriented reading of multiple documents: Online comprehension processes and offline products. Instructional Science, 41(5), 873–894.  https://doi.org/10.1007/s11251-013-9263-8.CrossRefGoogle Scholar
  4. Bohn-Gettler, C. M., & Kendeou, P. (2014). The interplay of reader goals, working memory, and text structure during reading. Contemporary Educational Psychology, 39(3), 206–219.  https://doi.org/10.1016/j.cedpsych.2014.05.003.CrossRefGoogle Scholar
  5. Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), 209–222.  https://doi.org/10.1037/0022-0663.100.1.209.CrossRefGoogle Scholar
  6. Cerdán, R., Gilabert, R., & Vidal-Abarca, E. (2011). Selecting information to answer questions: Strategic individual differences when searching texts. Learning and Individual Differences, 21(2), 201–205.  https://doi.org/10.1016/j.lindif.2010.11.007.CrossRefGoogle Scholar
  7. Cerdán, R., Gilabert, R., & Vidal-Abarca, E. (2013). Self-generated explanations on the question demands are not always helpful. The Spanish Journal of Psychology, 16(e26), 1–11.  https://doi.org/10.1017/sjp.2013.45.Google Scholar
  8. Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018). Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review, 25, 23–38.  https://doi.org/10.1016/j.edurev.2018.09.003.CrossRefGoogle Scholar
  9. Dunlosky, J., & Metcalfe, J. (2009). Metacognition: A textbook for cognitive, educational, life span and applied psychology. Thousand Oaks: Sage.Google Scholar
  10. Farr, R., Pritchard, R., & Smitten, B. (1990). A description of what happens when an examinee takes a multiple-choice reading comprehension test. Journal of Educational Measurement, 27(3), 209–226.  https://doi.org/10.1111/j.1745-3984.1990.tb00744.x.CrossRefGoogle Scholar
  11. Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344.  https://doi.org/10.1037/a0021663.CrossRefGoogle Scholar
  12. Fox, J.-P., Klein Entink, R., & Timmers, C. (2014). The joint multivariate modeling of multiple mixed response sources: Relating student performances with feedback behavior. Multivariate Behavioral Research, 49(1), 54–66.  https://doi.org/10.1080/00273171.2013.843441.CrossRefGoogle Scholar
  13. Fyfe, E. R., Rittle-Johnson, B., & DeCaro, M. S. (2012). The effects of feedback during exploratory mathematics problem solving: Prior knowledge matters. Journal of Educational Psychology, 104(4), 1094–1108.  https://doi.org/10.1037/a0028389.CrossRefGoogle Scholar
  14. Graesser, A. C., & Murachver, T. (1985). Symbolic procedures of question answering. In A. C. Graesser & J. B. Black (Eds.), The psychology of questions (pp. 15–88). Hillsdale: Erlbaum.Google Scholar
  15. Hattie, J., & Gan, M. (2011). Instruction based on feedback. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 249–271). New York: Routledge.Google Scholar
  16. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.  https://doi.org/10.3102/003465430298487.CrossRefGoogle Scholar
  17. Jordan, S. (2012). Student engagement with assessment and feedback: Some lessons from short-answer free-text e-assessment questions. Computers & Education, 58(2), 818–834.  https://doi.org/10.1016/j.compedu.2011.10.007.CrossRefGoogle Scholar
  18. Kendeou, P., & O’Brien, E. J. (2016). Prior knowledge: Acquisition and revision. In P. Afflerbach (Ed.), Handbook of individual differences in reading: Reader, text and context (pp. 151–163). New York: Routledge.Google Scholar
  19. Kendeou, P., McMaster, K. L., & Christ, T. J. (2016). Reading comprehension: Core components and processes. Policy Insights From the Behavioral and Brain Sciences, 3(1), 62–69.  https://doi.org/10.1177/2372732215624707.CrossRefGoogle Scholar
  20. Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction, 19(2), 158–170.  https://doi.org/10.1016/j.learninstruc.2008.03.003.CrossRefGoogle Scholar
  21. Llorens, A. C., Gil, L., Vidal-Abarca, E., Martínez, T., Mañá, A., & Gilabert, R. (2011). Evaluación de la competencia lectora: la prueba de Competencia Lectora para Educación Secundaria (CompLEC). Psicothema, 23(4), 808–817.Google Scholar
  22. Llorens, A. C., Vidal-Abarca, E., Cerdán, R., & Ávila, V. (2015). Does formative feedback on search behavior help students in answering comprehension questions from an available text? Infancia y Aprendizaje, 38(4), 808–841.  https://doi.org/10.1080/02103702.2015.1076269.CrossRefGoogle Scholar
  23. Llorens, A. C., Vidal-Abarca, E., & Cerdán, R. (2016). Formative feedback to transfer self-regulation of task-oriented reading strategies. Journal of Computer Assisted Learning, 32(4), 314–331.  https://doi.org/10.1111/jcal.12134.CrossRefGoogle Scholar
  24. Magliano, J. P., & Millis, K. K. (2003). Assessing reading skill with a think-aloud procedure and latent semantic analysis. Cognition and Instruction, 21(3), 251–283.  https://doi.org/10.1207/S1532690XCI2103_02.CrossRefGoogle Scholar
  25. Maier, U., Wolf, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85–98.  https://doi.org/10.1016/j.compedu.2015.12.002.CrossRefGoogle Scholar
  26. Máñez, I., Vidal-Abarca, E., & Martínez, T. (2016, July). Accuracy to select relevant text information to answer questions from a text. Paper presented at the 26th annual meeting of the Society for Text and Discourse, Kassel, Germany.Google Scholar
  27. Martínez, T., Vidal-Abarca, E., Sellés, P., & Gilabert, R. (2008). Evaluation of comprehension strategies and processes: Test of comprehension processes (TCP). Infancia y Aprendizaje, 31(3), 319–332.  https://doi.org/10.1174/021037008785702956.CrossRefGoogle Scholar
  28. Martínez, T., Vidal-Abarca, E., Gil, L., & Gilabert, R. (2009). On-line assessment of comprehension processes. The Spanish Journal of Psychology, 12(1), 308–319.  https://doi.org/10.1017/S1138741600001700.CrossRefGoogle Scholar
  29. Moreno, R. (2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32, 99–113.  https://doi.org/10.1023/B:TRUC.0000021811.66966.1d.CrossRefGoogle Scholar
  30. Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 745–783). Mahwah: Erlbaum.Google Scholar
  31. Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merril, J. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 125–144). New York: Erlbaum.Google Scholar
  32. Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multi-media learning. In H. M. Niegemann, D. Leutner, & R. Brünken (Eds.), Instructional design for multimedia learning (pp. 181–195). Münster: Waxmann.Google Scholar
  33. Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., & Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education, 71, 56–76.  https://doi.org/10.1016/j.compedu.2013.09.011.CrossRefGoogle Scholar
  34. Ness, M. (2011). Explicit reading comprehension instruction in elementary classrooms: Teacher use of reading comprehension strategies. Journal of Research in Childhood Education, 25(1), 98–117.  https://doi.org/10.1080/02568543.2010.531076.CrossRefGoogle Scholar
  35. Pekrun, R., Frenzel, A. C., Goetz, T., & Perry, R. P. (2007). The control-value theory of achievement emotions: An integrative approach to emotions in education. In P. A. Schutz & R. Pekrun (Eds.), Emotion in education (pp. 13–36). San Diego: Academic Press.CrossRefGoogle Scholar
  36. Perfetti, C. A. (1985). Reading ability. New York: Oxford University Press.Google Scholar
  37. Pressley, M., Wharton-McDonald, R., Mistretta-Hampston, J., & Echevarria, M. (1998). Literacy instruction in 10 fourth-grade classrooms in upstate New York. Scientific Studies of Reading, 2(2), 159–194.  https://doi.org/10.1207/s1532799xssr0202_4.CrossRefGoogle Scholar
  38. Ramos, J. L., & Cuetos, F. (1999). PROLEC-SE: Evaluación de los procesos lectores en alumnos de tercer ciclo de educación primaria y secundaria. Madrid: TEA Ediciones.Google Scholar
  39. Ramos, L., & Vidal-Abarca, E. (2013). Differences between students with high and low reading literacy skills: A study with think aloud methodology. Cultura y Educación, 25(3), 295–308.  https://doi.org/10.1174/113564013807749722.CrossRefGoogle Scholar
  40. Roelle, J., & Berthold, K. (2017). Effects of incorporating retrieval into learning tasks: The complexity of the tasks matters. Learning and Instruction, 49(2), 142–156.  https://doi.org/10.1016/j.learninstruc.2017.01.008.CrossRefGoogle Scholar
  41. Rouet, J.-F., Britt, M. A., & Durik, A. M. (2017). RESOLV: Readers' representation of reading contexts and tasks. Educational Psychologist, 52(3), 200–215.  https://doi.org/10.1080/00461520.2017.1329015.CrossRefGoogle Scholar
  42. Rupp, A. A., Ferne, T., & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective. Language Testing, 23(4), 441–474.  https://doi.org/10.1191/0265532206lt337oa.CrossRefGoogle Scholar
  43. Salmerón, L., Strømsø, H. I., Kammerer, Y., Stadtler, M., & van den Broek, P. (2018). Comprehension processes in digital reading. In M. Barzillai, J. Thomson, S. Schroeder, & P. van den Broek (Eds.), Learning to read in a digital world (pp. 91–120). Amsterdam, The Netherlands: John Benjamins.CrossRefGoogle Scholar
  44. Schooler, J. W. (2011). Introspecting in the spirit of William James: Comment on Fox, Ericsson, and Best (2011). Psychological Bulletin, 137(2), 345–350.  https://doi.org/10.1037/a0022390.CrossRefGoogle Scholar
  45. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.  https://doi.org/10.3102/0034654307313795.CrossRefGoogle Scholar
  46. Timmers, C. F., & Veldkamp, B. P. (2011). Attention paid to feedback provided by a computer-based assessment for learning on information literacy. Computers & Education, 56(3), 923–930.  https://doi.org/10.1016/j.compedu.2010.11.007.CrossRefGoogle Scholar
  47. Timms, M., DeVelle, S., & Lay, D. (2016). Towards a model of how learners process feedback: A deeper look at learning. Australian Journal of Education, 60(2), 128–145.  https://doi.org/10.1177/0004944116652912.CrossRefGoogle Scholar
  48. van den Broek, P., Kendeou, P., & White, M. J. (2009). Cognitive processes during reading: Implications for the use of multimedia to foster reading comprehension. In A. G. Bus & S. B. Neuman (Eds.), Multimedia and literacy development: Improving achievement for young learners (pp. 57–73). New York: Routledge.Google Scholar
  49. Van der Kleij, F. M., Eggen, T. J. H. M., Timmers, C. F., & Veldkamp, B. P. (2012). Effects of feedback in a computer-based assessment for learning. Computers & Education, 58(1), 263–272.  https://doi.org/10.1016/j.compedu.2011.07.020.CrossRefGoogle Scholar
  50. Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511.  https://doi.org/10.3102/0034654314564881.CrossRefGoogle Scholar
  51. Veenman, M. V. J., Elshout, J. J., & Groen, M. G. M. (1993). Thinking aloud: Does it affect regulatory processes in learning? Tijdschrift voor Onderwijsresearch, 18(6), 322–330.Google Scholar
  52. Vidal-Abarca, E., Gilabert, R., & Abad, N. (2002a). A proposal for good expository text: Toward an expository text technology. Infancia y Aprendizaje, 25(4), 499–514.  https://doi.org/10.1174/021037002762064064.CrossRefGoogle Scholar
  53. Vidal-Abarca, E., Reyes, H., Gilabert, R., Calpe, J., Soria, E., & Graesser, A. C. (2002b). ETAT: Expository text analysis tool. Behavior Research Methods, Instruments, & Computers, 34(1), 93–107.  https://doi.org/10.3758/BF03195428.CrossRefGoogle Scholar
  54. Vidal-Abarca, E., Mañá, A., & Gil, L. (2010). Individual differences for self-regulating task-oriented reading activities. Journal of Educational Psychology, 102(4), 817–826.  https://doi.org/10.1037/a0020062.CrossRefGoogle Scholar
  55. Vidal-Abarca, E., Martínez, T., Salmerón, L., Cerdán, R., Gilabert, R., Gil, L., Mañá, A., Llorens, A. C., & Ferris, R. (2011). Recording online processes in task-oriented reading with Read&Answer. Behavior Research Methods, 43(1), 179–192.  https://doi.org/10.3758/s13428-010-0032-1.CrossRefGoogle Scholar
  56. Wolfe, M. B., & Goldman, S. R. (2005). Relations between adolescents' text processing and reasoning. Cognition and Instruction, 23(4), 467–502.  https://doi.org/10.1207/s1532690xci2304_2.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.ERI Lectura, Department of Developmental and Educational PsychologyUniversity of ValenciaValenciaSpain
  2. 2.Educational PsychologyUniversity of MinnesotaMinneapolisUSA

Personalised recommendations