Advertisement

Metacognition and Learning

, Volume 14, Issue 2, pp 131–166 | Cite as

Calibration in multiple text use

  • Ying Wang
  • Alexandra ListEmail author
Article
  • 36 Downloads

Abstract

The literature on calibration suggests that students consider a multitude of factors when they self-evaluate task performance. Nevertheless, few studies have focused on calibration within a complex task enviornment, such as when students are asked to compose written responses based on multiple texts. In this study, we examined the criteria that undergraduate students considered when they were asked to self-evaluate their written responses, composed based on multiple texts. Moreover, we considered the extent to which these criteria had an effect on students' objective response quality, calibration, and confidence bias. Findings revealed that students indeed cited a variety of criteria in justifying their self-evaluations including task-, context-, and person-related factors, consistent with prior research. Further, our study indicated that high quality written responses were associated with accurate calibration and with students' relative under-confidence. We further found that low-performing students demonstrated less accurate calibration and greater over-confidence. Implications for improving students’ metacognitive awareness during complex task completion are discussed.

Keywords

Self-evaluation Calibration Confidence bias Writing composition Multiple texts 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest and that there is no funding to declare in association with this project.

References

  1. Afflerbach, P., & Cho, B. Y. (2009). Determining and describing reading strategies: Internet and traditional forms of reading. In H. S. Waters & W. Schneider (Eds.), Metacognition, strategy use, and instruction (pp. 201–225). New York: Guilford.Google Scholar
  2. Allwood, C. M., Granhag, P. A., & Jonsson, A. C. (2006). Child witnesses’ metamemory realism. Scandinavian Journal of Psychology, 47(6), 461–470.CrossRefGoogle Scholar
  3. Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multile conflicting documents. Learning and Individual Differences, 30, 64–76  https://doi.org/10.1016/j.lindif.2013.01.007.CrossRefGoogle Scholar
  4. Baker, L. (1989). Metacognition, comprehension monitoring, and the adult reader. Educational Psychology Review, 1(1), 3–38.CrossRefGoogle Scholar
  5. Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147.CrossRefGoogle Scholar
  6. Bol, L., & Hacker, D. J. (2001). A comparison of the effects of practice tests and traditional review on performance and calibration. The Journal of Experimental Education, 69(2), 133–151.  https://doi.org/10.1080/00220970109600653.CrossRefGoogle Scholar
  7. Bol, L., Hacker, D. J., O’Shea, P., & Allen, D. (2005). The influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. The Journal of Experimental Education, 73(4), 269–290.CrossRefGoogle Scholar
  8. Bol, L., Hacker, D. J., Walck, C. C., & Nunnery, J. A. (2012). The effects of individual or group guidelines on the calibration accuracy and achievement of high school biology students. Contemporary Educational Psychology, 37(4), 280–287.  https://doi.org/10.1016/j.cedpsych.2012.02.004.CrossRefGoogle Scholar
  9. Boud, D., Lawson, R., & Thompson, D. (2013). Does students engagement in self-assessment calibrate their judgement over time? Assessment and Evaluation in Higher Education, 38(8), 941–956.CrossRefGoogle Scholar
  10. Bråten, I., & Strømsø, H. I. (2009). Effects of task instruction and personal epistemology on the understanding of multiple texts about climate change. Discourse Processes, 47(1), 1–31.  https://doi.org/10.1080/01638530902959646.CrossRefGoogle Scholar
  11. Bråten, I., & Strømsø, H. I. (2011). Measuring strategic processing when students read multiple texts. Metacognition and Learning, 6(2), 111–130.CrossRefGoogle Scholar
  12. Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485–522.  https://doi.org/10.1207/S1532690XCI2004_2.CrossRefGoogle Scholar
  13. Britt, M. A., & Rouet, J. F. (2012). Learning with multiple documents: Component skills and their acquisition. In J. R. Kirby & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 276–314). New York: Cambridge University Press.CrossRefGoogle Scholar
  14. Britt, M. A., & Sommer, J. (2004). Facilitating textual integration with macro-structure focusing task. Reading Psychology, 25, 313–339.CrossRefGoogle Scholar
  15. Britt, M. A., Perfetti, C. A., Sandak, R. L., & Rouet, J. F. (1999). Content integration and source separation in learning from multiple texts. In S. R. Goldman, A. C. Graesser, & P. van den Broek (Eds.), Narrative comprehension, causality, and coherence: Essays in honor of tom Trabasso (pp. 209–233). Mahwah: Erlbaum.Google Scholar
  16. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281.CrossRefGoogle Scholar
  17. Cavaleri, M., & Dianati, S. (2016). You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students. Journal of Academic Language and Learning, 10(1), A22–A236.Google Scholar
  18. Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), 209–222.  https://doi.org/10.1037/0022-0663.100.1.209.CrossRefGoogle Scholar
  19. Chiu, M. M., & Klassen, R. M. (2010). Relations of mathematics self-concept and its calibration with mathematics achievement: Cultural differences among fifteen-year-olds in 34 countries. Learning and Instruction, 20, 2–17.CrossRefGoogle Scholar
  20. Cho, B. -Y., & Afflerbach, P. (2017). An evolving perspective of constructively responsive reading comprehension strategies in multilayered digital text environments. In S.E. Israel (Ed.), Handbook of research on reading comprehension (2nd ed., pp. 109–134). New York, NY: Guilford.Google Scholar
  21. Dinsmore, D. L., & Parkinson, M. M. (2013). What are confidence judgements made of? Students’ explanations for their confidence ratings and what that means for calibration. Learning and Instruction, 24, 4–14.  https://doi.org/10.1016/j.learninstruc.2012.06.001.CrossRefGoogle Scholar
  22. Dole, J. A., Duffy, G. G., Roehler, L. R., & Pearson, P. D. (1991). Moving from the old to new: Research on reading comprehension instruction. Review of Educational Research, 61(2), 239–264.CrossRefGoogle Scholar
  23. Du, H. & List, A. (under review). Writing based on multiple texts.Google Scholar
  24. Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction, 22(4), 271–280.CrossRefGoogle Scholar
  25. Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–86.CrossRefGoogle Scholar
  26. Fallahi, C. R., Wood, R. M., Austad, C. S., & Fallahi, H. (2006). A program for improving undergraduate psychology students’ basic writing skills. Teaching of Psychology, 33(3), 171–175.CrossRefGoogle Scholar
  27. Firetto, C. M. (forthcoming). Learning from multiple complementary perspectives: a systematic review. In: Van Meter, P., List, A., Kendeou, P., & Lombardi, D. (Eds.), Handbook of learning from multiple representations and multiple perspectives. New York: Routledge.Google Scholar
  28. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911.CrossRefGoogle Scholar
  29. Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010). Summary versus argument tasks when working with multiple documents: Which is better whom? Contemporary Educational Psychology, 35(3), 157–173.  https://doi.org/10.1016/j.cedpsych.2009.11.002.CrossRefGoogle Scholar
  30. Glenberg, A. M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11(4), 702–718.  https://doi.org/10.1037/0278-7393.11.1-4.702.CrossRefGoogle Scholar
  31. Glenberg, A. M., & Epstein, W. (1987). Inexpert calibration of comprehension. Memory & Cognition, 15(1), 84–93.CrossRefGoogle Scholar
  32. Glenberg, A. M., Sanoki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental Psychology: General, 116(2), 119–136.CrossRefGoogle Scholar
  33. Goldman, S. R., & Scardamalia, M. (2013). Managing, understanding, applying, and creating knowledge in the information age: Next-generation challenges and opportunities. Cognition and Instruction, 31(2), 255–269.  https://doi.org/10.1080/10824669.2013.773217.CrossRefGoogle Scholar
  34. Graham, S., Harris, K. R., & Mason, L. (2005). Improving the writing performance, knowledge, and self-efficacy of struggling young writers: The effects of self-regulated strategy development. Contemporary Educational Psychology, 30(2), 207–241.  https://doi.org/10.1016/j.cedpsych.2004.08.001.CrossRefGoogle Scholar
  35. Griffin, T. D., Jee, B. D., & Wiley, J. (2009). The effects of domain knowledge on metacomprehension accuracy. Memory & Cognition, 37(7), 1001–1013.  https://doi.org/10.3758/MC.37.7.1001.CrossRefGoogle Scholar
  36. Griffin, T. D., Wiley, J., & Salas, C. R. (2013). Supporting effective self-regulated learning: The critical role of monitoring. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 19–34). New York: Springer.CrossRefGoogle Scholar
  37. Hacker, D. J., Bol, L., Horgan, D. D., & Rakow, E. A. (2000). Test prediction and performance in a classroom context. Journal of Educational Psychology, 92(1), 160–170.  https://doi.org/10.1037/0022-0663.92.1.160.CrossRefGoogle Scholar
  38. Hacker, D. J., Bol, L., & Keener, M. C. (2008). Metacognition in education: A focus on calibration. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp. 429–456). New York: Taylor & Francis.Google Scholar
  39. Hadwin, A. F., & Webster, E. A. (2013). Calibration in goal setting: Examining the nature of judgements of confidence. Learning and Instruction, 24, 37–47.  https://doi.org/10.1016/j.learninstruc.2012.10.001.CrossRefGoogle Scholar
  40. Higham, P. A. (2013). Regulating accuracy on university tests with plurality option. Learning and Instruction, 24, 26–36.  https://doi.org/10.1016/j.learninstruc.2012.08.001.CrossRefGoogle Scholar
  41. Huff, J. D., & Nietfeld, J. L. (2009). Using strategy instruction and confidence judgements to improve metacognitive monitoring. Metacognition and Learning, 4, 161–176.  https://doi.org/10.1007/s11409-009-9042-8.CrossRefGoogle Scholar
  42. Keren, G. (1991). Calibration and probability judgements: Conceptual and methodological issues. Acta Psychologica, 77, 217–273.CrossRefGoogle Scholar
  43. Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling of knowing. Psychological Review, 100(4), 609–639.CrossRefGoogle Scholar
  44. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.CrossRefGoogle Scholar
  45. Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1(4), 279–308.CrossRefGoogle Scholar
  46. Labuhn, A. S., Zimmerman, B. J., & Hasselhorn, M. (2010). Enhancing students’ self-regulation and mathematics performance: The influence of feedback and self-evaluative standards. Metacognition and Learning, 5, 173–194.  https://doi.org/10.1007/s11409-010-9056-2.CrossRefGoogle Scholar
  47. Le Bigot, L., & Rouet, J. F. (2007). The impact of presentation format, task assignment, and prior knowledge on students’ comprehension of multiple online documents. Journal of Literacy Research, 39(4), 445–470.CrossRefGoogle Scholar
  48. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159–183.CrossRefGoogle Scholar
  49. Lin, L. M., & Zabrucky, K. M. (1998). Calibration of comprehension: Research and implication for education and instruction. Contemporary Educational Psychology, 23(4), 345–391.  https://doi.org/10.1006/ceps.1998.0972.CrossRefGoogle Scholar
  50. Lin, L. M., Moore, D., & Zabrucky, K. M. (2001). An assessment of students’ calibration of comprehension and calibration of performance using multiple measures. Reading Psychology, 22, 111–128.CrossRefGoogle Scholar
  51. List, A., & Alexander, P. (2015). Examining response confidence in multiple text tasks. Metacognition and Learning, 10, 407–436.  https://doi.org/10.1007/s11409-015-9138-2.CrossRefGoogle Scholar
  52. List, A., & Alexander, P. (2017a). Analyzing and integrating models of multiple text comprehension. Educational Psychologist, 52(3), 143–147.  https://doi.org/10.1080/00461520.2017.1328309.CrossRefGoogle Scholar
  53. List, A., & Alexander, P. A. (2017b). Text navigation in multiple source use. Computers in Human Behavior, 75, 364–375.Google Scholar
  54. List, A., Alexander, P. A., & Stephens, L. A. (2017). Trust but verify: Examining the association between students' sourcing behaviors and ratings of text trustworthiness. Discourse Processes, 54(2), 83–104.  https://doi.org/10.1080/0163853X.2016.1174654 CrossRefGoogle Scholar
  55. List, A. (2019). Six questions regarding strategy use when learning from multiple texts. In: D.L. Dinsmore, L.K. Fryer, & M.M. Parkinson (Eds.). Handbook of strategies and strategic processing: conceptualization, intervention, measurement, and analysis. New York: Routledge (In Press).Google Scholar
  56. List, A., & Alexander, P.A. (2019) Toward an integrated framework of multiple text use, Educational Psychologist, 54(1), 20–39.  https://doi.org/10.1080/00461520.2018.1505514.CrossRefGoogle Scholar
  57. List A., Du, H., & Wang, Y. (2019a). Understanding students' conceptions of task assignments. Contemporary Educational Psychology, 59, 1–16.  https://doi.org/10.1016/j.cedpsych.2019.101801.CrossRefGoogle Scholar
  58. List, A., Du, H., Wang, Y., & Lee, H. Y. (2019b). Toward a typology of integration: Examining the documents model framework. Contemporary Educational Psychology, 58, 228–242.  https://doi.org/10.1016/j.cedpsych.2019.03.003.CrossRefGoogle Scholar
  59. Mateos, M., & Solé, I. (2009). Synthesising information from various texts: A study of procedures and products at different educational levels. European Journal of Psychology of Education, 24, 435–451.CrossRefGoogle Scholar
  60. Miller, T. M., & Geraci, L. (2011). Unskilled but aware: Reinterpreting overconfidence in low-performing students. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(2), 502–506.Google Scholar
  61. Mosenthal, P. B. (1998). Defining prose task characteristics for use in computer-adaptive testing and instruction. American Educational Research Journal, 35(2), 269–307.CrossRefGoogle Scholar
  62. Nietfeld, J. L., & Schraw, G. (2002). The effect of knowledge and strategy training on monitoring accuracy. The Journal of Educational Research, 95(3), 131–142.  https://doi.org/10.1080/00220670209596583.CrossRefGoogle Scholar
  63. Nietfeld, J. L., Cao, L., & Osborne, J. W. (2006a). The effects of distributed monitoring exercises and feedback on performance, monitoring accuracy, and self-efficacy. Metacognition and Learning, 1, 159–179.CrossRefGoogle Scholar
  64. Nietfeld, J. L., Enders, C. K., & Schraw, G. (2006b). A Monte Carlo comparison of measures of relative and absolute monitoring accuracy. Educational and Psychological Measurement, 66(2), 258–271.CrossRefGoogle Scholar
  65. Perfetti, C. A., Rouet, J. F., & Britt, M. A. (1999). Towards a theory of documents representation. In H. van Oostendorp & S. R. Goldman (Eds.), The construction of mental representations during reading (pp. 99–122). Hillsdale, NJ: Erlbaum.Google Scholar
  66. Pierce, B. H., & Smith, S. M. (2001). The postdiction superiority effect in metacomprehension of text. Memory & Cognition, 29(1), 62–67.CrossRefGoogle Scholar
  67. Pieschl, S. (2009). Metacognitive calibration–an extended conceptualization and potential applications. Metacognition and Learning, 4(1), 3–31.  https://doi.org/10.1007/s11409-008-9030-4.CrossRefGoogle Scholar
  68. Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40.CrossRefGoogle Scholar
  69. Pressley, M., & Ghatala, E. S. (1990). Self-regulated learning: Monitoring learning from text. Educational Psychologist, 25(1), 19–33.CrossRefGoogle Scholar
  70. Ramdass, D., & Zimmerman, B. J. (2008). Effects of self-correction strategy training on middle school students' self-efficacy, self-evaluation, and mathematics division learning. Journal of Advanced Academics, 20(1), 18–41.CrossRefGoogle Scholar
  71. Reznitskaya, A., Kuo, L., Glina, M., & Anderson, R. C. (2009). Measuring argumentative reasoning: What’s behind the numbers? Learning and Individual Differences, 19, 219–224.  https://doi.org/10.1016/j.lindif.2008.11.001.CrossRefGoogle Scholar
  72. Rouet, J. F. (2006). The skills of document use: From text comprehension to web-based learning. Mahwah, NJ: Erlbaum.Google Scholar
  73. Rouet, J. F., & Britt, M. A. (2011). Relevance processes in multiple document comprehension. In M. T. McGrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (pp. 19–52). Charlotte, NC: Information Age.Google Scholar
  74. Rouet, J. F., Britt, M. A., & Durik, A. M. (2017). RESOLV: Readers’ representation of reading contexts and tasks. Educational Psychologist, 52(3), 200–215.  https://doi.org/10.1080/00461520.2017.1329015.CrossRefGoogle Scholar
  75. Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113–125.CrossRefGoogle Scholar
  76. Schraw, G., & Nietfeld, J. (1998). A further test of the general monitoring skill hypothesis. Journal of Educational Psychology, 90(2), 236–248.CrossRefGoogle Scholar
  77. Schraw, G., Dunkle, M. E., Bendixen, L. D., & Roedel, T. D. (1995). Does a general monitoring skill exist? Journal of Educational Psychology, 87(3), 433–444.CrossRefGoogle Scholar
  78. Schraw, G., Kuch, F., & Gutierrez, A. P. (2013). Measure for measure: Calibration ten commonly used calibration scores. Learning and Instruction, 24, 48–57.CrossRefGoogle Scholar
  79. Snyder, K. E., Nietfeld, J. L., & Linnenbrink-Garcia, L. (2011). Giftedness and metacognition: A short-term longitudinal investigation of metacognitive monitoring in the classroom. Gifted Child Quarterly, 55(3), 181–193.  https://doi.org/10.1177/0016986211412769.CrossRefGoogle Scholar
  80. Sperling, R. A., Howard, B. C., Staley, R., & DuBois, N. (2004). Metacognition and self-regulated learning constructs. Educational Research and Evaluation, 10(2), 117–139.CrossRefGoogle Scholar
  81. Spivey, N. N., & King, J. R. (1989). Readers as writers composing from sources. Reading Research Quarterly, 24, 7–26.CrossRefGoogle Scholar
  82. Stahl, E., Pieschl, S., & Bromme, R. (2006). Task complexity, epistemological beliefs and metacognitive calibration: An exploratory study. Journal of Educational Computing Research, 35(4), 319–338.CrossRefGoogle Scholar
  83. Stone, N. J. (2000). Exploring the relationship between calibration and self-regulated learning. Educational Psychology Review, 12(4), 437–475.CrossRefGoogle Scholar
  84. Stone, E. R., & Opel, R. B. (2000). Training to improve calibration and discrimination: The effects of performance of environmental feedback. Organizational Behavior and Human Decision Processes, 83(2), 282–309.CrossRefGoogle Scholar
  85. Strømsø, H. I., Bråten, I., Britt, M. A., & Ferguson, L. E. (2013). Spontaneous sourcing among students reading multiple documents. Cognition and Instruction, 31(2), 176–203.  https://doi.org/10.1080/07370008.2013.769994.CrossRefGoogle Scholar
  86. Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66–73.CrossRefGoogle Scholar
  87. Wallace, R., Pearman, C., Hail, C., & Hurst, B. (2007). Writing for comprehension. Reading Horizons, 48(1), 41–56.Google Scholar
  88. Wiley, J., & Voss, J. F. (1996). The effects of ‘playing historian’ on learning in history. Applied Cognitive Psychology, 10, 63–72.CrossRefGoogle Scholar
  89. Wiley, J., & Voss, J. F. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91(2), 301–311.CrossRefGoogle Scholar
  90. Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning internet science inquire tasks. American Educational Research Journal, 46(4), 1060–1106.CrossRefGoogle Scholar
  91. Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (2nd ed., pp. 153–189). Mahwah: Erlbaum.Google Scholar
  92. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 279–306). Mahwah: Erlbaum.Google Scholar
  93. Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572.  https://doi.org/10.1016/S0361-476X(02)00006-1.CrossRefGoogle Scholar
  94. Winne, P., & Perry, N. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). San Diego, CA: Academic Press.CrossRefGoogle Scholar
  95. Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents’ text processing and reasoning. Cognition and Instruction, 23(4), 467–502.  https://doi.org/10.1207/s1532690xci2304_2.CrossRefGoogle Scholar
  96. Yates, J. F. (1990). Judgement and decision making. Englewood Cliffs: Prentice-Hall.Google Scholar
  97. Zabrucky, K. M., Agler, L. L., & Moore, D. (2009). Metacognition in Taiwan: Students’ calibration of comprehension and performance. International Journal of Psychology, 44(4), 305–312.  https://doi.org/10.1080/00207590802315409.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Educational Psychology, Counseling, and Special EducationThe Pennsylvania State UniversityState CollegeUSA

Personalised recommendations