Advertisement

Introspective Verbal Reports: Think-Alouds and Stimulated Recall

  • Melissa A. Bowles

Abstract

Verbal reports (think-alouds and stimulated recalls) have been used in first and second language research for decades by researchers of various theoretical orientations to gather data about learners’ thought processes. Despite their widespread use, there has been controversy surrounding their validity, which hinges on (1) whether verbalizing alters thought processes (reactivity) and (2) whether verbalizations are an accurate reflection of thoughts (veridicality). This chapter provides an overview of the use of verbal reports and synthesizes research that has examined their validity, finding them to be valid, if implemented appropriately. The chapter includes a concise guide to the proper use of verbal reports in language research, from data collection to analysis, concluding with a discussion of the method’s limitations and possible triangulation with other data sources.

Keywords

Stimulated recall Think-aloud Introspection Reactivity Veridicality 

References

  1. Barkaoui, K. (2011). Think-aloud protocols in research on essay rating: An empirical study of their veridicality and reactivity. Language Testing, 28(1), 51–75.CrossRefGoogle Scholar
  2. Bloom, B. (1953). Thought-processes in lectures and discussions. Journal of General Education, 7(3), 160–169.Google Scholar
  3. Bowles, M. A. (2010). The think-aloud controversy in second language research. New York: Routledge.CrossRefGoogle Scholar
  4. Burgoyne, J., & Hodgson, V. (1983). Natural learning and managerial action: A phenomenological study in the field setting. Journal of Management Studies, 20(3), 387–399.CrossRefGoogle Scholar
  5. Calderhead, J. (1981). A psychological approach to research on teachers’ classroom decision making. British Educational Research Journal, 7(1), 51–57.CrossRefGoogle Scholar
  6. Cohen, A. (1996). Verbal reports as a source of insights into second language learner strategies. Applied Language Learning, 7(1 & 2), 5–24.Google Scholar
  7. Cohen, A. (1998). Strategies in learning and using a second language. London: Longman.Google Scholar
  8. Corder, S. P. (1973). The elicitation of interlanguage. In J. Svartvik (Ed.), Errata. Papers in error analysis (pp. 36–48). Lund: CKW Geerup.Google Scholar
  9. Daly, W. (2001). The development of an alternative method in the assessment of critical thinking as an outcome of nursing education. Journal of Advanced Nursing, 36(1), 120–130.CrossRefGoogle Scholar
  10. Egi, T. (2007). Recasts, learners’ interpretations, and L2 development. In A. Mackey (Ed.), Conversational interaction in second language acquisition: A series of empirical studies (pp. 249–267). Oxford: Oxford University Press.Google Scholar
  11. Egi, T. (2008). Investigating stimulated recall as a cognitive measure: Reactivity and verbal reports in SLA research methodology. Language Awareness, 17(3), 212–228.CrossRefGoogle Scholar
  12. Elstein, A., Shulman, L., & Sprafka, S. (1978). Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
  13. Ericsson, K., & Simon, H. (1993). Protocol analysis: Verbal reports as data (2nd ed.). Cambridge, MA: MIT Press.Google Scholar
  14. Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344.CrossRefGoogle Scholar
  15. Fox-Turnbull, W. (2009, August). Stimulated recall using autophotography: A method for investigating technology education. In A. Bekker, I. Mottier, & M. J. de Vries (Eds.), Strengthening the position of technology education in the curriculum. Proceedings PATT-22 Conference (pp. 204–217). Delft, The Netherlands: International Technology and Engineering Educators Association.Google Scholar
  16. Fujii, A., & Mackey, A. (2009). Interactional feedback in learner-learner interactions in a task-based EFL classroom. International Journal of Applied Linguistics, 47(3-4), 267–301.Google Scholar
  17. Gass, S. M., & Mackey, A. (2016). Stimulated recall methodology in applied linguistics and L2 research (2nd ed.). New York: Routledge.CrossRefGoogle Scholar
  18. Gilbert, W., Trudel, P., & Haughian, L. (1999). Interactive decision making factors considered by coaches of youth ice hockey during games. Journal of Teaching in Physical Education, 18(3), 290–311.CrossRefGoogle Scholar
  19. Godfroid, A., & Schmidtke, J. (2013). What do eye movements tell us about awareness? A triangulation of eye-movement data, verbal reports and vocabulary learning scores. In J. M. Bergsleithner, S. N. Frota, & J. K. Yoshioka (Eds.), Noticing and second language acquisition: Studies in honor of Richard Schmidt (pp. 183–205). Honolulu, HI: University of Hawai’i, National Foreign Language Resource Center.Google Scholar
  20. Godfroid, A., & Spino, L. (2015). Under the radar: Triangulating think-alouds and finger tracking to detect the unnoticed. In A. Mackey & E. Marsden (Eds.), Advancing methodology and practice: The IRIS repository of instruments for research into second languages (pp. 73–90). New York: Routledge.Google Scholar
  21. Goo, J. (2010). Working memory and reactivity. Language Learning, 60(4), 712–752.CrossRefGoogle Scholar
  22. Hama, M., & Leow, R. P. (2010). Learning without awareness revisited: Extending Williams (2005). Studies in Second Language Acquisition, 32(3), 465–491.CrossRefGoogle Scholar
  23. Jourdenais, R. (2001). Cognition, instruction, and protocol analysis. In P. Robinson (Ed.), Cognition and second language instruction (pp. 354–375). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  24. Kagan, N., Krathwohl, D., & Miller, R. (1963). Stimulated recall in therapy using video tape: a case study. Journal of Counseling Psychology, 10, 237–243.CrossRefGoogle Scholar
  25. Krauskopf, C. J. (1963). Use of written responses in the stimulated recall method. Journal of Educational Psychology, 54(3), 172–176.CrossRefGoogle Scholar
  26. Kressel, K., Henderson, T., Reich, W., & Cohen, C. (2012). Multidimensional analysis of conflict mediator style. Conflict Resolution Quarterly, 30(2), 135–171.CrossRefGoogle Scholar
  27. Kuusela, H., & Paul, P. (2000). A comparison of concurrent and retrospective verbal protocol analysis. The American Journal of Psychology, 113(3), 387–404.CrossRefGoogle Scholar
  28. Leow, R. P., Grey, S., Marijuan, S., & Moorman, C. (2014). Concurrent data elicitation procedures, processes, and the early stages of L2 learning: A critical overview. Second Language Research, 30(2), 111–127.CrossRefGoogle Scholar
  29. Mackey, A., Gass, S. M., & McDonough, K. (2000). How do learners perceive interactional feedback? Studies in Second Language Acquisition, 22(4), 471–497.CrossRefGoogle Scholar
  30. Mackey, A., & Marsden, E. (Eds.). (2016). Instruments for research into second language. New York: Taylor and Francis.Google Scholar
  31. Nassaji, H. (2003). L2 vocabulary learning from context: Strategies, knowledge sources, and their relationship with success in L2 lexical inferencing. TESOL Quarterly, 37(4), 645–670.CrossRefGoogle Scholar
  32. Nisbett, R., & Wilson, T. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.CrossRefGoogle Scholar
  33. Payne, J. W., Braunstein, M. L., & Carroll, J. S. (1978). Exploring predecisional behavior: An alternative approach to decision research. Organizational Behavior and Human Performance, 22(1), 17–44.CrossRefGoogle Scholar
  34. Peterson, P., & Clark, C. (1978). Teachers’ reports of their cognitive processes during teaching. American Educational Research Journal, 15(4), 555–565.CrossRefGoogle Scholar
  35. Poehner, M. (2007). Beyond the test: L2 dynamic assessment and the transcendence of mediated learning. Modern Language Journal, 91(3), 323–340.CrossRefGoogle Scholar
  36. Qi, D., & Lapkin, S. (2001). Exploring the role of noticing in a three-stage second language writing task. Journal of Second Language Writing, 10(4), 277–303.CrossRefGoogle Scholar
  37. Robinson, K. M. (2001). The validity of verbal reports in children’s subtraction. Journal of Educational Psychology, 93(1), 211–222.CrossRefGoogle Scholar
  38. Sachs, R., & Polio, C. (2007). Learners’ uses of two types of written feedback on an L2 writing revision task. Studies in Second Language Acquisition, 29(1), 67–100.CrossRefGoogle Scholar
  39. Selinker, L. (1972). Interlanguage. International Review of Applied Linguistics, 10, 209–231.CrossRefGoogle Scholar
  40. Siegel, L., Siegel, L., Capretta, P., Jones, R., & Berkowitz, H. (1963). Students’ thoughts during class: A criterion for educational research. Journal of Educational Psychology, 54(1), 45–51.CrossRefGoogle Scholar
  41. Smagorinsky, P. (Ed.). (1994). Speaking about writing: Reflections on research methodology. Thousand Oaks, CA: SAGE.Google Scholar
  42. Smagorinsky, P. (2001). Rethinking protocol analysis from a cultural perspective. Annual Review of Applied Linguistics, 21, 233–245.CrossRefGoogle Scholar
  43. Smith, B. (2012). Eye tracking as a measure of noticing: A study of explicit recasts in SCMC. Language Learning and Technology, 16(3), 53–81.Google Scholar
  44. Swain, M. (2006). Languaging, agency and collaboration in advanced language proficiency. In H. Byrnes (Ed.), Advanced language learning: The contribution of Halliday and Vygotsky (pp. 95–108). London: Continuum.Google Scholar
  45. Tjeerdsma, B. (1997). A comparison of teacher and student perspectives of tasks and feedback. Journal of Teaching in Physical Education, 16(4), 388–400.CrossRefGoogle Scholar
  46. Upton, T. A., & Lee-Thompson, L. (2001). The role of the first language in second language reading. Studies in Second Language Acquisition, 23(4), 469–495.Google Scholar
  47. Vygotsky, L. (1987). The collected works of L. S. Vygotsky (Vol. I). New York: Plenum.Google Scholar
  48. Walters, F. S. (2007). A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing, 24(2), 155–183.CrossRefGoogle Scholar
  49. Wei, J., & Llosa, L. (2015). Investigating differences between American and Indian raters in assessing TOEFL iBT speaking tasks. Language Assessment Quarterly, 12(3), 283–304.CrossRefGoogle Scholar
  50. Winikoff, A. (1967). Eye movements as an aid to protocol analysis of problem solving behavior. Pittsburgh, PA.Google Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.University of IllinoisUrbanaUSA

Personalised recommendations