Skip to main content
SpringerLink
Log in
Book cover

Qualitätssicherung sozialwissenschaftlicher Erhebungsinstrumente pp 195–224Cite as

Die Ergänzung kognitiver Interviews um Eye Tracking

Ein Methodenvergleich

  • Chapter
  • First Online:

  • 4277 Accesses

Zusammenfassung

Umfrageforschern stehen eine Vielzahl von Methoden zur Verfügung, um Survey-Fragen zu evaluieren. Dabei stellt sich die Frage, welche Methoden am effektivsten sind, um mögliche Probleme in Erhebungsinstrumenten festzustellen. Als Beitrag zu dieser zentralen Frage wurde in einem Methodenvergleich untersucht, ob eine Ergänzung kognitiver Interviews um Eye Tracking wirksamer ist beim Identifizieren von Problemen in Fragen als die Methode des kognitiven Interviews allein. Verglichen werden die Gesamtanzahl der gefundenen Probleme und die Anzahl der Fragen, die als fehlerhaft identifiziert wurden. Die Ergebnisse zeigen, dass sich die beiden Methoden Eye Tracking und kognitives Interview effektiv ergänzen. Durch den Einsatz der hybriden Methode wurden im Vergleich zum alleinigen Einsatz des kognitiven Interviews mehr Probleme aufgedeckt und mehr Fragen als problematisch identifiziert.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   44.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Literatur

  • Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287-311. https://doi.org/10.1093/poq/nfm006

    Article  Google Scholar 

  • Blair, J., & Conrad, F.G. (2011). Sample size for cognitive interview pretesting. Public Opinion Quarterly, 75(4), 636-658. https://doi.org/10.1093/poq/nfr035

    Article  Google Scholar 

  • Blair, J., & Srinath, K. P. (2008). A note on sample size for behavior coding pretests. Field Methods, 20(1), 85-95. https://doi.org/10.1177/1525822x07303601

    Article  Google Scholar 

  • Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment 6(4), 284–290.

    Article  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.

    Article  Google Scholar 

  • Collins, D. (2003). Pretesting survey instruments: an overview of cognitive methods. Quality of Life Research, 12(3), 229-238.

    Article  Google Scholar 

  • Conrad, F. G., & Blair, J. (2001). Interpreting verbal reports in cognitive interviews: Probes matter. Proceedings of the Section on Survey Research Methods, American Statistical Association.

    Google Scholar 

  • Conrad, F. G., & Blair, J. (2004). Data quality in cognitive interviews: the case of verbal reports. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (67-87). New York: Wiley. https://doi.org/10.1002/0471654728.ch4

  • Conrad, F. G., & Blair, J. (2009). Sources of error in cognitive interviews. Public Opinion Quarterly, 73(1), 32-55.

    Article  Google Scholar 

  • Conrad, F. G., Blair, J. & Tracy, E. (1999). Verbal reports are data! A theoretical approach to cognitive interviews. In Proceedings of the Federal Committee on Statistical Methodology Research Conference. Arlington, VA, pp. 11-20.

    Google Scholar 

  • DeMaio, T. J., & Landreth, A. (2004). Do different cognitive interview techniques produce different results? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (89-108). New York: Wiley. https://doi.org/10.1002/0471654728.ch5

  • Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behaviour Research Methods, 39, 175-191.

    Article  Google Scholar 

  • Forsyth, B. H., & Lessler, J. T. (1991). Cognitive laboratory methods: a taxonomy. In P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (Eds.), Measurement Errors in Surveys (393-418). New York: Wiley.

    Google Scholar 

  • Fowler, F. J. (1992). How unclear terms affect survey data. Public Opinion Quarterly 56(2), 218-31. https://doi.org/10.1086/269312

  • Fowler, F. J. (1995). Improving survey questions. Design and evaluations. Thousand Oaks: Sage.

    Google Scholar 

  • Fowler, F. J., & Cannell, C. F. (1996). Using behavioral coding to identify cognitive problems with survey questions. In N. Schwarz & S. Schuman (Eds.), Answering questions. Methodology for determining cognitive and communicative processes in survey research (15-36). San Francisco: Jossey-Bass.

    Google Scholar 

  • Galesic, M., Tourangeau, R., Couper, M. P., & Conrad, F.G. (2008). Eye-Tracking data: New insights on response order effects and other cognitive shortcuts in survey responding. Public Opinion Quarterly, 72(5), 892-913. https://doi.org/10.1093/poq/nfn059

    Article  Google Scholar 

  • Galesic, M., & Yan, T. (2011). Use of eye tracking for studying survey response processes. In M. Das, P. Ester, & L. Kaczmirek (Eds.), Social and behavioral research and the internet: Advances in applied methods and research strategies (349-370). New York: Routledge.

    Google Scholar 

  • Graesser, A. C., Cai, Z., Louwerse, M. M., & Daniel, F. (2006). Question understanding aid (QUAID). A web facility that tests question comprehensibility. Public Opinion Quarterly, 70(1), 3–22. https://doi.org/10.1093/poq/nfj012

    Article  Google Scholar 

  • Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey Methodology. Hoboken, NJ: John Wiley & Sons.

    Google Scholar 

  • Holmqvist, K., Holsanova, J., Barthelson, M., & Lundqvist, D. (2003). Reading or scanning? A study of newspaper and net paper reading. In J. Hyöna, R. Radach, & H. Deubel (Eds.), The mind’s eye. Cognitive and applied aspects of eye movement research (657-670). Amsterdam: North-Holland.

    Chapter  Google Scholar 

  • Just, M. A., & Carpenter, P.A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.

    Article  Google Scholar 

  • Landis, J. R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  Google Scholar 

  • Lenzner, T., Kaczmirek, L., & Galesic, M. (2011). Seeing through the eyes of the respondent: An eye-tracking study on survey question comprehension. International Journal of Public Opinion Research, 23(3), 361-73. https://doi.org/10.1093/ijpor/edq053

    Article  Google Scholar 

  • Lenzner, T., Kaczmirek, L., & Galesic, M. (2014). Left feels right: A usability study on the position of answer boxes in web surveys. Social Science Computer Review, 32(6), 743-764. https://doi.org/10.1177/0894439313517532

    Article  Google Scholar 

  • Menold, N., Kaczmirek, L., Lenzner, T., & Neusar, A. (2014). How do respondents attend to verbal labels in rating scales? Field Methods, 26(1), 21-39. https://doi.org/10.1177/1525822x13508270

    Article  Google Scholar 

  • Neuert, C. E., & Lenzner, T. (2016). Incorporating eye tracking into cognitive interviewing to pretest survey questions. International Journal of Social Research Methodology, 19(5), 501-519.

    Article  Google Scholar 

  • Miller, K. (2011). Cognitive interviewing. In J. Madans, K. Miller, A. Maitland, G. Willis (Eds.), Question evaluation methods: Contributing to the science of data quality (51-75). New York: Wiley.

    Google Scholar 

  • Presser, S., & Blair, J. (1994). Survey pretesting: Do different methods produce different results. Sociological methodology, 24(1), 73-104.

    Article  Google Scholar 

  • Presser, S., Couper, M. P, Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130. https://doi.org/10.1093/poq/nfh008

    Article  Google Scholar 

  • Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422.

    Article  Google Scholar 

  • Redline, C. D., & Lankford, C. P. (2001, May). Eye-movement analysis: a new tool for evaluating the design of visually administered instruments (paper and web). Paper prepared for presentation at the annual meeting of the American Association for Public Opinion Research, Montreal.

    Google Scholar 

  • Romano, J. C., & Chen, J. M. (2011). A usability and eye-tracking evaluation of four versions of the online national survey of college graduates (NSCG): Iteration 2. Study Series: Survey Methodology 2011-01, Washington D.C.: U.S. Census Bureau.

    Google Scholar 

  • Tourangeau, R. (1984). Cognitive science and survey methods. In T.B. Jabine, M. L. Straf, J. M. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (73-100). Washington, DC: National Academy Press.

    Google Scholar 

  • Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.

    Google Scholar 

  • Tries, S. (2010). Usability tests of online questionnaires. In Federal Statistical Office (Ed.), Methods, Approaches, Developments: Information of the German Federal Statistical Office (5-8). Wiesbaden: Federal Statistical Office.

    Google Scholar 

  • Van den Haak, M., De Jong, M., & Schellens, P. J. (2003). Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behaviour & Information Technology, 22(5), 339-351. https://doi.org/10.1080/0044929031000

    Article  Google Scholar 

  • Willis, G. B. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (23-44). New York: Wiley. https://doi.org/10.1002/0471654728.ch2

  • Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. London: Sage.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mannheim, Deutschland

    Cornelia E. Neuert & Timo Lenzner

Authors
  1. Cornelia E. Neuert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timo Lenzner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cornelia E. Neuert .

Editor information

Editors and Affiliations

  1. GESIS – Leibniz-Institut für Sozialwissenschaften, Mannheim, Germany

    Natalja Menold

  2. FAU Erlangen-Nürnberg, Nürnberg, Germany

    Tobias Wolbring

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Neuert, C.E., Lenzner, T. (2019). Die Ergänzung kognitiver Interviews um Eye Tracking. In: Menold, N., Wolbring, T. (eds) Qualitätssicherung sozialwissenschaftlicher Erhebungsinstrumente. Schriftenreihe der ASI - Arbeitsgemeinschaft Sozialwissenschaftlicher Institute. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-24517-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-658-24517-7_7

  • Published:

  • Publisher Name: Springer VS, Wiesbaden

  • Print ISBN: 978-3-658-24516-0

  • Online ISBN: 978-3-658-24517-7

  • eBook Packages: Social Science and Law (German Language)

Publish with us

Policies and ethics

Search

Navigation