Skip to main content
SpringerLink
Log in

Factors Affecting Measurement Error in Mobile Phone Interviews

  • Chapter
  • First Online:
Telephone Surveys in Europe

Introduction

Imagine a survey question that asks, “How many hours do you spend watching TV in a typical week?” A respondent may answer “16”. Whether that is the ‘correct’ answer depends on what exactly the researcher intended the question to measure. The respondent may have come up with his answer by thinking along the following lines: “The TV is on for about two hours each evening, so that’s 14 hours, plus I usually watch the sport for a couple of hours on Saturday afternoon, so that makes about 16 in total.” But maybe the researcher only had in mind time spent actually paying attention to the content of a TV broadcast, so having the TV turned on while devoting one’s attention to something different should not be counted. If, during the two hours each evening that the respondent’s TV is on, he is also eating dinner and talking to his children, then he may only be paying attention to the TV for half of that time. So, in the researcher’s terms the ‘correct’ answer would have been nine hours. There may be many people in the survey sample who, like this respondent, have included in their response to this question time that the TV is switched on even if they are not really paying attention to it. This will result in the researcher over-estimating the time people spend attentively watching TV. This error in estimation could have important consequences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • AAPOR (American Association of Public Opinion Research - Cell Phone Task Force) (2008). Guidelines and Considerations for Survey Researchers when Planning and Conducting RDD and Other Telephone Surveys in the U.S. with Respondents Reached via Cell Phone Numbers, http://www.aapor.org (accessed December 30, 2009)

  • AAPOR (American Association of Public Opinion Research - Cell Phone Task Force) (2010). New Considerations for Survey Researchers when Planning and Conducting RDD Telephone Surveys in the U.S. with Respondents Reached via Cell Phone Numbers, http://www.aapor.org (accessed January 24, 2011)

  • Aquilino, W.: Telephone versus face-to-face interviewing for household drug use surveys. International Journal of the Addictions 27, 71–91 (1992)

    Google Scholar 

  • Aquilino, W.: Interview mode effects in surveys of drug and alcohol use: a field experiment. Public Opinion Quarterly 58, 210–240 (1994)

    Article  Google Scholar 

  • Aquilino, W.: Effects of interview mode on measuring depression in younger adults. Journal of Official Statistics 14, 15–29 (1998)

    Google Scholar 

  • Bentler, P.M., Jackson, D.N., Messick, S.: Identification of Content and Style: a Two-Dimantional Interpretation of Aquiescence. Psychological Bulletin 76(3), 186–204 (1971)

    Article  Google Scholar 

  • Blair, J., Menon, G., Bickart, B.: Measurement Effects in Self vs. Proxy Responses to Survey Questions: An Information Processing Perspective. In: Biemer, P.P., et al. (eds.) Measurement Errors in Surveys, pp. 145–166. Wiley, New York (1991)

    Google Scholar 

  • Bradburn, N.M., Sudman, S.: The Current Status of Questionnaire Research. In: Biemer, P.P., et al. (eds.) Measurement Errors in Surveys, pp. 29–40. John Wiley & Sons, Hoboken (2004)

    Google Scholar 

  • Brick, J.M., Brick, P.D., Dipko, S., Presser, S., Tucker, C., Yuan, Y.: Cell Phone Survey Feasibility in the U.S.: Sampling and Calling Cell Numbers versus Landline Numbers. Public Opinion Quarterly 71, 23–29 (2007)

    Article  Google Scholar 

  • Cannell, C.F., Fowler, F.J.: Comparison of Self-enumerated Procedure and a Personal Interview: a Validity Study. Public Opinion Quarterly 27, 250–264 (1963)

    Article  Google Scholar 

  • Couper, M.P.: Measuring survey quality in a CASIC environment. In: Proceedings Survey Research Methods Section of American Statistical Association, pp. 41–49 (1998)

    Google Scholar 

  • Couper, M.P., Singer, E., Tourangeau, R.: Understanding the Effects of Audio-CASI on Self-Reports of Sensitive Behavior. Public Opinion Quarterly 67, 385–395 (2003)

    Article  Google Scholar 

  • Couper, M.P., Singer, E., Conrad, F.G., Groves, R.M.: Experimental Studies of Disclosure Risk, Disclosure Harm, Topic Sensitivity, and Survey Participation. Journal of Official Statistics 26, 287–300 (2010)

    Google Scholar 

  • Czaja, R., Blair, J.: Designing surveys: a guide to decisions and procedures, vol. 2. Pine Forge Press, Thousand Oaks (2005)

    Google Scholar 

  • DeMaio, T.J.: Social Desirability and Survey Measurement: A Review. In: Turner, C.F., Martin, E. (eds.) Surveying Subjective Phenomena, vol. 2, pp. 257–282. Russell sage Foundation, New York (1984)

    Google Scholar 

  • Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Mail and Mixed-Mode Surveys. Wiley, New York (2009)

    Google Scholar 

  • Dipko, S., Brick, P., Brick, J.M., Presser, S.: An investigation of response difference between cell phone and landline interviews. In: Paper presented at the Annual Conference of the American Association for Public Opinion Research, Miami Beach, Florida, May 12-15 (2005)

    Google Scholar 

  • Döring, N.: Psychological Aspects of Interviewing by Cellular Telephone. In: Maxl, E., et al. (eds.) Mobile Market Research, vol. 12, Herbert von Halem, Köln (2009)

    Google Scholar 

  • Fowler, F.J.: Improving Survey Questions: Design and Evaluation. Sage Publication, Inc. (1995)

    Google Scholar 

  • Gaskell, G., O’Muircheartaigh, C., Wright, D.: Survey questions about the frequency of vaguely defined events: the effect of response alternatives. Public Opinion Quarterly 58, 241–254 (1994)

    Article  Google Scholar 

  • Gfroerer, J.C., Hughes, A.L.: The Feasibility of Collecting Drug Abuse Data by Telephone. Public Health Reports 106, 384–393 (1991)

    Google Scholar 

  • Groves, R.M., Fowler Jr., F.J., Couper, M.P., Lepkowski, J.M., Singer, E., Tourangeau, R.: Survey Methodology, vol. 2. Wiley, Hoboken (2009)

    Google Scholar 

  • Holbrook, A.L., Green, M.C., Krosnick, J.A.: Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly 67, 79–125 (2003)

    Article  Google Scholar 

  • Holbrook, A.L., Krosnick, J.A., Moore, D., Tourangeau, R.: Response Order Effects in Dichotomous Categorical Questions Presented Orally. Public Opinion Quarterly 71, 325–348 (2007)

    Article  Google Scholar 

  • Huttenlocher, J., Hedges, L.V., Bradburn, N.M.: Reports of Elapsed Time: Bounding and Rounding Processes in Estimation. Journal of Experimental Psychology: Learning, Memory, and Cognition 16, 196–213 (1990)

    Article  Google Scholar 

  • Jobe, J.B., Pratt, W.F., Tourangeau, R., Baldwin, A., Rasinski, K.: Effects of Interview Mode on Sensitive Questions in a Fertility Survey. In: Lyberg, L., et al. (eds.) Survey Measurement and Process Quality. Wiley, New York (1997)

    Google Scholar 

  • Kaminska, O., McCutcheon, A.L., Billiet, J.B.: Satisficing among Reluctant Respondents in a Cross-National Context. Public Opinion Quarterly 74, 956–984 (2010)

    Article  Google Scholar 

  • Kellogg, R.T.: Fundamentals of Cognitive Psychology. Sage Publications, Los Angeles (2007)

    Google Scholar 

  • Kennedy, C.: Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys. Public Opinion Quarterly 71, 750–771 (2007)

    Article  Google Scholar 

  • Kennedy, C.: Use of cognitive shortcuts in landline and cell phone surveys. In: Paper presented at the Annual Conference of the American Association of Public Opinion Research, Chicago, Illinois (2010)

    Google Scholar 

  • Krosnick, J.A.: Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5, 213–236 (1991)

    Article  Google Scholar 

  • Krosnick, J.A., Alwin, D.F.: An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly 51, 201–219 (1987)

    Article  Google Scholar 

  • Krosnick, J.A., Narayan, S., Smith, W.R.: Satisficing in surveys: initial evidence. In: Braverman, M.T., Slater, J.K. (eds.) Advances in Survey Research, pp. 29–44. Jossey-Bass, San Francisco (1996)

    Google Scholar 

  • Lavrakas, P.J., Shuttles, C.D., Steeh, C., Fienberg, H.: The State of Surveying Cell Phone Numbers in the United States. Public Opinion Quarterly 71, 840–854 (2007)

    Article  Google Scholar 

  • Lavrakas, P.J., Tompson, T.N., Benford, R., Fleury, C.: Investigating Data Quality in Cell Phone Surveying. In: Paper presented at the 65th Annual Conference of the American Association for Public Opinion Research, Chicago (2010)

    Google Scholar 

  • Link, M.W., Mokdad, A.: Advance Letters as a Means of Improving Respondent Cooperation in Random Digit Dial Studies: A Multistate Experiment. Public Opinion Quarterly 69, 572–587 (2005)

    Article  Google Scholar 

  • Lynn, P., Kaminska, O.: The impact of mobile phones on survey measurement error. ISER Working Paper 2011-07 (2011)

    Google Scholar 

  • McClendon, M.J.: Acquiescence and Recency Response-Order Effects in Interview Surveys. Sociological Methods and Research 20(1), 60–103 (1991)

    Article  Google Scholar 

  • McCutcheon, A.L., Callegaro, M., Ludwig, J.: Who’s Calling? The Impact of Caller ID on Telephone Survey Response. Field Methods 22, 175–191 (2010)

    Article  Google Scholar 

  • Pew Research Center. The Cell Phone Challenge to Survey Research (2006), http://people-press.org/report/276/the-cell-phone-challenge-to-survey-research (accessed September 8, 2010)

  • Pudney, S.: Heaping and leaping: Survey response behavior and the dynamics of self-reported consumption expenditure. ISER Working Paper 2008-09 (2008), http://www.iser.essex.ac.uk/publications/working-papers/iser/2008-09

  • Saukshag, J.: An Examination of Mode Switch Nonresponse in a “Recruit-and-Switch” Survey. Presented at International Workshop on Household Survey Nonresponse. Nürnberg (August 2010)

    Google Scholar 

  • Schaeffer, N.C.: Hardly ever or constantly? Group comparisons using vague quantifiers. Public Opinion Quarterly 55, 395–423 (1991)

    Article  Google Scholar 

  • Schaeffer, N.C., Presser, S.: The Science of Asking Questions. Annual Review of Sociology 29, 65–88 (2003)

    Article  Google Scholar 

  • Schneiderat, G., Häder, S.: Anlage und Vorstudien des DFG-Projektes “Telefonbefragungen in der Allgemeinbevölkerung über das Mobilfunknetz” (System and preliminary studies of the DFG project. “telephone surveys in the general population over mobile phone“). In: Gabler, S., Häder, S. (eds.) Mobilfunktelefonie – Eine Herausforderung für die Umfrageforschung, Gesis, Mannheim (2007)

    Google Scholar 

  • Schwarz, N.: Cognition and Communication: Judgmental Biases, Research Methods, and the Logic of Conversation. Lawrence Erlbaum Associates, Inc., Publishers (1996)

    Google Scholar 

  • Schwarz, N., Strack, F., Hippler, H.-J., Bishop, G.: The impact of administration mode on response effects in survey measurement. Applied Cognitive Psychology 5, 193–212 (1991)

    Article  Google Scholar 

  • Schuman, H., Presser, S.: Questions and answers in attitude surveys: Experiments on question form, wording, and context. Academic Press, New York (1981)

    Google Scholar 

  • Smith, T.W.: An Experimental Comparison of Clustered and Scattered Scale Items. Social Psychology Quarterly 46, 163–168 (1983)

    Article  Google Scholar 

  • Sudman, S., Bradburn, N.: Effect of time and memory factors on response in surveys. Journal of the American Statistical Association 68, 805–815 (1973)

    Google Scholar 

  • Sudman, S., Bradburn, N.: Response Effects in Surveys. Aldine, Chicago (1974)

    Google Scholar 

  • Tourangeau, R., Rips, L., Rasinski, K.: The Psychology of Survey Response. Cambridge University Press, Cambridge (2000)

    Google Scholar 

  • Tourangeau, R., Smith, T.W.: Asking sensitive questions: The impact of data collection mode, question format and question context. Public Opinion Quarterly 60, 275–304 (1996)

    Article  Google Scholar 

  • Tourangeau, R., Steiger, D.M., Wilson, D.: Self-administered questions by telephone: evaluating interactive voice response. Public Opinion Quarterly 66, 265–278 (2002)

    Article  Google Scholar 

  • Van der Zouwen, J., Dijkstra, W., Smit, J.H.: Studying Respondent-Interviewer Interaction: the Relationship between Interviewing Style, Interviewer Behavior, and Response Behavior. In: Biemer, P.P., et al. (eds.) Measurement Errors in Surveys, pp. 419–437. John Wiley & Sons, Hoboken (2004)

    Google Scholar 

  • Vicente, P., Reis, E., Santos, M.: Using mobile phones for survey research: a comparison with fixed phones. International Journal of Market Research 51, 613–633 (2009)

    Article  Google Scholar 

  • Weeks, M.F., Kulka, R.A., Pierson, S.A.: Optimal call scheduling for a telephone survey. Public Opinion Quarterly 51, 540–549 (1987)

    Article  Google Scholar 

  • Willis, G.B.: The Use of the Psychological Laboratory to Study Sensitive Survey Topics. In: Harrison, L., Hughes, A. (eds.) The Validity of Self Reported Drug Use: Improving the Accuracy of Survey Estimates. NIDA Research Monograph 167, Washington, DC: Supt. of Docs., U.S. Govt. Printing Office (1997)

    Google Scholar 

  • Witt, L., ZuWallack, R., Conrey, F.: Out and About: An Evaluation of Data Quality in Cell Phone Surveys. In: Paper presented at the Annual Conference of the American Association of Public Opinion Research, Hollywood, FL (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Survey Methodology, University of Essex, Essex, UK

    Peter Lynn

  2. Survey Statistician, The University of Essex, Essex, UK

    Olena Kaminska

Authors
  1. Peter Lynn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olena Kaminska
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Methodology at GESIS, Center for Survey Design and, Postfach 12-21-55, Mannheim, 68072, Germany

    Sabine Häder

  2. , Philosophische Fakultät -, Technische Universität Dresden, Helmholtzstraße 10, Dresden, 01069, Sachsen, Germany

    Michael Häder

  3. Helmholtzstr. 10, Dresden, 01069, Germany

    Mike Kühne

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag GmbH Berlin Heidelberg

About this chapter

Cite this chapter

Lynn, P., Kaminska, O. (2011). Factors Affecting Measurement Error in Mobile Phone Interviews. In: Häder, S., Häder, M., Kühne, M. (eds) Telephone Surveys in Europe. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25411-6_14

Download citation

Publish with us

Policies and ethics

Search

Navigation