Repeated Cross-Sectional Surveys Using FTF

Chapter

Abstract

When designing a survey two key problems have to be addressed, namely how to make sure that the final respondents represent the target population (or how to minimise non-observation errors) and how to make sure that the answers to survey questions represent the concepts under investigation (or how to minimise observation errors). This chapter discusses key factors in survey design, such as administration mode, coverage, sampling and non-response, the role of survey organisations and interviewers, questionnaire development, pretesting, fieldwork monitoring and archiving and dissemination. The main focus is on face-to-face surveys. In a cross-national survey there are additional considerations: the national context, variations in survey practice, the relevance of concepts in different countries, differences in the use of answer scales, translation of questions and, more generally, cross-national equivalence. This chapter provides a large number of examples, mainly drawn from the European Social Survey.

Keywords

Contact Rate European Social Survey Target Person Data Collection Mode Interviewer Effect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Anderson, A. H. (2008). Video-Mediated Interactions and Surveys. In F. G. Conrad & M. F. Schober (Eds.), Envisioning the survey interview of the future (pp. 95–118). Hoboken, NJ: Wiley.Google Scholar
  2. Behr, D., Kaczmirek, L., Bandilla, W., & Braun, M. (2011). Testing the validity of gender ideology items by implementing probing questions in web surveys. Paper presented at the 5th MESS Workshop, Oisterwijk, The Netherlands.Google Scholar
  3. Bell, M. W., Castronova, E., & Wagner, G. G. (2008). Virtual assisted self interviewing (VASI): An expansion of survey data collection methods to virtual worlds by means of VDCI. Berlin, University of Technology, DIW & Max Planck Institute for Human Development.Google Scholar
  4. Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  5. Conrad, F. G., Schober, M. F., & Nielsen, D. (2011). Race-of-Virtual-Interviewer Effects. Paper presented at the 5th MESS Workshop, Oisterwijk, The Netherlands.Google Scholar
  6. Conrad, F. G., Couper, M. P., Tourangeau, R., & Peytchev, A. (2006). Use and non-use of clarification features in web surveys. Journal of Official Statistics, 22(2), 245–269.Google Scholar
  7. Conrad, F. G., & Schober, M. F. (2000). Clarifying question meaning in a household telephone survey. Public Opinion Quarterly, 64, 1–28.CrossRefGoogle Scholar
  8. D’arrigo, J., Durrant, G. B., & Steele, F. (2009). Using field process data to predict best times of contact, conditioning on household and interviewer influences. Working Paper M09-12. University of Southampton.Google Scholar
  9. De Leeuw, E. D. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255.Google Scholar
  10. De Leeuw, E. D. (2008). Choosing the Method of Data Collection. In E. D. de Leeuw, J. J. Hox & D. A. Dillman (Eds.), International handbook of survey methodology (pp. 113–135). New York, NY: Taylor & Francis Group/Lawrence Erlbaum Associates.Google Scholar
  11. De Leeuw, E. D., Hox, J. J., & Dillman, D. A (Eds.). (2008a). International handbook of survey methodology. New York: Lawrence Erlbaum Associates.Google Scholar
  12. De Leeuw, E., Hox, J., & Scherpenzeel, A. (2011). Emulating interviewers in an online survey: experimental manipulation of ‘do-not-know’ over the phone and on the web. JSM Proceedings, Survey Research Methods Section (6305–6314). Alexandria, VA: American Statistical Association.Google Scholar
  13. Dean, E., Murphy, J., & Cook, S. (2009). Social presence in virtual world surveys. International Society for Presence Research. Proceedings of the 12th Annual International Workshop on Presence.Google Scholar
  14. Dept, S., Ferrari, A., & Wäyrynen, L. (2010). Developments in Translation Verification Procedures in Three Multilingual Assessments: a Plea for an Integrated Translation and Adaptation Monitoring Tool. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph Mohler (Eds.), Cross-cultural survey methods . (pp. 157–173). New York: Wiley.Google Scholar
  15. Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York: John Wiley Co.Google Scholar
  16. Dillman, D. A., Eltinge, J. L., Groves, R. M., & Little R. J. A. (2002). Survey nonresponse in design, data collection, and analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge & R. J. A. Little (Eds.), Survey nonresponse (pp. 3–26). New York: WileyGoogle Scholar
  17. ESF (1999) The European Social Survey (ESS):A research instrument for the social sciences in Europe. Report prepared for the Standing Committee for the Social Sciences (SCSS) of the European Science Foundation. Strasbourg, European Science Foundation.Google Scholar
  18. European Social Survey (2002) Core questionnaire development. www.europeansocialsurvey.org/index.php?option=com_content&view=article&id=62&Itemid=404 (downloaded 8 August, 2011).
  19. European Social Survey (2011a) ESS4-2008 Documentation Report, Edition 4.0, The ESS Archive, Norwegian Social Science Data Services, Bergen.Google Scholar
  20. European Social Survey (2011b) Round 6 specification for participating countries. Centre for Comparative Social Surveys, City University, London. Available at: www.europeansocialsurvey.org.
  21. Fowler, F. J., & Mangione, T. W. (1990). Standardized survey interviewing minimizing interviewer-related error. Newbury Park: Sage.Google Scholar
  22. Groves, R. M., (1989). Survey Errors and Survey Costs. New York: Wiley, Inc.Google Scholar
  23. Groves, R. M., Fowler, F.J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology, 2 nd ed. Hoboken, NJ: Wiley.Google Scholar
  24. Groves, R. M., & Couper, M. P. (1998). Nonresponse in household interview surveys. New York: Wiley.CrossRefGoogle Scholar
  25. Häder, S., & Lynn, P. (2007). How representative can a multi-nation survey be? In R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.), Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey (33–52). London: Sage.Google Scholar
  26. Hansen, S. -E., & Lee, H. J. (2010). Instrument Technical Design. In: Survey Research Center (2010) Guidelines for Best Practice in Cross-Cultural Surveys (Chapter IX). Ann Arbor, MI, Survey Research Center, Institute for Social Research, University of Michigan. Retrieved August, 8 2011, from www.ccsg.isr.umich.edu.
  27. Harkness, J. A. (2003). Questionnaire Translation. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph Mohler (Eds.), Cross-cultural survey methods (35–56). New York: Wiley.Google Scholar
  28. Harkness, J. A. (2007). Improving the comparability of translations. In R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.), Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey (79-93). London: Sage.Google Scholar
  29. Harkness, J. A., Braun, M., Edwards, B., Johnson, T. P., Lyberg, L., Mohler, P. Ph, et al. (Eds.). (2010a). Survey methods in multinational, multiregional, and multicultural contexts (33–58). Hoboken, NJ: Wiley.Google Scholar
  30. Harkness, J. A., Edwards, B., Hansen, S.-E., Millar, D. R., & Vilar, A. (2010b). Designing Questionnaires for Multipopulation Research. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. Lyberg, P. Ph Mohler, B.-E. Pennell, & T. W. Smith (Eds.), Survey methods in multinational, multiregional, and multicultural contexts (pp. 33–58). Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  31. Harkness, J., van de Vijver, F. J. R., & Johnson, T. (2003). Questionnaire Design in Comparative Research. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph Mohler (Eds.), Cross-cultural survey methods (19–34). New York: Wiley.Google Scholar
  32. Heerwegh, D., & Loosveldt, G. (2008). Face-to-face versus web surveying in a high-internet-coverage population. differences in response quality. Public Opinion Quarterly, 72, 836–846.CrossRefGoogle Scholar
  33. Holbrook, A. L., Green, M., & Krosnick, J. A. (2003). Telephone vs. face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly, 67, 79–125.CrossRefGoogle Scholar
  34. Jäckle, A., Roberts, C., & Lynn, P. (2010). Assessing the effect of data collection mode on measurement. International Statistical Review, 78(1), 3–20.CrossRefGoogle Scholar
  35. Jowell, R., Kaase, M., Fitzgerald, R., & Eva, G. (2007). The European social survey as a measurement model. In R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.), Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey (1–31). London: Sage.Google Scholar
  36. Kish, L. (1997). ‘Foreword’ In T. Lê and V. Verma, DHS analytical reports No. 3: an analysis of sample designs and sampling errors of the demographic and health surveys, Calverton, Maryland USA: Macro International Inc.Google Scholar
  37. Koch, A., Blom, A. G., Stoop, I., & Kappelhof, J. (2009). Data collection quality assurance in cross-national surveys: the example of the ESS. Methoden Daten Analysen. Zeitschrift für Empirische Sozialforschung, Jahrgang 3. Heft, 2, 219–247.Google Scholar
  38. Koch, A., Fitzgerald, R., Stoop, I., & Widdop, S. (2010). Field procedures in the European social survey round 5: enhancing response rates. Mannheim: European Social Survey, GESIS.Google Scholar
  39. Kolsrud, K., Kallgraf Skjåk, K., & Henrichsen, B. (2007). Free and immediate access to data. In R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.), Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey (139–156). London: Sage.Google Scholar
  40. Krosnick, J. A., & Presser, S. (2010). Question and Questionnaire Design. In P. Marsden & J. Wright (Eds.), Handbook of survey research (263–314). Emerald Group Publishing Ltd, 2nd Ed.Google Scholar
  41. Krosnick, J. A., Narayan, S. S., & Smith, W. R. (1996). Satisficing in surveys: Initial evidence. In M. T. Braverman & J. K. Slater (Eds.), Advances in survey research (pp. 29–44). San Francisco: Jossey-Bass.Google Scholar
  42. Lesser, J., Eyerman, J., & Wang, K. (2008a). Interviewer Training. In E. de Leeuw, J. Hox, & D. Dillman (Eds.), International handbook of survey methodology (pp. 442–460). New York: Lawrence Erlbaum Associates.Google Scholar
  43. Loosveldt, G. (2008a). Face-to-Face Interviews. In E. D de Leeuw, J. J. Hox, & D. A. Dillman (Eds.), International handbook of survey methodology (pp. 201–220). New York: Lawrence Erlbaum Associates.Google Scholar
  44. Malakhoff, L. A. & Jans, M. (2011). Toward usage of avatar interviewers in web surveys. Survey practice, June. www.surveypractice.org.
  45. Marsden P., & Wright, J. (Eds.). (2010). Handbook of survey research. Emerald Group Publishing Ltd, 2nd Ed.Google Scholar
  46. Peytchev, A., Conrad, F. G., Couper, M. P., & Tourangeau, R. (2010). Increasing respondents’ use of definitions in web surveys. Journal of Official Statistics, 26(4), 633–650.Google Scholar
  47. Purdon, S., Campanelli, P., & Sturgis, P. (1999). Interviewers’ calling strategies on face-to-face interview survey. Journal of Official Statistics, 15(2), 199–216.Google Scholar
  48. Roberts, C. (2007) Mixing modes of data collection in surveys: A methodological review. ESRC National Centre for Research Methods. NCRM Methods Review Papers NCRM/008.Google Scholar
  49. Saris, W. E., & Gallhofer, I. N. (2007a). Design, evaluation, and analysis of questionnaires for survey research wiley series in survey methodology. New York: Wiley.CrossRefGoogle Scholar
  50. Saris, W. E., & Gallhofer, I. N. (2007b). Can questions travel successfully?. In R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.), Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey (53–77). London: Sage.Google Scholar
  51. Schaeffer, N. C., Dykema, J., & Maynard, D. W. (2010). Interviewers and interviewing. In: P. Marsden & J. Wright (Eds.), Handbook of Survey Research (437–470). Emerald Group Publishing Ltd, 2nd Ed.Google Scholar
  52. Schober, M. F., & Conrad, F. G. (2008). Survey Interviews and New Communication Technologies. In F. G. Conrad & M. F. Schober (Eds.), Envisioning the survey interview of the future (pp. 1–30). Hoboken, NJ: Wiley.Google Scholar
  53. Schuman, H., & Presser Stanley, S. (1981). Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. Academic Press; 1996. Reprinted, with new Preface, New York: Sage.Google Scholar
  54. Smith, T. W. (1997). The impact of the presence of others in a respondent’s answers to questions. International Journal of Public Opinion Research, 9(1), 33–47.CrossRefGoogle Scholar
  55. Stoop, I. (2005). The hunt for the last respondent. The Hague: Social and Cultural Planning Office.Google Scholar
  56. Stoop, I. (2007). If it bleeds, it leads: the impact of media-reported events. In: R. Jowell, C. Roberts, R. Fitzgerald & G. Eva (Eds.),Measuring attitudes cross-nationallylessons from the European Social Survey (95–111). London: Sage.Google Scholar
  57. Stoop, I., Billiet, J., Koch, A., & Fitzgerald, R. (2010). Improving survey response. Lessons learned from the European social survey. Chichester:Wiley.Google Scholar
  58. Survey Research Center (2010) guidelines for best practice in cross-cultural surveys. Ann Arbor, MI, Survey Research Center, Institute for Social Research, University of Michigan. Retrieved August, 8 2011, from www.ccsg.isr.umich.edu.
  59. West, B., & Olson, K. (2010). How much of interviewer variance is really nonresponse error variance. Public Opinion Quarterly, 74(5), 1004–1026.CrossRefGoogle Scholar
  60. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.Google Scholar

Further reading: Survey Methodology and Quality

  1. Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  2. De Leeuw, E. D., Hox, J. J., & Dillman, D. A. (Eds.). (2008). International handbook of survey methodology. New York: Lawrence Erlbaum Associates.Google Scholar
  3. Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey Methodology, 2 nd ed. Hoboken, NJ: Wiley.Google Scholar
  4. Marsden P., & Wright, J. (Eds.). (2010). Handbook of survey research. Emerald Group Publishing Ltd, 2nd Ed.Google Scholar

Questionnaire design

  1. Harkness, J., van de Vijver, F. J. R., & Johnson, T. (2003). Questionnaire design in comparative research. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph Mohler (Eds.), Cross-cultural survey methods (19–34). New York: Wiley.Google Scholar
  2. Harkness, J. A., Edwards, B., Hansen, S. E., Millar, D. R., & Vilar, A. (2010). Designing questionnaires for multi population research. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, L. Lyberg, P. Ph Mohler, B.-E. Pennell, & T. W. Smith (Eds.), Survey methods in multinational, multiregional, and multicultural contexts (pp. 33–58). Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  3. Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research. New York, Wiley: Wiley Series in Survey Methodology.CrossRefGoogle Scholar

Interviewers

  1. Lesser, J., Eyerman, J., & Wang, K. (2008). Interviewer training. In E. de Leeuw, J. Hox, & D. Dillman (Eds.), International handbook of survey methodology (pp. 442–460). New York: Lawrence Erlbaum Associates.Google Scholar
  2. Loosveldt, G. (2008). Face-to-face interviews. In E. de Leeuw, J. Hox, & D. Dillman (Eds.), International handbook of survey methodology (pp. 201–220). New York: Lawrence Erlbaum Associates.Google Scholar
  3. Morton-Williams, J. (1993). Interviewer approaches. Aldershot: Dartmouth Publishing.Google Scholar
  4. Schaeffer, N. C., Dykema, J., & Maynard, D. W. (2010). Interviewers and interviewing. In P. Marsden & J. Wright (Eds.), Handbook of survey research (437–470). Emerald Group Publishing Ltd, 2nd Ed.Google Scholar

Cross-national surveys

  1. Harkness, J. A., van de Vijver, F. J. R., & Mohler, P. Ph. (Eds.). (2003). Cross-cultural survey methods. New York: Wiley.Google Scholar
  2. Harkness, J. A., Braun, M., Edwards, B., Johnson, T. P., Lyberg, L., Mohler, P. Ph (Eds.). (2010). Survey methods in multinational, multiregional, and multicultural contexts (33–58). Hoboken, NJ: Wiley.Google Scholar
  3. Survey Research Center (2010) Guidelines for best practice in cross-cultural surveys. Ann Arbor, MI, Survey Research Center, Institute for Social Research, University of Michigan. Retrieved August, 8 2011, from www.ccsg.isr.umich.edu.

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.The Netherlands Institute for Social Research/SCPThe HagueThe Netherlands
  2. 2.Centre for Comparative Social SurveysCity University, Northampton SquareLondonUK

Personalised recommendations