A reply to “Do surveys accurately report voters over 80 years old?”: testing for bias in probability-based surveys of private households
- 5 Downloads
This article is a reply and extension to a paper published by Jean-Yves Dormagen and Laura Michel in this journal in early 2018. In their paper “Do surveys accurately report voters over 80 years old?”, the authors examined the deviations of aggregated self-reported turnout in three French electoral surveys from the factual turnout in the French presidential elections of 2002, 2007 and 2012. The tested election studies used a quota sampling method, which suffers most likely from selection bias. Indeed, the authors found a clear underrepresentation of the oldest population older than 85 years in all three studies and a strong overestimation of voting in this age cohort. In this article, I replicate their analysis with probability-based surveys and test whether the bias in turnout of the elderly also occurs in randomly drawn samples. For this purpose, I analysed data of three cross sections of the European Social Survey (ESS-1, ESS-4 and ESS-6). My results show that the probability-based surveys are indeed closer to true values derived from large-scale administrative data. The bias in turnout in the ESS data is nevertheless still stronger in the oldest age cohorts compared to all the other age cohorts. This could be the cause of a higher nonresponse among the elderly. Moreover, the ESS does not cover the institutionalised population. In France, more than 13% of the population older than 80 years are institutionalised and live in retirement and nursing homes. I assume that this part of the population has a lower probability to vote than the elderly living in private households. Their exclusion from the ESS therefore increases the distortion of self-reported voting behaviour compared to the factual turnout of old-aged residents.
KeywordsReplication Sampling methods Old respondents Institutionalised population French presidential elections
- CEVIPOF-Ministère de l’Intérieure. 2007. Enquête post électorale présidentielle 2007: Résultats d’ensemble.Google Scholar
- ESS. 2002. European social survey round 1: Data file edition 6.5, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
- ESS. 2008. European social survey round 4: Data file edition 4.4, NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
- ESS. 2012. European social survey round 6: Data file edition 2.3., NSD—Norwegian Centre for Research Data, Norway—Data Archive and distributor of ESS data for ESS ERIC.Google Scholar
- ESS. 2013. ESS6—2012 documentation report.Google Scholar
- ESS. 2014. Documentation of ESS post-stratification weights. https://www.europeansocialsurvey.org/docs/methodology/ESS1-5_post_stratification_weights_documentation.pdf. Accessed 14 Jan 2019.
- Eurostat. 2016. European statistical system—Census Hub. https://ec.europa.eu/CensusHub2. Accessed 14 Jan 2019.
- Groves, R., et al. 2009. Survey methodology, vol. 2. New York: Wiley.Google Scholar
- INSEE. 2012. L’inscription et la participation électorales en 2012: Qui est inscrit et qui vote. https://www.insee.fr/fr/statistiques/1281060. Accessed 14 Jan 2019.
- Jowell, R., and CCT. 2003. European social survey 2002/2003: Technical Report, London.Google Scholar
- Laferrère, A., A. Van Den Heede, and K. Van Den Bosch. 2012. Entry into institutional care: Predictors and alternatives. In Active ageing and solidarity between generations in Europe, ed. A. Börsch-Supan et al., 253–264. Berlin: de Gruyter.Google Scholar
- Lessler, J.T., and W.D. Kalsbeek. 1992. Nonsampling error in surveys. New York: Wiley.Google Scholar
- Levy, P., and S. Lemeshow. 2013. Sampling of populations: Methods and applications. New York: Wiley.Google Scholar
- Lohr, S. 2008. Coverage and Sampling. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 97–112. New York: European Association of Methodology.Google Scholar
- Lynn, P. 2008. The problem of nonresponse. In International handbook of survey methodology, ed. E. de Leeuw, J. Hox, and D. Dillman, 35–55. New York: European Association of Methodology.Google Scholar
- Pirou, D., N. Poullain, and S. Rochelle. 2013. La vie en communauté.Google Scholar
- Schanze, J.-L. 2017. Report on sampling practices for the institutionalized population in social. https://seriss.eu/resources/deliverables/. Accessed 14 Jan 2019.
- Voogt, R. 2004. I’m not interested—Nonresponse bias, response bias and stimulus effects in election research. Universiteit van Amsterdam.Google Scholar