Advertisement

Survey mode effects on measured income inequality

  • Pirmin Fessler
  • Maximilian Kasy
  • Peter Lindner
Article

Abstract

We study the effect of interview modes on estimates of economic inequality which are based on survey data. We exploit variation in interview modes in the Austrian EU-SILC panel, where between 2007 and 2008 the interview mode was switched from personal interviews to telephone interviews for some but not all participants. We combine methods from the program evaluation literature with methods from the distributional decomposition literature to obtain causal estimates of the effect of interview mode on estimated inequality. We find that the interview mode has a large effect on estimated inequality, where telephone interviews lead to a larger downward bias. The effect of the mode is much smaller for robust inequality measures such as interquantile ranges, as these are not sensitive to the tails of the distribution. The magnitude of effects we find are of a similar order as the differences in many international and intertemporal comparisons of inequality.

Keywords

Income inequality Survey methodology Survey modes Distributional decompositions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

We would like to thank Markus Knell and Alyssa Schneebaum for valuable comments and discussion. Additional to the usual disclaimer, the opinions expressed in this work are those of the authors and do not necessarily reflect the ones of the OeNB or the Eurosystem.

Supplementary material

10888_2018_9378_MOESM1_ESM.pdf (300 kb)
(PDF 300 KB)

References

  1. Angrist, J., Pischke, J.-S.: Mostly Harmless Econometrics. Princeton University Press, Princeton (2008)Google Scholar
  2. Atkeson, L.R., Adams, A.N., Alvarez, R.M.: Nonresponse and Mode Effects in Self- and Interviewer-Administered Surveys. Political Analysis (2014)Google Scholar
  3. Cowell, F.A., Victoria-Feser, M.-P.: Robustness properties of inequality measures. Econometrica 64(1), 77–101 (1996)CrossRefGoogle Scholar
  4. de Leeuw, E.D.: Data Quality in Mail, Telephone and Face to Face Surveys. TT-Publikaties, Amsterdam (1992)Google Scholar
  5. Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method. Wiley, New York (2014)Google Scholar
  6. DiNardo, J., Fortin, N., Lemieux, T.: Labor market institutions and the distribution of wages, 1973-1992: A semiparametric approach. Econometrica 64, 1001–1044 (1996)CrossRefGoogle Scholar
  7. Firpo, S., Fortin, N.M., Lemieux, T.: Unconditional quantile regression. Econometrica 77(3), 953–973 (2009)CrossRefGoogle Scholar
  8. Huber, P.J.: Robust Statistics. Wiley Series in Probability and Statistics. Wiley, New York (2003)Google Scholar
  9. Iacus, S.M., King, G., Porro, G.: Matching for Causal Inference without Balance Checking. Working paper series, Harvard (2008)Google Scholar
  10. Imbens, G.W.: Nonparametric estimation of average treatment effects under exogeneity: a review. Rev. Econ. Stat. 86(1), 4–29 (2004)CrossRefGoogle Scholar
  11. Imbens, G.W., Rubin, D.B.: Causal Inference fo Statistics, Social and Biomedical Sciences an Introduction. Cambridge University Press, Cambridge (2015)CrossRefGoogle Scholar
  12. Jäckle, A., Roberts, C., Lynn, P.: Assessing the effect of data collection mode on measurement. Int. Stat. Rev. 78(1), 3–20 (2010)CrossRefGoogle Scholar
  13. Klausch, T., Hox, J.J., Schouten, B.: Assessing the Mode-Dependency of Sample Selectivity Across the Survey Response Process. Technical Report 2013-03, Statistics Netherlands (2013a)Google Scholar
  14. Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013b)CrossRefGoogle Scholar
  15. Klausch, T., Hox, J.J., Schouten, B.: Evaluating bias of sequential mixed-mode designs against benchmark surveys. Sociol. Methods Res. 46(3), 456–489 (2015a)CrossRefGoogle Scholar
  16. Klausch, T., Hox, J.J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc.: Ser. A (Statistics in Society) 178(4), 945–961 (2015b)CrossRefGoogle Scholar
  17. Lohmann, H.: Comparability of eu-silc survey and register data: the relationship among employment, earnings and poverty. J. Eur. Social Policy 21(1), 1–18 (2011)CrossRefGoogle Scholar
  18. OECD: Divided We Stand - Why Inequality Keeps Rising (2011)Google Scholar
  19. Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)CrossRefGoogle Scholar
  20. Schouten, B., van den Brakel, J., Buelens, B., van der Laan, J., Klausch, T.: Disentangling mode-specific selection and measurement bias in social surveys. Soc. Sci. Res. 42, 1555–1570 (2013)CrossRefGoogle Scholar
  21. Statistik Austria: Standard Dokumentation Metainformationen - Definitionen, Erlaeuterungen, Methoden, Qualitaet (2010)Google Scholar
  22. Vannieuwenhuyze, J.T., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys: three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013)CrossRefGoogle Scholar
  23. Vannieuwenhuyze, J.T., Loosveldt, G., Molenberghs, G.: Evaluating mode effects in mixed-mode survey data using covariate adjustment models. J. Off. Stat. 30 (1), 1–21 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Pirmin Fessler
    • 1
  • Maximilian Kasy
    • 2
  • Peter Lindner
    • 1
  1. 1.Economic Analysis DivisionOesterreichische NationalbankViennaAustria
  2. 2.Department of EconomicsHarvard UniversityCambridgeUSA

Personalised recommendations