Abstract
We study the effect of interview modes on estimates of economic inequality which are based on survey data. We exploit variation in interview modes in the Austrian EU-SILC panel, where between 2007 and 2008 the interview mode was switched from personal interviews to telephone interviews for some but not all participants. We combine methods from the program evaluation literature with methods from the distributional decomposition literature to obtain causal estimates of the effect of interview mode on estimated inequality. We find that the interview mode has a large effect on estimated inequality, where telephone interviews lead to a larger downward bias. The effect of the mode is much smaller for robust inequality measures such as interquantile ranges, as these are not sensitive to the tails of the distribution. The magnitude of effects we find are of a similar order as the differences in many international and intertemporal comparisons of inequality.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Angrist, J., Pischke, J.-S.: Mostly Harmless Econometrics. Princeton University Press, Princeton (2008)
Atkeson, L.R., Adams, A.N., Alvarez, R.M.: Nonresponse and Mode Effects in Self- and Interviewer-Administered Surveys. Political Analysis (2014)
Cowell, F.A., Victoria-Feser, M.-P.: Robustness properties of inequality measures. Econometrica 64(1), 77–101 (1996)
de Leeuw, E.D.: Data Quality in Mail, Telephone and Face to Face Surveys. TT-Publikaties, Amsterdam (1992)
Dillman, D.A., Smyth, J.D., Christian, L.M.: Internet, Phone, Mail, and Mixed-Mode Surveys: the Tailored Design Method. Wiley, New York (2014)
DiNardo, J., Fortin, N., Lemieux, T.: Labor market institutions and the distribution of wages, 1973-1992: A semiparametric approach. Econometrica 64, 1001–1044 (1996)
Firpo, S., Fortin, N.M., Lemieux, T.: Unconditional quantile regression. Econometrica 77(3), 953–973 (2009)
Huber, P.J.: Robust Statistics. Wiley Series in Probability and Statistics. Wiley, New York (2003)
Iacus, S.M., King, G., Porro, G.: Matching for Causal Inference without Balance Checking. Working paper series, Harvard (2008)
Imbens, G.W.: Nonparametric estimation of average treatment effects under exogeneity: a review. Rev. Econ. Stat. 86(1), 4–29 (2004)
Imbens, G.W., Rubin, D.B.: Causal Inference fo Statistics, Social and Biomedical Sciences an Introduction. Cambridge University Press, Cambridge (2015)
Jäckle, A., Roberts, C., Lynn, P.: Assessing the effect of data collection mode on measurement. Int. Stat. Rev. 78(1), 3–20 (2010)
Klausch, T., Hox, J.J., Schouten, B.: Assessing the Mode-Dependency of Sample Selectivity Across the Survey Response Process. Technical Report 2013-03, Statistics Netherlands (2013a)
Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013b)
Klausch, T., Hox, J.J., Schouten, B.: Evaluating bias of sequential mixed-mode designs against benchmark surveys. Sociol. Methods Res. 46(3), 456–489 (2015a)
Klausch, T., Hox, J.J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc.: Ser. A (Statistics in Society) 178(4), 945–961 (2015b)
Lohmann, H.: Comparability of eu-silc survey and register data: the relationship among employment, earnings and poverty. J. Eur. Social Policy 21(1), 1–18 (2011)
OECD: Divided We Stand - Why Inequality Keeps Rising (2011)
Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity in observational studies for causal effects. Biometrika 70(1), 41–55 (1983)
Schouten, B., van den Brakel, J., Buelens, B., van der Laan, J., Klausch, T.: Disentangling mode-specific selection and measurement bias in social surveys. Soc. Sci. Res. 42, 1555–1570 (2013)
Statistik Austria: Standard Dokumentation Metainformationen - Definitionen, Erlaeuterungen, Methoden, Qualitaet (2010)
Vannieuwenhuyze, J.T., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys: three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013)
Vannieuwenhuyze, J.T., Loosveldt, G., Molenberghs, G.: Evaluating mode effects in mixed-mode survey data using covariate adjustment models. J. Off. Stat. 30 (1), 1–21 (2014)
Acknowledgements
We would like to thank Markus Knell and Alyssa Schneebaum for valuable comments and discussion. Additional to the usual disclaimer, the opinions expressed in this work are those of the authors and do not necessarily reflect the ones of the OeNB or the Eurosystem.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Fessler, P., Kasy, M. & Lindner, P. Survey mode effects on measured income inequality. J Econ Inequal 16, 487–505 (2018). https://doi.org/10.1007/s10888-018-9378-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10888-018-9378-x