Incentivised Online Panel Recruitment and Subjective Wellbeing: Caveat Emptor
- 267 Downloads
It is generally assumed that if a sample represents its broader population on key demographic variables, the data it yields will also be representative. Here we present evidence to suggest that this is not necessarily so when subjective wellbeing is measured from participants recruited through online panels. Using data from six countries: Australia, Germany, Sweden, the Netherlands, UK and USA, we reveal significant differences in subjective wellbeing between online panel data and nationally representative data, even though both are demographically comparable. These findings indicate that the online panels comprised an abnormally high proportion of people with low subjective wellbeing, thus rendering their data non-representative. Given the widespread use of online panels to collect data in the modern era, we issue a caveat emptor.
KeywordsOnline panels Data collection Subjective wellbeing Sampling
Compliance with Ethical Standards
Conflict of Interest Statement
On behalf of all authors, the corresponding author states that there is no conflict of interest.
- Australian Bureau of Statistics (2016). Household use of information technology, Australia, 2014–2015, Category no. 8146.0. Canberra, Government of Australia.Google Scholar
- Baker, R., Blumberg, S. J., Brick, J. M., Couper, M. P., Courtright, M., Dennis, J. M., Dillman, D., Frankel, M. R., Garland, P., Groves, R. M., Kennedy, C., Krosnick, J., Lavrakas, P. J., Lee, S., Link, M., Piekarski, L., Rao, K., Thomas, R. K., & Zahs, D. (2010). Research synthesis: AAPOR report on online panels. Public Opinion Quarterly, 74(4), 711–781.CrossRefGoogle Scholar
- Bethlehem, J., & Stoop, I. (2007). Online panels - a paradigm theft. In The challenges of a changing world. Proceedings of the fifth international conference of the Association for Survey Computing, September, 113–137.Google Scholar
- Blanchflower, D. G., & Oswald, A. J. (2004). Well-being over time in Britain and the USA. Journal of Public Economics, 88, 1359–1386. https://doi.org/10.1016/S0047-2727(02)00168-8.
- Capic, T., Li, N., & Cummins, R. A. (2017). Set-points for subjective wellbeing: a replication and extension. Social Indicators Research, (in press).Google Scholar
- Cummins, R. A., & Weinberg, M. K. (2015). Multi-item measurement of subjective wellbeing: Subjective approaches. In W. Glatzer, L. Camfield, V. Møller, & M. Rojas (Eds.), Global handbook of quality of life: exploration of well-being of nations and continents (pp. 239–268). Dordrecht: Springer.CrossRefGoogle Scholar
- Cummins, R. A., Woerner, J., Weinberg, M., Collard, J., Hartley-Clark, L., & Horfiniak, K. (2013). Australian Unity Wellbeing Index: Report 30.0 - The wellbeing of Australians: Social media, personal achievement, and work. Melbourne: Australian Centre on Quality of Life, School of Psychology, Deakin http://www.acqol.com.au/uploads/surveys/survey-030-report-part-a.pdf.
- De Leeuw, E. D. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21, 233–255.Google Scholar
- Douglas, S. P., & Craig, C. S. (1983). International Marketing Research. Englewood Cliffs: Prentice-Hall.Google Scholar
- European Social Survey. (2012). ESS round 6 (2012) project instructions (PAPI). London: Centre for Comparative Social Surveys, City University London http://www.europeansocialsurvey.org/docs/round6/fieldwork/source/ESS6_source_project_instructions.pdf. Last accessed 13 April 2015.Google Scholar
- Gallup. (2009). Gallup World Poll. Washington, DC: Gallup.Google Scholar
- Helliwell, J., Layard, R., & Sachs, J. (2017). World Happiness Report 2017. New York: Sustainable Development Solutions Network.Google Scholar
- Maggino, F. (2016) Challenges, needs and risks in defining wellbeing indicators. In F. Maggino (Ed.), A life devoted to quality of life. Social indicators research series, vol 60. Cham: Springer. https://doi.org/10.1007/978-3-319-20568-7_13
- Meade, A.W., & Craig, S.B. (2011). Identifying careless responses in survey data. Paper presented at the 26th Annual Meeting of Society for Industrial and Organizational Psychology, Chicago, IL.Google Scholar
- Prince, K. R., Litovsky, A. R., & Friedman-Wheeler, D. G. (2012). Internet-mediated research: beware of bots. The Behavior Therapist, 35, 85–88.Google Scholar
- The World Bank (2016). Accessed online January 25 2017 via https://data.worldbank.org/indicator/it.net.user.zs
- Tomyn, A. J., Weinberg, M. K., & Cummins, R. A. (2015). Intervention efficacy among 'at-risk' adolescents: a test of subjective wellbeing homeostasis theory. Social Indicators Research, 120, 883–895. https://doi.org/10.1007/s11205-014-0619-5
- WHO Regional Office for Europe. (2012). Measurement of and target-setting for well-being Second meeting of the expert group, Paris, 25–26 June 2012. Copenhagen: WHO Regional Office for Europe.Google Scholar