Does Device or Connection Type Affect Health Preferences in Online Surveys?
Background and Objective
Recent evidence has shown that online surveys can reliably collect preference data, which markedly decrease the cost of health preference studies and expand their representativeness. As the use of mobile technology continues to grow, we wanted to examine its potential impact on health preferences.
Two recently completed discrete choice experiments using members of the US general population (n = 15,292) included information on respondent device (cell phone, tablet, Mac, PC) and internet connection (business, cellular, college, government, residential). In this analysis, we tested for differences in respondent characteristics, participation, response quality, and utility values for the 5-level EQ-5D (EQ-5D-5L) by device and connection.
Compared to Mac and PC users, respondents using a cell phone or tablet had longer completion times and were significantly more likely to drop out during the surveys (p < 0.001). Tablet users also demonstrated more logical inconsistencies (p = 0.05). Likewise, respondents using a cellular internet connection exhibit significantly less consistency in their health preferences. However, matched samples for tablets and cell phones produced similar EQ-5D-5L utility values (mean differences < 0.06 on a quality-adjusted life-year [QALY] scale for all potential health states).
Allowing respondents to complete online surveys using a cell phone or tablet or over a cellular connection substantially increases the diversity of respondents and the likelihood of obtaining a representative sample, as many individuals have cell phones but not a computer. While the results showed systematic variability in participation and response quality by device and connection type, this study did not show any meaningful changes in utility values.
JH and BC shared much of the responsibility in creating the manuscript. JH performed the literature review and wrote the Introduction, Methods, and Discussion sections. BC wrote the Results section. The authors contributed equally in the data analysis.
Compliance with Ethical Standards
Funding support for this research was provided by a grant from the EuroQol Research Foundation (2016690). The views presented in the study do not necessarily reflect those of the EuroQol Group, and the publication of study results was not contingent on the sponsor’s approval or censorship of the manuscript.
Conflict of interest
John Hartman and Benjamin Craig declare that they have no conflicts of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in either study.
- 23.de Bruijne M, Wijnant A. Comparing survey results obtained via mobile devices and computers: an experiment with a mobile web survey on a heterogeneous group of mobile devices versus a computer-assisted web survey. Soc Sci Comput Rev. 2013;31(4):482–504. https://doi.org/10.1177/0894439313483976.CrossRefGoogle Scholar
- 25.Toninelli D, Revilla M. Smartphones vs PCs: does the device affect the web survey experience and the measurement error for sensitive topics? A replication of the Mavletova & Couper’s 2013 experiment. Surv Res Methods. 2016;10(2):153–69. https://doi.org/10.18148/srm/2016.v10i2.6274.Google Scholar
- 26.Mavletova A, Couper MP. Sensitive topics in PC web and mobile web surveys: is there a difference? Surv Res Methods. 2013;7(3):191–205.Google Scholar
- 30.Euroqol. About the EQ-5D-5L. 2018. https://euroqol.org/eq-5d-instruments/eq-5d-5l-about/. Accessed 22 Mar 2018.
- 31.Survey K. Mobile surveys. https://www.keysurvey.com/survey-software/mobile-surveys/2017. Accessed 9 May 2017.
- 39.Leuven E, Sianesi B. PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. Statistical Software Components. 2015. https://econpapers.repec.org/software/bocbocode/s432001.htm. Accessed 30 May 2017.
- 51.Al Sayah F, Johnson JA, Ohinmaa A, Xie F, Bansback N, on behalf of the Canadian EQ-5D Valuation Study Group. Health literacy and logical inconsistencies in valuations of hypothetical health states: results from the Canadian EQ-5D-5L valuation study. Qual Life Res. 2017;26(6):1483–92. https://doi.org/10.1007/s11136-016-1495-z.CrossRefPubMedGoogle Scholar
- 55.Wooldridge JM. Econometric analysis of cross section and panel data. Cambridge: MIT Press; 2010.Google Scholar
- 56.Greene WH. Econometric analysis. 5th ed. Chennai: Pearson Education India; 2003.Google Scholar