An experimental examination of measurement disparities in public climate change beliefs

  • Matthew MottaEmail author
  • Daniel Chapman
  • Dominik Stecula
  • Kathryn Haglin


The extent to which Americans—especially Republicans—believe in anthropogenic climate change (ACC) has recently been the subject of high profile academic and popular disagreement. We offer a novel framework, and experimental data, for making sense of this debate. Using a large (N = 7,019) and demographically diverse sample of US adults, we compare several widely used methods for measuring belief in ACC. We find that seemingly trivial decisions made when constructing questions can, in some cases, significantly alter the proportion of the American public who appear to believe in human-caused climate change. Critically, we find that some common measurement practices may nearly double estimates of Republicans’ acceptance of human-caused climate change. We conclude by discussing how this work can help improve the consumption of research on climate opinion.


Supplementary material

10584_2019_2406_MOESM1_ESM.pdf (145 kb)
(PDF 145 KB)


  1. Benjamin D, Por HH, Budescu D (2017) Climate change versus global warming: who is susceptible to the framing of climate change? Environ Behav 49(7):745–770CrossRefGoogle Scholar
  2. Brenan M, Saad L (2018) Global warming concern steady despite some partisan shifts. Gallup. Retrieved from:
  3. Capstick S, Whitmarsh L, Poortinga W, Pidgeon N, Upham P (2015) International trends in public perceptions of climate change over the past quarter century. Wiley Interdiscip Rev Clim Chang 6(1):35–61CrossRefGoogle Scholar
  4. Coppock A, McClellan OA (2019) Validating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondents.
  5. Funk C, Kennedy B (2016) The politics of climate. Pew Research Center. Retrieved from:
  6. Greenhill M, Leviston Z, Leonard R, Walker I (2014) Assessing climate change beliefs: response effects of question wording and response alternatives. Public Underst Sci 23(8):947–965CrossRefGoogle Scholar
  7. Groothuis PA, Whitehead JC (2002) Does don’t know mean no? Analysis of ‘don’t know’ responses in dichotomous choice contingent valuation questions. Appl Econ 34(15):1935–1940CrossRefGoogle Scholar
  8. Heerwegh D (2009) Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. International Journal of Public Opinion Research 21(1):111–121CrossRefGoogle Scholar
  9. Kahan DM (2015) Climate-science communication and the measurement problem. Polit Psychol 36:1–43CrossRefGoogle Scholar
  10. Kahan D (2017) The “Gateway Belief” illusion: reanalyzing the results of a scientific-consensus messaging study. JCOM: J Sci Commun 16:1bCrossRefGoogle Scholar
  11. Kahan DM, Peters E, Wittlin M, Slovic P, Ouellette LL, Braman D, Mandel G (2012) The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat Clim Chang 2(10):732CrossRefGoogle Scholar
  12. Krosnick JA (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol 5(3):213–236CrossRefGoogle Scholar
  13. Krosnick JA (2018) Questionnaire design. In: The Palgrave handbook of survey research. Palgrave Macmillan, pp 439–455Google Scholar
  14. Krosnick JA, Berent MK (1993) Comparisons of party identification and policy preferences: The impact of survey question format. Am J Polit Sci, 941–964Google Scholar
  15. Krosnick JA, Malhotra N, Mittal U (2014) Public misunderstanding of political facts: how question wording affected estimates of partisan differences in birtherism. Public Opin Q 78(1):147–165CrossRefGoogle Scholar
  16. Kunda Z (1990) The case for motivated reasoning. Psychol Bull 108(3):480CrossRefGoogle Scholar
  17. Kwak N, Radler B (2002) A comparison between mail and web surveys: response pattern, respondent profile, and data quality. J Off Stat 18(2):257Google Scholar
  18. Leiserowitz A, Maibach E, Roser-Renouf C, Rosenthal S, Cutler M, Kotcher J (2018) Politics & global warming, March 2018. Yale University and George Mason University. New Haven, CT: Yale program on climate change communicationGoogle Scholar
  19. Maitland A, Tourangeau R, Sun H (2018) Separating science knowledge from religious belief two approaches for reducing the effect of identity on survey responses. Public opinion quarterly, nfx049Google Scholar
  20. McAdam D (2017) Social movement theory and the prospects for climate change activism in the United States. Annu Rev Polit Sci 20:189–208CrossRefGoogle Scholar
  21. McCright AM, Dunlap RE (2011) The politicization of climate change and polarization in the American public’s views of global warming, 2001–2010. Sociol Q 52 (2):155–194CrossRefGoogle Scholar
  22. Mutz DC (2011) Population-based survey experiments. Princeton University Press, PrincetonCrossRefGoogle Scholar
  23. Nadler JT, Weston R, Voyles EC (2015) Stuck in the middle: the use and interpretation of mid-points in items on questionnaires. J Gen Psychol 142(2):71–89CrossRefGoogle Scholar
  24. Pasek J (2017) It’s not my consensus: motivated reasoning and the sources of scientific illiteracy. Public Understanding of Science 0963662517733681Google Scholar
  25. Polk E (2018) Communicating climate change where did we go wrong, how can we do better?. Handbook of communication for development and social change, pp 1–19Google Scholar
  26. Schuldt JP, Roh S, Schwarz N (2015) Questionnaire design effects in climate change surveys: Implications for the partisan divide. The ANNALS of the American Academy of Political and Social Science 658(1):67–85CrossRefGoogle Scholar
  27. Schuldt JP, Enns PK, Cavaliere V (2017) Does the label really matter? Evidence that the US public continues to doubt “global warming” more than “climate change”. Clim Chang 143(1–2):271–280CrossRefGoogle Scholar
  28. Schuman H, Presser S (1981) Questions and answers in attitude surveys: experiments on question form, wording, and context. Sage, Newbury ParkGoogle Scholar
  29. van Boven L, Sherman D (2018) Actually, republicans do believe in climate change. New York TimesGoogle Scholar
  30. Van Boven L, Ehret PJ, Sherman DK (2018) Psychological barriers to bipartisan public support for climate policy. Perspect Psychol Sci 13(4):492–507CrossRefGoogle Scholar
  31. van der Linden SL, Leiserowitz AA, Feinberg GD, Maibach E (2015) The scientific consensus on climate change as a gateway belief: experimental evidence. PLoS ONE, p 10Google Scholar
  32. Villar A, Krosnick JA (2011) Global warming vs. climate change, taxes vs. prices: does word choice matter? Clim Change 105(1):11–12Google Scholar
  33. Whitmarsh L, Capstick S (2018) Perceptions of climate change. In: Psychology and climate change, pp 13–33Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Annenberg Public Policy CenterUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.Yale Law SchoolNew HavenUSA
  3. 3.Simon Fraser UniversityBurnabyCanada

Personalised recommendations