Pitfalls in comparing Paris pledges
The Paris pledges are unique documents in climate governance that outline what each country intends to do to combat climate change. Often, these documents contain headline greenhouse gas percentage reduction targets that appear to summarize countries’ contributions to mitigation. This is a boon for comparative climate policy research. However, I show in this paper that the Paris pledges require detailed interpretation to be comparable. I demonstrate the risks in comparing these targets by re-visiting a recent studying linking national public opinion to the stringency of countries’ mitigation goals. I develop new indicators that better account for the structure of the targets and show in replications that the original finding is inconsistent with the underlying data. I conclude by drawing lessons for studying the Paris pledges.
The Paris pledges are unique documents in climate politics that outline what each country intends to do to combat climate change. The documents—called (Intended) Nationally Determined Contributions (INDCs)—are often oriented around a headline national greenhouse gas (GHG) emissions reduction target expressed in percentage terms.1 For example, the Canadian government announced its intention to cut Canadian GHG emissions by 25% by 2030, while the Russian government announced its intention to cut Russian emissions by 25–30% in the same time period. However, even though these two targets may appear similar at face value, they mask actual contributions to mitigation since each country benchmarks their percentage cuts using a different historical reference point. Canada benchmarked its Paris GHG target to 2005 emissions levels, while Russia benchmarked its target to 1990 emissions levels. Any actual GHG mitigation depends on both the percentage reduction and the reference point that emissions are to be cut from.
This point about measuring mitigation in INDCs would be purely semantic were it not distorting climate research. The Paris Agreement’s institutional design creates a unique opportunity for researchers to observe each country’s willingness to contribute to collective mitigation through the content of their INDCs. However, the Agreement’s design also left an opportunity for governments to communicate their pledges in ways that mask the true nature of their contributions, since states were not obligated to submit documents in a common format. Changing the reference years or other parameters of GHG accounting allows governments to submit ambitious sounding GHG targets that are disconnected from the actual GHG emissions reductions that are necessary to keep global mean temperatures below 1.5∘C or 2∘C.
I illustrate the nature of this problem by re-visiting a recent study by Aaron Drummond and co-authors in Climatic Change (Drummond et al. 2018). This study finds that countries with higher public awareness of climate change adopted more stringent GHG targets in the Paris Agreement than countries with lower public awareness of climate change, controlling for a number of covariates. This finding suggests that efforts to raise public awareness may help catalyze more ambitious climate policies and overcome persistent collective action problems that have plagued climate politics. However, errors in interpretation and measurement in the study produce misleading inferences about the relationship between public opinion and GHG targets. Essentially, the study takes each country’s headline GHG emissions reduction percentage as its measure of the stringency of mitigation without accounting for the fact that countries expressed their targets in fundamentally different ways. As countries pledged to reduce their emissions from different reference years or forecasts of future emissions growth, the naive percentage target may be disconnected from actual GHG emissions reductions and is therefore not directly comparable across countries.
I propose two alternative measures of mitigation targets that better account for the structure of the Paris targets and use these to re-visit the findings in Drummond et al. (2018). My two indicators standardize each country’s target relative to their own historical emissions. The indicators do not address the distributional implications or equitable mitigation across countries. Instead, they purge opportunistic choices of reference levels that allow governments to declare large percentage cuts while actually pledging minor changes in absolute emissions. In this sense, they re-focus measurement on emissions flows and remove elements of political communication from target-setting. Using these new indicators, I do not find that public opinion predicts the stringency of countries’ Paris targets.
This paper makes contributions for those interested in climate politics, public opinion, and international cooperation. For climate scholars, this paper demonstrates the challenges in comparing countries’ Paris targets since the pledge documents vary dramatically in form as well as in substance. Future studies of countries’ Paris pledges ought to be very attentive to the manner in which countries express their climate targets. For scholars of public opinion, this paper shows that public opinion does not appear to be related to the ambition of governments’ GHG targets in the Paris Agreement. Future research should explore when and where public awareness of climate change filters through domestic political institutions to affect governments’ climate policies. Finally, for scholars of international cooperation, this paper cautions that the “bottom-up” model of international cooperation—whereby governments deliberate nationally and put forward cooperative pledges—may create an opportunity structure for governments to mislead interested publics about the scale of their contributions. If the Paris Agreement is a new model for future international cooperation, then negotiators and academics should be aware of this challenge to measuring and comparing cooperative efforts.
2 Paris targets: nationally determined contortions?
The INDC model of target setting reflects a compromise that allowed for maximum participation in the Paris Agreement at the expense of transparency and precision (Keohane and Oppenheimer 2016). Instead of negotiating country-level GHG targets multilaterally—as was done with the 1997 Kyoto Protocol and was attempted at the 2009 United Nations climate conference in Copenhagen—governments selected their own, nationally determined, contributions to addressing climate change before meeting in Paris in December 2015. However, many states had reservations about accepting costly mitigation commitments given that there was uncertainty about their counterparts’ willingness to contribute as well. Therefore, while the 2014 Lima climate conference outcome document invited parties to include “quantifiable information on the reference point (including, as appropriate, a base year), time frames and/or periods for implementation, scope and coverage, planning processes, assumptions and methodological approaches including those for estimating and accounting for anthropogenic greenhouse gas emissions and, as appropriate, removals,” this information was not mandatory (UNFCCC 2014, para. 14). Governments interpreted this guidance loosely and submitted INDCs that vary tremendously in their content, detail, as well as ambition.
Since domestic GHG mitigation imposes costs in the present and creates uncertain future benefits, political leaders must navigate difficult intertemporal and distributive bargains (Jacobs 2011). In contexts where domestic publics are more aware of climate change, politicians have incentives to appear to be taking responsible action to mitigate risks, while remaining sensitive to the political risks if policy reforms are costly (Harrison and Sundstrom 2010). When policymakers have access to complex policy instruments, such as with the largely unstructured INDC system, these incentives may lead actors to increase the complexity of policy measures rather than to match public preferences for policy outcomes (Kono 2006). Similarly, national governments may engage in “symbolic regulation” that seeks to reassure domestic regulatory advocates, without providing the actual policy reforms advocates desire (Peacock 2018). In sum, recent political science research suggests that when governments face pressure to impose new regulations but also fear the costs of policy reforms, they may adopt policies that mask their true impacts from public audiences.
In climate mitigation, this obfuscation may take the form of a divergence between stated GHG emissions reductions and actual reductions. If such a divergence exists, then using the headline percentage emissions reduction to measure GHG emissions reductions—as Drummond et al. (2018) do—may be uninformative about actual emissions reductions. Governments may pledge ambitious sounding policy targets to appear engaged, while cushioning the impact of such policies by changing the terms to minimize the actual emissions reductions associated with a stated percentage point change. I now explain how governments could use the vagueness of the INDC design to disguise their true GHG mitigation commitments.
The lack of common reporting requirements has led the INDCs to vary along many dimensions and the documents therefore require further interpretation to make them comparable. The World Resources Institute has undertaken an exhaustive data collection and synthesis project on the INDCs (WRI 2016; Ross et al. 2016). Political scientists have already begun analyzing this data (Tobin et al. 2018). For the purposes of this paper, I outline the different categories of targets that countries adopted in their INDCs and use these to explain how different types of targets create opportunities for headline percentage reductions to diverge from actual emissions reductions. Interested readers should consult Ross et al. (2016), Climate Watch (2016), and Tobin et al. (2018) for more in-depth treatments of target types.
INDCs may contain three types of non-mutually exclusive climate targets. First, countries may declare policy goals that have mitigation co-benefits, such as increasing energy efficiency by 20%. Second, they may announce specific mitigation of adaption actions without linking these explicitly to GHG emissions flows, such as improving coastal management to reduce flood risk. Finally, countries may express economy-wide GHG emissions reduction targets. The above non-GHG targets and actions targets cannot be compared directly with GHG targets without a host of non-trivial modelling decisions. Ultimately, not all countries have directly comparable targets, which may lead to misleading inferences for the raw data.
Economy-wide GHG emissions reductions targets can also be expressed in different ways. Base year targets specify GHG targets in terms of reductions from GHG emissions in a historical base year. This GHG target format is adopted by Annex 1, industrialized countries, as well as a few developed countries. As noted above, Canada has pledged to reduce GHG emissions by 30% from 2005 GHG emissions levels by 2030, while Russia will cut by 25–30% from 1990 GHG emissions levels by 2030. Canada and Russia’s targets may be similar in headline percentages (30% and 25–30%, respectively), but the choice of base years (2005 and 1990, respectively) informs the size of a percentage reduction. Russia’s emissions fell substantially during its prolonged recession at the end of the Cold War, but have been very stable since the mid-1990s and never recovered to their earlier levels. In this situation, choosing an earlier base year makes a percentage reduction seem larger than choosing a later base year. Canada’s emissions trajectory is reversed, as its emissions have risen consistently over the past twenty years, making percentage reductions seem larger if a later base year is chosen. Governments may choose base years strategically to make an emissions reduction seem larger and so they ought to be standardized with common base years.
Scenario targets specify emissions reductions from a hypothetical “business as usual” (BAU) GHG emissions growth scenario. This type of target is common among non-Annex 1, developing countries. For example, Brazil announced its intention to cut GHG emissions by 37% relative to BAU levels by 2025. However, governments are not always transparent about how they calculate BAU emissions growth and some countries forecast much quicker GHG growth in the 2015–2030 period than in the previous 20 years. The BAU forecasts are not simple extrapolations of historical growth rates. Just as countries may select base years opportunistically, they may also forecast high exceptionally BAU emissions growth to make a nominal emissions cut seem larger. Furthermore, some countries do not specify a future BAU emissions level that they intend to cut emissions from, thereby making their targets fundamentally non-quantifiable. Without an emissions reference point, it will not be possible to assess whether actual GHG emissions in 2030 are below “BAU” levels by the target amount.
Countries may also index their GHG targets to another variable, such as national gross domestic product (GDP) or population. These intensity targets require forecasting the index variable (i.e., GDP or population). A 30% reduction in the emissions intensity of GDP is not the same as a 30% reduction in absolute emissions even if both can be evaluated at the expiry of the Paris Agreement. Trajectory targets outline intentions to peak emissions by a certain year. However, if no hard cap on absolute GHG emissions is given to this peaking level, then peaking cannot be compared with other targets ex ante and will require many additional time periods after the target year to verify that emissions have actually peaked.
The INDC design allowed countries high levels of discretion in drafting their Paris pledges. This led to wide variation in the form and substance of the documents. Comparing GHG emissions reductions targets across countries requires knowing the percentage reduction, the reference absolute emissions level from which emissions are to be cut, the target year, and historical emissions levels. The headline percentage reduction target is uninformative without this complementary information, which may be omitted by governments or manipulate to inflate perceptions of their mitigation contributions. In the next section, I outline two alternative measures of countries’ targets that are less sensitive to opportunistic choice of reference levels and that allow all countries with quantifiable GHG targets to be compared on a common scale.
3 New measures of Paris targets
I now present two alternative measures of GHG emissions mitigation that better account for the structure of GHG targets than the headline percentage term that Drummond et al. (2018) use. The headline percentage indicator is highly sensitive to the choice of reference levels, since governments may make their target appear more ambitious by selecting reference levels with higher emissions. The two new indicators that I evaluate are not sensitive to opportunistic choices of reference levels, since I standardize countries’ targets using their own historical emissions and using common reference points for all countries. As such, the measures are transparent, available for all countries with explicit GHG targets, and remove aspects of political communication that may lead headline percentages to diverge from actual emissions reductions. These attributes allow discussion to focus on GHG emissions flows in the near term and relative comparisons across states. The first measure compares each country’s annual GHG emissions growth rate under compliance with their selected Paris target to their annual emissions growth rate in the preceding two decades. The second measure re-scales each country’s Paris target in terms of a percentage difference from every year between 1990 and 2014 to evaluate countries in common.
However, neither indicator that I evaluate perfectly summarizes national climate policy effort. As Aldy et al. (2017) explain, measures of implicit carbon prices or the marginal cost of abatement better capture government policy but are not directly observed or are not widely available cross-nationally. Furthermore, the two indicators I evaluate make no adjustments for national-level mitigation costs, equitable mitigation, or the consistency of national targets with the Paris Agreement’s goal of limiting temperature rise to 1.5∘C or 2∘C. Instead, they highlight the importance of identifying emissions baselines and changes in emissions flows. They allow for relative comparisons across countries without distortions that may arise from the vagueness of the bottom-up design of the Paris Agreement. Future work could consider equitable mitigation more explicitly (Robiou du Pont et al. 2017).
4 Public opinion and international mitigation commitments
I use these two new indicators to re-estimate the relationship between public opinion and national GHG targets that Drummond et al. (2018) analyzed previously. To measure public opinion, Drummond et al. (2018) use survey data from Lee et al. (2015), who studied the determinants of public awareness of and perceptions of risk from climate change in 119 countries in 2007–2008. In this respect, Drummond et al. (2018) join a growing research program exploring public support for international climate cooperation (Tingley and Tomz 2013; Bechtel et al. 2017; McGrath and Bernauer 2017). To study national mitigation targets, Drummond et al. (2018) compile each country’s unconditional GHG emissions reduction target in the Paris Agreement, using summary information contained in the Supplementary Information of a study by Rogelj et al. (2016). The original study uses only the headline percentage reduction to measure stringency, so a 30% reduction is always more stringent than a 20% reduction. As such, it does not account for how countries have used different reference levels in calculating these reductions. The study regresses country-level mitigation commitments on country-level public opinion, controlling for a number of confounding variables.
I replicate model 4 from table 2 in Drummond et al. (2018) with my new measures. The data are compiled from Climate Watch (2016) and cover only INDCs submitted before the Paris climate conference. I make three further changes to the data. First, the original study scores countries lacking unconditional GHG targets as “0”. This implies that these countries actually have headline GHG targets to hold emissions constant under the Paris Agreement. As shown in the previous section, many countries with GHG targets allow their GHG emissions to grow under compliance with their Paris targets, which makes the assumption that lacking an unconditional target implies holding emissions constant tenuous. I recode these as missing data, since they are missing unconditional targets. Second, since European Union (EU) countries actually have differentiated GHG emissions reductions targets as set forth in a 2016 effort sharing decision (EU 2018/842), I disaggregate the EU and include member states individually in my analyses. Finally, I cross-reference the unconditional target measure in Rogelj et al. (2016) and Drummond et al. (2018) with data from the World Resources Institute and identify a handful of countries coded as having unconditional targets in the original paper, but whose targets are actually communicated conditionally (Climate Watch 2016).
Relationship between country-level public opinion and mitigation targets
Δ GHG growth rate
For comparison, Table 1 model I re-estimates Drummond et al. (2018)’s main specification with their data. I find the same relationships for awareness and perceived threat as in the original study. In model II, I use my re-coded headline target outcome variable, described above. The sample composition differs slightly from model I, as a number of developing countries that did not communicate unconditional targets are removed from the sample and EU countries are added in.3 I recover the same relationship as in the original study and model I.
In model III, I estimate the relationship between public opinion and changes in countries’ GHG growth rates under their Paris targets. I find a negative coefficient for public awareness, suggesting that countries with greater awareness of climate change adopted less stringent Paris targets than countries with lower levels of awareness. Nonetheless, this coefficient is not statistically significant at the p < 0.05 level. I interpret this as evidence for a disconnect between states’ headline GHG percentage targets and the underlying stringency of their targets.
The discrepancy between the headline percentages and standardized GHG targets merits closer scrutiny. One explanation for the relationship between nominal targets and public opinion could be that governments respond to public awareness of climate change, but that they rely on symbolic headline reductions to appear to be taking action while actually deferring costly mitigation policies. If this were the case, then the relationship should be stronger in countries where citizens perceived greater threats from climate change, but this relationship is not borne out in the data. Future work could consider more explicitly how public opinion is filtered through domestic political institutions to influence government climate policy.
Another explanation for the divergence between different measures of targets may stem from the fact that not all countries expressed quantifiable and comparable GHG targets in their INDCs. While nearly every country submitted an INDC before COP21 in Paris, not all of these documents contained explicit GHG targets, and even fewer contained economy-wide GHG mitigation targets that governments could be held accountable to in 2030. Saudi Arabia, for example, pledged to pursue economic diversification with mitigation co-benefits in its INDC, but did not specify an emissions target. Lee et al. (2015) estimate only 48.8% of Saudi citizens are aware of climate change, roughly 1.5 standard deviations below the mean level of awareness worldwide. If countries with lower levels of awareness or perceived threat systematically drop out of the analysis because they lack quantifiable GHG targets, then this could introduce bias into the models. A simple difference of means test shows that roughly 69.7% [64.1, 75.2] of citizens in countries with quantifiable targets were aware of climate change, while only 54.5% [49.7, 59.4] of citizens in countries with non-quantifiable targets were aware of climate change, a difference that is statistically significant at the p < 0.01 level. There may be a relationship between public awareness of climate change and the stringency of governments’ GHG targets, but since we can only compare states with a certain level of commitment—states that propose quantifiable GHG targets—the relationship may be obscured by the censoring in the data.
Drummond et al. (2018) argue that greater public awareness of climate change is associated with more stringent national GHG emissions reductions targets. However, I show that this finding does not hold when targets are standardized using common base years or historical emissions growth rates. In the Paris Agreement on Climate Change, countries pledged to reduce their GHG emissions by a percentage chosen by their national government. Furthermore, governments also chose the reference emissions levels used to benchmark their reductions. For governments worried about the costs of domestic GHG mitigation and worried that domestic or transnational publics would hold them accountable for their targets, this target structure created opportunities for obfuscation. Since actual emissions mitigation is a function of both reference emissions levels and percentage changes, the headline emissions reduction percentage contained in national pledges can be a misleading summary of mitigation. When mitigation pledges are standardized in terms of historical GHG emissions growth rates or using base years common to all countries, I find no relationship between public opinion and GHG mitigation targets using conventional standards of statistical significance.
Changes in absolute emissions or relative to historical emissions may be disconnected from notions of equity and fairness that underpin fragile existing climate cooperation (Climate Watch 2016; Robiou du Pont et al. 2017). Countries have made very different contributions to global warming and have very different abilities to respond. Failing to consider other priorities, such as development and the eradication of poverty, may bias findings against poorer countries (Kartha et al. 2018). Future work should consider the role of equity more explicitly in evaluating national climate policies.
This paper demonstrates that studies of INDCs ought to pay close scrutiny to the manner in which governments articulate their commitments. Headline percentages do not map clearly onto absolute or relative emissions mitigation. INDCs are a novel treaty design strategy, but they may provide opportunities for governments to feign cooperation.
This has implications for the durability and effectiveness of the Paris Agreement. It is widely held that robust climate cooperation depends on reciprocity, with counterparts making roughly equal contributions to collective governance and with provisions for differentiation based on responsibility and capabilities (Tingley and Tomz 2013; Keohane and Oppenheimer 2016). Transparency is key for comparing states’ contributions and holding them accountable for their pledges (Aldy et al. 2017). If the INDC system rewards vagueness, then accountability, reciprocity, and ultimately effectiveness are likely to suffer. In this respect, the current impasses at the United Nations climate conferences relating to transparency in the “Paris Rulebook” is unsurprising given the imprecision of national pledges.
Targets are “intended” until a country ratifies the Paris Agreement, at which point they become “Nationally Determined Contributions.” Countries may submit new NDCs upon ratification, but evidence to date suggests countries are not changing their GHG targets in re-submissions.
These are governmental effectiveness, the rule of law, regulatory quality, corruption, and political voice, accountability, and stability.
I would like to thank Ben Abraham, Duncan Snidal, and Alexandra Zeitz for helpful comments. This research was supported by the Robertson Foundation (grant number 9908921).
- Bechtel M, Genovese F, Scheve K (2017) Interests, norms and support for the provision of global public goods: the case of climate co-operation. Br J Polit Sci, 1–23Google Scholar
- Climate Watch (2016) Data for climate action. https://www.climatewatchdata.org
- Harrison K, Sundstrom LM (2010) Global commons, domestic decisions: the comparative politics of climate change. MIT PressGoogle Scholar
- Jacobs A (2011) Governing for the long term: democracy and the politics of investment. Cambridge University PressGoogle Scholar
- Peacock C (2018) Symbolic regulation: human rights provisions in preferential trade agreements. PhD thesis, University of OxfordGoogle Scholar
- Ross K, Rich D, Ge M (2016) Translating targets into numbers: quantifying the greenhouse gas targets of the G20 countriesGoogle Scholar
- UNFCCC (2014) United Nations Framework Convention on Climate Change. Decision 1/CP.20 Lima call for climate actionGoogle Scholar
- Victor D (2011) Global warming gridlock: creating more effective strategies for protecting the planet. Cambridge University PressGoogle Scholar
- WRI (2016) World Resources Institute. Paris contributions map (INDCs). http://www.wri.org/resources/data-sets/cait-paris-contributions-data
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.