Forecasting and Polling

Abstract

As with all contemporary elections, opinion polling and forecasting have become head-line-grabbing analyses during the campaign period. However, the uniqueness of the competitive array in 2017 made traditional political economy models of forecasting very difficult to implement, with a resulting lack of such models in contrast to previous presidential races. Opinion polls, on the other hand, were more prevalent than ever and, despite performing poorly in the primary races, proved remarkably accurate in the two presidential rounds, as well as identifying the winning party, if not its score, for the first round of the legislatives. This chapter discusses the inapplicability of the structural forecast models, before turning to an examination of the opinion polls, and how well they depicted the nature of competition in the presidential race.

Electoral forecasting has become an increasingly common phenomenon across liberal democracies in the last decade. While forecasting has always been an informal feature of elections—from media punditry to a voter’s own opinion as to the likely outcome—a more rigorous approach to predicting an election outcome based upon statistical modelling of econometric and polling data has moved over the last 20 years from an academic pursuit to a headline-grabbing activity. Whilst the United States and the UK have been the main generators of election forecasts, France has enjoyed some prominence in forecasters’ analyses, particularly since 2002. The 2002 presidential election sparked particular interest in forecasting terms because of the largely unexpected success of Jean-Marie Le Pen in the first round of voting—a surge which the opinion polls had seemingly failed to anticipate (Durand et al. 2004). But polls had similarly performed poorly in the preceding 1997 legislative election (Jérôme et al. 1999). Not only did these result in pollsters’ soul-searching to identify why this had been the case, it also spurred a wider interest in building models to predict vote shares in France that used polls as only one variable, if at all.

Forecast models can be broken down into three main approaches—adjusted polling forecasts; ‘structural’, political economic forecasts; and synthetic models, which combine both polls and structural indicators (Lewis-Beck and Dassonneville 2015a). The first treat vote intention polls as statistical estimates of support which can be combined to address problems of statistical error, and house effects—the statistical impact of each pollster’s method of polling—and, using assumptions about trends in public opinion between polls and election day, provide an estimate of eventual vote share for each party or candidate (Jennings and Wlezien 2016). Structural models eschew vote intention and instead use a set of economic indicators, such as unemployment, GDP, perceived improvement in economic situation, plus polling data on governmental support, and, based upon the relationship between these and a party or candidate’s vote share in previous elections, extrapolate the likely vote share for that competitor in the forthcoming election. Finally, synthetic models combine the two approaches, expecting that the inevitable error that each approach implies can be reduced by using both simultaneously (Lewis-Beck and Dassonneville 2015b).

The literature on French elections using the structural approach in particular is extensive, across presidential elections (Nadeau et al. 2010, 2012; Foucault and Nadeau 2012), legislatives (Auberger and Dubois 2005; Arzheimer and Evans 2010) as well as second-order elections (Jérôme and Jérôme-Speziari 2000; Auberger 2005), with some focus on how the semi-presidential executive affects economic models (Lewis-Beck and Nadeau 2004). Polling forecasts are less common, being used more recently to track candidate varying position through the campaign, but not used to provide a direct forecast of outcome. The synthetic approach, then, with the exception of Lewis-Beck and Dassonneville (2015b) is by definition also absent.

The 2017 elections posed a set of problems to electoral forecasters, particularly those looking to use structural economic approaches, and consequently formalized academic forecasts of the result were noticeable by their relative absence, as compared with previous elections (Jérôme et al. 1999; French Politics 2008, 2012). As a result, the body of work which developed from 2002 onwards has not continued to expand, and indeed the pre-election forecasts available prior to 2017 constitute a very mixed bag of approaches. In this chapter, we discuss why 2017 posed such problems to forecasters, review what the few forecasts which existed did predict, and consider the significant success of the opinion polls in anticipating not only the winners of the first and second rounds but also the precise vote shares that each candidate enjoyed.

1 2017: A Difficult Year for Structural Forecasting

Structural models forecasting the vote share for incumbent parties fared particularly badly in light of the 2017 roster of candidates, and the very specificity characterizing the election which we discussed in the first two chapters of the book is the root of forecasting’s problems. The absence of the incumbent president, and in many ways of the incumbent party of government, given Hamon’s position as frondeur, meant that any model using an incumbent penalty/gain variable—generally, the dependent variable in political economy models even in France, as we will discuss shortly—was impossible to run, at least in its standard version.

Similarly, opposition vote share models would find little to base their forecasts on. François Fillon as the Republicans candidate would be identifiable as the official opposition. However, in a race with three other candidates from different political positions performing as well as or better than Fillon, and each constituting an opposition of sorts, a model attempting to forecast their collective vote share would produce nothing insightful. Moreover, any model looking at all opposition candidates would be doomed, not least as the ten non-incumbent candidates’ vote share summed to some 93 per cent of the vote.

Indeed, in the French context, with two-round run-off electoral systems, binary incumbent/opposition models are generally ill-suited to the task. Largely drawing upon two seminal papers from the United States in the 1970s (Mueller 1970; Kramer 1971), the vote-popularity (VP) function is premised upon the identification of some simple causal relationships between vote choice (or polling popularity, hence ‘VP’) and socio-economic indicators, usually measured at the national level, but also at the sub-national level—in the French case, often the départements. These relationships posit an influence upon the overall vote share for a party or an incumbent government from the state of the economy, whether absolute or relative, the country’s involvement in conflict, the number of terms served by the incumbent and a range of other institutional controls (Bélanger and Trotter 2017). Using previous election results and measurements of these indicators at the appropriate time, the econometric model fits the relationship between these indicators and extrapolates to the current election. Measurement of variables such as unemployment, GDP and inflation, historically, is usually taken at a lagged point—six months prior to the election—to allow for the effect of these to impact upon the electorate’s perceptions.

The model has been applied across a vast range of countries (Lewis-Beck and Stegmaier 2013), but as a usually binary model, it is discomfited by multi-party systems. The second round in the French system is of course better suited to this approach methodologically, and in the elections where the incumbent party, if not candidate, has faced the main opposition candidate, the incumbent model logic pertains. Again, however, 2017 presents a problem, in that neither incumbent nor opposition were represented in the second round. Furthermore, the ‘anyone but Le Pen’ strategy of the moderate parties saw incumbents and opposition grouped in supporting Macron, making his run-off support more heterogeneous politically and ideologically. As will be discussed in Chap. 8, the unwillingness of variable tranches of Fillon, Mélenchon and to a much lesser degree Hamon supporters to follow their candidates’ line and vote for Macron—or, in Mélenchon’s case, not vote for Le Pen—further crosscuts the binary simplicity. This causes issues in an explanatory model of vote, as we will consider in the next two chapters, but these can be rectified through more complex modelling. However, the structural political economy approach relies upon the previous relationship between its predictors and the relevant vote variable to extrapolate to the current race.

A final possibility remains, namely to divide between the two political blocs of left and right, to look at their respective vote shares. For legislative election forecasts of seat shares, this method has been broadly appropriate, given that the key disruptor of the left/right duality, the FN, has not in the past won a significant share of National Assembly seats (Sauger and Grofman 2016). But Macron and LREM’s stated centrist position, bridging left and right—a strategy which then combined voters of both blocs—would render any coding of the president and his party as ‘left’ or ‘right’ entirely arbitrary, and fallacious.

The FN candidate has been the subject of a subset of the French forecasting literature (Jérôme and Jérôme-Speziari 2003; Auberger 2008; Evans and Ivaldi 2013). However, these forecasts have generally used a set of variables relevant to radical right-wing support, such as unemployment, crime rates and immigration, rather than the more standard VP function variables. While this seems an appropriate choice for a ‘protest’ party or candidate, it is more problematic for a situation where Le Pen opposes another candidate in the second-round run-off—where such issues are less relevant, or even irrelevant, to the other candidate’s performance. One model with a long lead-in time did attempt to model FN seats in the legislative election (Evans and Ivaldi 2016), but was predicated upon competitive conditions similar to those that pertained in 2012, and consequently overestimated significantly the performance of a party beset by low turnout, presidential disappointment and a unique competitive array against a presidential party owning the democratic renewal agenda.

Moreover, in the case of the FN, the other finalist of 2017, the left–right bipolarity is doubly challenged in terms of its supply and demand. As regards party policy, the FN now occupies a specific competitive location which, in simple terms, combines left-leaning economic policies with its more traditional culturally right-wing ideology. At the demand level, previous FN research has identified a group of ‘ninistes’ amongst FN voters, who see themselves as neither left nor right (Mayer 2017)—something we find in the 2017 election, as we discuss in Chaps. 7 and 8—thus making it more difficult to place the party and its voters on the left–right spectrum. Overall, a far more heroic set of assumptions than are normally brought to bear in political economy models would need to have been made to fit the 2017 case into this framework—a task upon which, to our knowledge, forecasters mostly did not embark.

The few attempts to use such approaches to forecast the presidential elections encountered significant issues precisely because the conditions governing these models were not met. Consequently, in one case, the predictors indicated that François Fillon, rather than Emmanuel Macron, would reach the second round.¹ At least one model did attempt to forecast Hamon’s first-round vote share based upon incumbent popularity, and came within a percentage point of the socialist candidate’s actual score.² However, one must be sceptical that the incumbent themselves would have performed quite as poorly as this model suggests, given that part of Hamon’s failure was in running a radical left campaign against Mélenchon, and ignoring the centre–left space which featured no candidate between him and Macron. The perverse primary outcome discussed in Chap. 3, identifying a sub-optimal candidate in political spatial terms, presents the ’incumbent’ failing worse than the true incumbent would have done in reality. Despite Hollande’s parlous opinion poll ratings, occupying the centre–left ground spatially would almost certainly have ensured a score superior to that predicted by the model, and similarly would have done so for Manuel Valls. It is also worth noting that the popularity score from a Cevipof survey used in this model—4 per cent—is itself one of the lowest recorded for Hollande. Compare this with, say, Sofres favourability ratings which never dropped below 11 per cent.

As Jérôme and Jérôme-Speziari note, econometric forecast models rely upon three conditions to remain workable.³ First, the drivers of electoral support need to remain the same as in the previous elections upon which the parameters of the model are based. Second, and by extension, these drivers must remain observable. Third, the model cannot be disrupted by any exogenous shock. For 2017, none of these three conditions obtained. Incumbent models fell on the absence of the incumbent in the race, and even the incumbent party was a poor fit, given Hamon’s role as a political opponent ‘from within’ throughout most of the Hollande presidency. Similarly, left and right blocs were disrupted by the presence of Macron as a candidate bridging the moderate wings of both blocs, and drawing support from these. Indeed, the absence of Hollande and the presence of Macron constituted shocks to the modelling process, as well as the latter not abiding by the alternation logic inherent in most econometric models. As we discussed in the previous chapter, it is difficult to say with certainty that Hollande’s withdrawal from the race constituted a major event in the election, but it is highly significant as a disruption to a VP function.

Beyond the academic realm of structural models, the socio-economic data they draw on forms part of a recent, broader quantitative approach to forecasting. In an era when ‘Big Data’—the rigorous combination of multiple, varying data sources through data linkage and statistical algorithms—increasingly represents a step forward in data analysis both for private corporations and public sector organizations, it is perhaps inevitable that similar techniques have begun to be brought to bear on election forecasting. A relatively simple version of forecast and data aggregation has existed for a number of years in the PollyVote project (Graefe et al. 2014). By combining forecasts from a range of approaches, including expert opinion, opinion polls, betting markets and econometric models, the error associated with each can be averaged out to produce a more accurate forecast. PollyVote relies on a wealth of different forecasting data available in the United States, and more recently in Germany (Graefe 2017: 878). However, Big Data approaches combine available data such as social media messaging, web searches, economic data, census profiling and the like to provide an algorithmic estimation of likely vote.

Companies such as Filteris, Vigiglobe, Leonie Hill Capital and Enigma all provided forecasts of the first-round vote, and all turned out to be wrong, both in rank ordering and in vote share.⁴ These models’ success in forecasting both the Trump victory and the Brexit referendum—and polls’ apparent failure—had led to a heightened expectation of their efficacy. However, as with more traditional forecasting approaches, the difficulty of deriving precise estimates across a multi-candidate race, and particularly one so tightly bunched across the four candidates, proved beyond their capacity to estimate correctly, both for the presidentials and the legislatives.⁵ Exactly why they did not work is more difficult to ascertain, as the method used to combine and adjust the data is not generally circulated. In terms of clarity and usability, two of the basic criteria for rating model quality (see later), these models fall down.

Whilst social media have proved useful in some aspects of forecasting, they have shown that, in addition to the problems of skewed profile of social media users relative to the electorate—which certainly strongly mattered in 2017 given Fillon’s support relying heavily on older voters notably underrepresented on social media—reliance upon automated coding of language via sentiment analysis and keyword recognition, and the more generalized use of algorithmic information processing, can render estimations unstable. They are better used as a possible inflection to more established sources such as opinion polls (Ceron et al. 2017: 884–885). On a more prosaic level, a ‘correct’ estimation of the winner, if not the exact voter share, of a two-horse race such as an American election or a referendum is more probable than that of a four- or five-horse race.

2 Presidential Polling: Snatching Victory from the Jaws of Defeat

Despite their own problems, which we examine later, vote intention polls have the advantage that they do not require an understanding of the ‘why’ of vote choice. Voters’ motivations are irrelevant to the outcome. Moreover, previous fit, in terms of specific parties and candidates, is less relevant—voters can simply be polled on the candidates or parties in the race or even, in the case of trial heat polls, candidates or parties that might be in the race. The need to adjust data to account for biases in sampling, and use of polling data in modelling that needs to account for so-called house effects do, nonetheless, often require a trend analysis of polls to understand idiosyncrasies relating to particular parties or their candidates. In that respect, the presence of Macron, a candidate standing in the unusual political space of the centre, and thereby being accessible to voters of both centre–left and centre–right, and with no established party backing him, raised the possibility of polling being unable to pin down his support precisely.

A key issue underpinning many people’s concerns about polling in France in 2017 was stability of vote intention. All polling forecasts must take into account turnout, usually by factoring in variable probabilities for different types of voters’ actual vote likelihood, or simply using expressed probability of vote by the polling subjects themselves. However, in this election, a further complication was added by the relatively low certainty of Macron’s supporters until quite late in the campaign, compared with those of the other leading candidates, and in particular Marine Le Pen. As alluded to in Chap. 3 and discussed more extensively below, a similar issue had notably arisen from the Republican primary polls during 2016, which all had failed to predict Fillon’s victory, precisely, albeit not exclusively, because a substantial proportion of right-wing primary voters had left their decision to the final hours of the campaign.

As with turnout, the spectre of late deciders and switchers always hovers over any poll. However, the figures in Table 6.1 suggest that this was much more visible, and unsettling, for Macron’s support than in normal elections. Uncertainty over the actual level of support turned out to be exaggerated, but in the period preceding the election when forecasts are most commonly issued, this will have discouraged many forecasters, as well as encouraging others to speculate about eventual outcomes, should Macron’s vote have been affected by this, or indeed by the broader concerns over polling stability.⁶

Table 6.1 Certainty of vote choice for main candidates in the first round of presidential election (February and April 2017)

More broadly, in the run-up to the first round of the presidential election, polling was under intense scrutiny. First, and relating to the internationalization of electoral coverage which we discussed in the previous chapter, polling in recent high-profile ballots, namely the 2015 UK General Election, the 2016 US election and the 2016 Brexit referendum, had apparently failed to anticipate the result correctly. In both the UK and US cases in 2016, the criticisms were exaggerated, no doubt because the forecast result appeared to be wrong. UK polls had shown the ‘Remain’ camp to be slightly ahead for most of the campaign (Hobolt 2016: 1262). US polls had forecast a Hillary Clinton victory, giving a three-point lead on average. In the UK case, the lead of Remain was only small, and margin of error would account for much of the difference. However, given it was a threshold result, at 50 per cent + 1 for the winner, the impact of this error was much the greater, changing the outcome entirely. For the US case, the national polls were actually very accurate—Hillary Clinton did indeed win the popular vote by an almost 3 million surplus, or around 2 percentage points. State polling, particularly in the Mid-West region, was less accurate, and did not pick up on the electoral college split which resulted in Donald Trump’s victory (AAPOR 2017). Overall, a more apt comparator might have been the 2015 UK General Election, where polls did not predict the result accurately predicting party support concomitant with a hung parliament ended in giving the Conservative Party majority.⁷

Historically, the performance of polls in other countries would have been deemed irrelevant to polling in France. However, as we noted in the previous chapter, a level of international coverage of the elections, as well as the heightened outreach of candidates internationally, meant that the performance of the French polls was to be judged against that of the preceding elections in other countries, as well as in terms of French polling more narrowly.

2.1 Primary Polls

In France itself, polling’s track record prior to the campaign was less than auspicious, in its apparent failure to anticipate the election of François Fillon as the LR candidate or Benoît Hamon for the socialists. As Figs. 6.1 and 6.2 show, polls for both primaries had the eventual winners well behind.⁸ For the Belle Alliance Populaire, Hamon only overtook Montebourg in the last couple of weeks, despite eventually beating him by almost 20 per cent, and Valls still enjoyed on average a five-point lead over the frondeur. Nonetheless, the dynamics in the last days show a substantial upturn in Hamon’s support. Similarly, on the right, the final polls in November 2016 had Fillon behind both Sarkozy and Juppé, but the upswing in support from the beginning of the month clearly indicated where the outcome was likely heading. Given polling was carried out at least two days before the primary race, an extrapolation on these trend lines brings both Hamon and Fillon closer to their eventual scores. But even this silent lag cannot account for the entirety of the gap. Other issues clearly afflicted the primary polls.

Fig. 6.1

Vote intention polling for the primaires citoyennes (Left) primaries. Source: Authors’ collation of vote intention polls, https://fr.wikipedia.org/wiki/Sondages_sur_la_primaire_citoyenne_de_2017, polling institute archives

Fig. 6.2

Vote intention polling fir the centre/right primaries. Source: Authors’ collation of vote intention polls, https://fr.wikipedia.org/wiki/Sondages_sur_la_primaire_ouverte_de_la_droite_et_du_centre_de_2016, polling institute archives

The first issue with polling the primaries was the fuzzy profile of the voting population. Both primary elections were open primaries, allowing any voter on the electoral register to vote, conditional upon the payment of one euro (for the Belle Alliance Populaire) or two euros (for the right and centre), and signing a statement of their holding ‘values of the Republic and the left’ or ‘values of the Republic and the Right and Centre, and support alternation of power to support France’, respectively. Pollsters generally used either self-proclaimed supporters of the left or right, depending on the primary, or those respondents saying they were certain of voting in the primary ballot. Disaggregating primary support suggests that the ‘boundary’ of the selectorate in primary elections may matter substantially to the outcome. In an analysis of a TNS-SOFRES BAP primary poll conducted in July 2016—at a time when both Hollande and Macron were considered as potential runners—we found that Macron had the highest level of support in the general electorate, reflecting his appeal to centre–right voters, but that his lead would vanish when the selectorate was narrowed down to PS voters. In contrast, there was evidence that the more the primary would resemble an internal party race, the better the odds that Hollande could win the nomination.⁹

The uncertainty regarding the profile of the voting population had two other effects. First, the sub-sample of the original sample was very much reduced in size. For example, with a sample size for the Belle Alliance Populaire primary polls of 500—larger than many of the samples used by polling companies—the margin of error is ± 3.5 per cent for an estimate of 20 per cent vote, that is, a confidence interval from 23.5 per cent to 16.5 per cent. But this would only be true, were the sample a random sample of the true population of voters. Quota samples of voters for Internet panels are routinely adjusted by pollsters to try to approximate true vote intentions, but adjusting smaller sub-samples for a particular group introduces additional uncertainty.

Second, the eventual turnout was not restricted to supporters of the parties fielding candidates in the primaries. For the right and centre in November, only 63 per cent of those voting described themselves as supporters of this political bloc.¹⁰ Around 14–15 per cent identified with the left, and a similar number with the FN. For les primaires citoyennes on the left, the party preference was remarkably similar—with around seven in ten voters identifying with the left, and between 11 and 15 per cent identifying with the right or centre, and similar for the FN.¹¹ Again, we should be cautious in imputing too much accuracy to these figures, although the larger samples of over 1000 Republican primary voters in the successive waves of the 2017 CEVIPOF-Enquête Electorale Française (ENEF) suggest that these proportions were very stable over time—the period considered in the survey being March to November 2016. Whilst not a factor affecting the final outcome,¹² the presence of a significant minority of non-aligned voters for each primary adds, however, further uncertainty to the capacity of the bloc-focused vote intention polls to reflect the outcome.

Finally, the primary election takes place in a much narrower political space than a general election. As Jaffré (2016) suggests, primary voters have a very distinct sociological and ideological profile, which clearly separates them from the rest of the electorate and also from the non-primary voters within their own camp.¹³ Consequently, candidates tend to be in much greater proximity than each other, programmatically, and as a result, voters are much more easily able to change voting intention than they would be in a presidential election.¹⁴ Supporters of Nicolas Sarkozy, for example, could shift on a relative whim to François Fillon, other things being equal. As a result, variation in polling estimates could be relatively more likely to occur.

2.2 Presidential Polls

Across the two rounds of voting in the presidentials, polling performed very well, although two types of doubt were cast on the two rounds. At a basic forecast level, both first and second rounds were beyond reproach. First-round polling predicted the order of the first two candidates, who would proceed to the run-off ballot. They also estimated the two runners-up in the correct order, and very close to their actual scores.

There are two key ‘shocks’ in the election that polling picked up. First, François Bayrou’s announcement on 22 February that he would be endorsing Emmanuel Macron, rather than François Fillon, or indeed running as a Modem candidate himself, resulted in a substantial increase in support for the eventual president. As Fig. 6.3 shows, increase in support for Macron slightly predated this announcement, but it is clear that the ascent to beyond 25 per cent was conditional upon the Modem leader’s backing. Second, in the entre deux tours period, Marine Le Pen’s poor performance in the televised debate on 3 May with Macron resulted, as we discussed in the previous chapter, in a late slump in the polls. Some criticized the polls for underestimating Macron’s eventual winning margin. Or, put in a more sensationalist though mathematically identical way, the polls had overestimated Marine Le Pen’s support (see Fig. 6.4).

Fig. 6.3

Vote intention polling for the presidential first round. Source: Authors’ collation of vote intention polls, https://fr.wikipedia.org/wiki/Liste_de_sondages_sur_l’élection_présidentielle_française_de_2017, polling institute archives, polling institute archives

Fig. 6.4

Vote intention polling for the presidential second round. Source: Authors’ collation of vote intention polls, https://fr.wikipedia.org/wiki/Liste_de_sondages_sur_l’élection_présidentielle_française_de_2017, polling institute archives

Looking at the trend in polls directly before and after the debate, there is a clear inflection, with Macron’s support rising more steeply. However, only one of these polls was fielded on 5 May—the rest were in the field either the day immediately after the debate or, in the case of the rolling polls, included respondents who had been polled before the debate. Unless we expect the trends in public opinion to stabilize immediately after the polls stop, or reaction to the debate to only occur within a 48-hour window, the trend post-debate should continue across the subsequent 48 hours to polling day itself. As a snapshot of the days previous to the election, therefore, the score is likely accurate. Discussion with friends and neighbours, media discussion and voters’ own reflection are likely responsible for the shift from 5 May to 7 May.

More broadly, criticism of these polls being inaccurate, however, seems unfounded. As pollsters and academic forecasts alike go to great lengths to emphasize, polls are snapshots of public opinion at a point in time. They can be used to generate forecasts, but they are not forecasts in and of themselves. As polling day approaches, there is an expectation that voter preferences will become manifest and stabilize, and the polls should approach the final result. However, this requires the rider ‘other things being equal’. In a situation where, in the late campaign, a shock occurs that might influence public opinion, late swing may be beyond the reach of polls the fieldwork for which has taken place days earlier.

One final concern expressed before the election result was possible herding among the polling companies. As a statistical sample, one would expect a certain amount of variation in polling estimates as a natural outcome of sampling error. Where the amount of observed error in the polls is less than the predicting sampling error for surveys of a given size, suspicion inevitably arises that pollsters are adjusting methods in the light of other pollsters’ results. Inevitably, the occasional sample will regularly, if not frequently, produce an outlier result by chance. The inevitable tendency, however, is to consider this might be due to a flawed methodology, and try to correct it, rather than accept it as a random statistical outlier. As polls converge, pollsters could potentially be tempted to try to ‘correct’ even small variations. Should such corrections occur when polls are close to the margin of error, over time pollsters will start to converge excessively. For some analysts, polling among French polling companies did look overly consistent not to be the result of some herd-like behaviour.¹⁵ Given their strong performance, particularly in the first round, however, such concerns soon dissipated after the election.

A final step to assessing the accuracy of polls is to look at the difference from the final result across time. A number of measures of polling accuracy are available with varying applicability to multi-party races (Mosteller et al. 1949; Martin et al. 2005; Jennings and Wlezien 2016) but we choose to use the B measure (Arzheimer and Evans 2014) for comparability with our analysis of the 2012 elections (Evans and Ivaldi 2013). The B measure provides a single index (with Bw being a version weighted by candidate score) of polling accuracy for each poll based upon the actual result, with higher scores indicating greater inaccuracy. Whilst in theory the index can be constructed using all candidates’ scores separately, we plot the accuracy by polling institute of the first-round presidential election polls, using an index constructed from the five main candidates and combining all other candidates in an ‘other’ miscellaneous category.

Figure 6.5 presents the across-time B and Bw scores for the eight pollsters for whom we have sufficient polling data points. First, it is noticeable that the broad trends across the pollsters are very similar. We would expect inaccuracy to reduce as the election draws closer, but both the start and end index scores are similar across the eight pollsters. There is a slight rise in inaccuracy in the polls in early March, but the patterns in the polling scores are mostly notable for the absence of changes or inflections related to critical junctures. The reference line, 18 March, indicates the publication of the final roster of candidates where, at the equivalent date in 2012, pollsters either saw a marked increase or a marked decrease in polling inaccuracy (Evans and Ivaldi 2013: 141–142). In 2017, as Fig. 6.5 shows, this is not the case. For all pollsters, inaccuracy continues to decline, in many cases more steeply. In polling terms then, if we discount issues of possible herding (which could result in conformity as illustrated here) the accuracy measures simply confirm the relative success of the polls in the first round of the presidentials.

Fig. 6.5

Trends in polling accuracy for first-round presidential election polls. Note: Vertical line indicates 18 March 2017 (publication of official list of candidates). Source: As Fig. 6.3

3 From Unpredictable to Highly Predictable: The Legislative Election Polls

Throughout this book, we emphasize the disruption in the French political landscape both causing and caused by Emmanuel Macron’s victory. In forecast terms, this extends to the legislative elections as well as the presidential elections. A forecast model based upon time series data since 1981 on unemployment and incumbency variables at the departmental level, which had been run relatively successfully on both 2007 and 2012 (Arzheimer and Evans 2010; Evans and Ivaldi 2013), finds it difficult to cope with the centrist candidacy of Macron and LRM, given the absence of a significant centrist party with structurally defined predictors prior to 2017. Again, as we have noted for the presidential race, any bloc forecast for left or right cannot reasonably be replicated. In this regard, the interruption of the time series of bloc voting, even when extended to include a third FN bloc, plays havoc with the backlog of forecasting models.

One implication of this which has been looked at has been the apparent bypassing of the assumed institutional imperatives of the Fifth Republic—bipolar competition, alternation between left and right, marginalization and exclusion of the extremes. In the context of political forecasting, where stable context is required as a ceteris paribus condition for standard variables to be used to model the likely outcome, this proved a significant obstacle to any forecast endeavour. However, in the legislative elections which followed, the expected institutional effects remained very solid.

The distinction between mid-term and confirmatory elections has always pertained to legislatives, both in France and abroad (Shugart 1995; Dupoirier and Sauger 2010) but the realignment of the electoral calendar since 2002 to ensure that the legislative election followed the presidential, and the de facto continuation of that ordering given the lack of early dissolutions, has reinforced the sense of legislative elections coat-tailing on the presidential result and returning the Head of State a strong majority, even if the strength of that majority has varied.

2017 was no exception. From the first polls of vote intentions held after the presidential election, the LRM-Modem coalition led as the first placed label in the first round of the elections, with a slight lengthening of its lead in the latter stages of the legislative campaign.

We will discuss the elections themselves in greater detail in Chap. 9, but here it suffices to look at the accuracy of the polls in reflecting the eventual outcome. As Fig. 6.6 shows, the consolidation of LRM’s lead after the election of Macron shows a steady trend upwards. Across the other main parties, the left forecasts are relatively accurate, but both LR/UDI and the FN are overestimated. As we will consider in Chap. 9, this doubtless reflects for the former a successful ‘incursion’ of LRM into a moderate right electorate that had remained relatively loyal during the presidentials, but who had succumbed to a party in government with senior LR figures such as Edouard Philippe and Bruno Le Maire. For the FN, the trend downwards from near parity with LRM prior to the second round simply reflects, as with LFI, a steady erosion of optimism and support for parties neither of whose candidates had performed to expectations.

Fig. 6.6

Vote intention polling for the legislative first round. Source: Authors’ collation of vote intention polls, https://fr.wikipedia.org/wiki/Liste_de_sondages_sur_les_élections_législatives_françaises_de_2017, polling institute archives

In the lead-up to the presidential election, a key concern over Macron’s suitability was less his ‘fit’ to the presidential role, and more over his capacity to mobilize a majority, either through his own party or through a coalition of supportive parties.¹⁶ For many commentators, the inclusion of a party other than LRM, or indeed a majority formed by LR, would have immediately constituted a form of cohabitation¹⁷—something which the new electoral calendar had precisely been designed to avoid. Matthew Shugart and Robert Elgie have countered that cohabitation stricto sensu would have meant a prime minister from a party directly opposed to Macron, and no presidential party representation in the government—a highly unlikely event.¹⁸ In particular, honeymoon elections—what we have referred to here as confirmatory elections—in semi-presidential systems almost inevitably return a supportive majority (Evans and Ivaldi 2017).

In that respect, the voting intention polls exactly followed the institutionalist path, demonstrating the expected willingness of an electorate, who had in the majority supported Macron’s election as president, to return an ‘enabling’ majority. As Shugart demonstrates more completely, this honeymoon effect is visible in the relationship between legislative election performance of the presidential party and the presidential approval rating, itself a ‘honeymoon’ before political reality dawns on the electorate. In 2007, Nicolas Sarkozy’s UMP won 39.5 per cent of the vote in June, when his approval rating was 63 per cent (TNS-Sofres). In 2012, François Hollande’s PS won only 29.4 per cent of the vote, given an approval rating of 55 per cent. At 57 per cent in the same rating, we would therefore expect Macron’s LRM to return around the first-round vote share the party was polling in the week before the election¹⁹—which turned out to be the case.

Of course, the second-round results determine the seat share, and, as we will discuss in Chap. 9, the constellation of competition by circonscription is key in how party coordination and vote transfers determine the eventual winner. Unlike the United States, local polling at the level of circonscription is very unusual in France, and generally only occurs in constituencies with high-profile duels, for example, Marine Le Pen against Jean-Luc Mélenchon in Hénin-Beaumont in 2012 (11e circonscription in the Pas-de-Calais) or the FN Gilbert Collard against LRM’s Marie Sara in the 2e circonscription of Gard in 2017. In polling and forecasting terms, then, large assumptions about the distribution of votes in the second round and their aggregate conversion to seats have to be made—what has been described as une délicate alchimie for pollsters.²⁰

The effect of turnout through the 12.5 per cent rule for second-round participation, and party fragmentation and cooperation by circonscription cause issues in any election. The virtual elimination of any straight left–right duel through LRM’s competitive presence increased the uncertainty in 2017 by an order of magnitude. Even the events detailed in the previous chapter could potentially influence seat shares in either direction. Consequently, seat projections varied widely, with some predicting 450-plus seats for LRM-Modem,²¹ others more modest around the 400 seat mark,²² and others still closer to the eventual result, with forecasts around 350 seats.²³ In this respect, limiting oneself to ex-post-explanatory analysis of the outcome seems a safer option, and one we choose in Chap. 9.

4 Conclusion

Somewhat perversely, the election where the polls were remarkably accurate turned out to be the election where forecasting of the outcome was virtually impossible. In his guide to assessing the quality of a forecast (1985: 60ff), Lewis-Beck identifies six criteria—accuracy, lead time, usability, clarity, parsimony and specification. Accuracy is clearly primus inter pares—without this, any forecast fails. In this respect, polling aggregations worked very well. However, they fell down on lead time. Only in the final weeks of the election did the positioning of the lead candidates become clear. Polling proponents would quite rightly protest that the polls were not inaccurate, as they were simply reflecting the state of public opinion at the time. This is a useful reminder that polls per se are not forecasts, even if we are tempted to treat them as such.

As far as we can ascertain, all other approaches failed on the accuracy criterion. Big Data approaches failed largely on clarity, usability and parsimony, relying upon proprietary algorithms amassing vast quantities of variables. But in the longer term, structural models from previous elections have also fallen down on usability. Simply, the equations which performed with varying degrees of success on previous presidential elections were apparently inapplicable to the 2017 race. Whilst the more marginal models used to forecast Le Pen vote could potentially have been brought to bear in this election (even if they were not), the government/incumbent models were not applicable.

In that sense, polling approaches in the long run appear a more flexible forecasting tool, particular for multi-party systems like France. In the United States, even a ‘left-field’ candidate such as Donald Trump can be fitted easily into an incumbent/opposition model, given his nomination by the Republican Party against the ‘incumbent’ Democrat Hillary Clinton. Of course, success of polling approaches still relies upon the quality of the data, and the capacity of researchers to adjust these in the light of an understanding of polling’s likely biases. The quality of the data is also crucial given the undoubted role that polls can have in voters’ electoral decisions. As information about the state of competition, voters in particular who have not yet made up their mind may be influenced on whether or how to vote by what the polls report. In a tight race like the 2017 presidential first round, such influences can be important, and consequently the expectation that polls should indeed reflect the state of public opinion at a given time is a reasonable one. Even where data quality is high, the media’s use of polling data, and especially the temptation to portray small, margin-of-error fluctuations as substantial changes in a candidate’s success, does not reflect the status of electoral competition accurately, and needs to be resisted. In that regard, structural models with longer lead times and predictors set in stone months before the election have offered greater transparency and rigour, and their taking a back seat in 2017 has left a worrying hole in the French electoral forecasting time series.

However, it would seem hasty to write off structural models in the French case on the basis of 2017 alone. As we will discuss in the very close of this book in a more general sense, much depends on whether the centrist dynamic in place after Macron and LRM’s victory remains a stable realignment of the French political system, or whether precisely those elements which we have associated with the Fifth Republic—bipolar, two-bloc competition—reassert themselves in the longer run. If so, the structural political economic models which have served French forecasting so well to date will retain their usability. A similar concern applies to the focus of our next two chapters, namely the applicability of traditional models of vote choice in an explanatory sense. To what extent do the standard models of policy array and social–psychological determinants of voting behaviour fit the 2017 presidential race?