Skip to main content
Log in

Yes, but what about the authority of policy analysts? A commentary and discussion of Perl et al., ‘Policy-making and truthiness: Can existing models cope with politicized evidence and willful ignorance in a post-fact world?’

  • Discussion and Commentary
  • Published:
Policy Sciences Aims and scope Submit manuscript

Abstract

The paper discusses issues raised in Perl et al. (Policy Sci, 2018. https://doi.org/10.1007/s11077-018-9334-4), specifically the tensions between policy sciences’ search for ways reliably to link results to inputs and activities and the evidence they present for their conclusions that the three main approaches to analysis of policy processes do not focus upon this aspect of policy-making and are so well placed to cope with the ‘post-fact’ political era. The paper disagrees, arguing that this argument rather suggests that a main prop of modern political legitimacy—that policy can be reasonable and rational, and so not necessarily partisan—is under serious challenge, threatening a retreat to a politics of force and manipulation. The paper argues that this is best understood by appreciating that any empirical foundations of policy logics are not and never have been derived through a search for predictive power, no matter what is believed and taught, and told to those who pay for policy work, but are rather evidentially founded metaphors: theories. Correspondingly, in the norms of mainstream policy science, we cannot find any attempt to establish for a given empirical field whether there is or not adequate regularity to support assertions of (albeit with uncertainty) known outcomes. Given this, recent major failures of policy science’s expectations to bear fruit, despite assertions that evidentially based policy is reliably predictive, suggest that (amplified by academic interest in subjective aspects of knowledge construction) populist and popular shifts to reckless treatment of ‘facts’ appear as a not unreasonable reaction to ‘bad situations that policy-makers said would not happen’. The apparent success, reported by Perl et al. of the three main approaches to analysis of policy processes, then appear as somewhat irrelevant to those who pay for policy advice, if policy science is seen, not as a predictive science, but as one amongst many sources of political authority and so political order. The wise response by policy workers is then to reduce the ‘over-sell’ so as to restore their authority, in part by arguing that predictive power may be impossible (or would spend too much of the limited budget), so the mode of engagement should be non-instrumental action, and/or place far greater emphasis upon ensuring that those who are the objects of policy work are given adequate voice (lest they vote for Trump or Brexit). In that these options would imply less inconsistency, it would increase the authority of policy analysts, ceteris paribus.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. My first degree was in Engineering, and then I received M.Sc. and Ph.D. degrees in Economics. After a post-doc I then spent most of my career as a development consultant, close to policy work. My main scholarly field is contemporary Vietnam.

  2. I am not suggesting that thorough efforts to ensure that analytical frameworks map well to empirics are not rigorous or fail to provide evidential basis for policy; rather, as I explain below, that a predictive criterion requires that theories contest ‘unto death’ as a matter of agreed normative procedure, so that the ‘multiple truths’ characteristic of many sciences that do not conventionally have that criterion are unacceptable. I read the quotation from Wilfred Owen’s poem above in this vein. My way of viewing prediction is I think innovative (Fforde 2017).

  3. A wandering around the corridors of the ‘history of policy’ sections of any decent database can illuminate the wisdom of Kuhn’s remark that for an anomaly to matter it must be more than an anomaly, and often it is not (Kuhn 1970). For example, Levine and Zervos (1993) found that there were almost no robust relationships between policies and outcomes in the data space of international comparative economic growth analysis. Fforde (2005) looked at citations of this article and found that the great majority ignored this. Fforde (2018) reported that the pattern of structural change in developing countries globally since the early 1900 s has been, not industrialization, but servicisation, and that the faster the growth the more servicisation. Yet examination of all the World Bank’s World Development Reports showed that the term servicisation was not to be found, and a search of EBSCO Abstracts 1960–2015 recorded 6330 references to industrialization and 34 to servicisation (Fforde 2018: 4–5).

  4. See, for example, a German military officer attempting to explain to a foreign audience the assumptions about knowledge inherent in the use of what he calls auftragstaktik, which assumes that combat is inherently chaotic, and so instrumental rationality at superior levels will tend to make you lose (Hoffman 1994). Contrast Freier et al. (2017) and see Simon (1986).

  5. See Kenny and Williams (2001) for an argument that this assumption may be linked to assumptions that there is ontological and epistemological universalism—that things are the same, and to be understood in the same ways, everywhere. Statistically, this encourages assumptions that one is sampling from a single population; practically, it encourages belief that ‘what works there will work here’, and denial of local voice and ignorance of local specificities.

  6. “Intermittent consulting work turned into a fulltime position at the World Bank during 1982–1986, as President of Economics and Research. Although the position allowed Krueger to play a direct role in the formulation and analysis of structural adjustment policies, the Bank had already made the shift toward the ``market-friendly'' approach before she arrived. At the Bank she made her largest impact on the research department, which went through a major transformation, possibly at the expense of driving away those who were not fully committed to the market-friendly approach.” (Rodgers and Cooley 1999: 1405).

  7. Data comes from Hill et al. (2010) and are used in his exercise 5.13, p. 205. This data contains details about 1080 houses sold in Baton Rouge, Louisiana, during mid-2005. Again, full details are available from the author.

References

  • Cohen, J. (1994). The earth is round (p < 0.5). American Psychologist, 49(12), 997–1003.

    Article  Google Scholar 

  • Crombie, A. C. (1953). Robert Grosseteste and the origins of experimental science 1100–1700. Oxford: Clarendon Press.

    Google Scholar 

  • Dunn, J. (2000). The cunning of unreason: Making sense of politics. New York, NY: Basic Books.

    Google Scholar 

  • Fforde, A. (2005). Persuasion: Reflections on economics, data and the ‘homogeneity assumption’. Journal of Economic Methodology, 12(1), 63–91.

    Article  Google Scholar 

  • Fforde, A. (2013). Understanding development economics: Its challenge to development studies. London: Routledge.

    Google Scholar 

  • Fforde, A. (2016). Confirmation bias: Methodological causes and a palliative response. Quality & Quantity. https://doi.org/10.1007/s11135-016-0389-z.

    Google Scholar 

  • Fforde, A. (2017). The sceptical change agent: Reinventing development. New York: Palgrave.

    Google Scholar 

  • Fforde, A. (2018). Yes, but what about services?—Is development doctrine changing? The Canadian Journal of Development Studies. https://doi.org/10.1080/02255189.2018.1410469.

    Google Scholar 

  • Freier, N. P., et al. (2017). At our own peril: DOD risk assessment in a post-primacy world. Carlisle, PA: Strategic Studies Institute and US Army War College Press.

    Google Scholar 

  • Held, D., et al. (1999). Global transformations (1st ed.). Cambridge: Polity Press.

    Google Scholar 

  • Hill, R. C., Griffiths, W. E., & Lim, G. C. (2010). Principles of econometrics (4th ed.). Princeton: Princeton University Press.

    Google Scholar 

  • Hoffman, K. (1994). Auftragstaktik: Mission-based leadership. Engineer, 24(4), 50–55.

    Google Scholar 

  • Kenny, C., & Williams, D. (2001). What do we know about economic growth? Or, why don’t we know very much? World Development, 29(1), 1–22.

    Article  Google Scholar 

  • Kingdon, J. W. (1984). Agendas, alternatives and public policies. Boston: Little, Brown.

    Google Scholar 

  • Kuhn, T. S. (1970). The structure of scientific revolutions. International encyclopedia of unified science (2nd enlarged ed.). Chicago: The University of Chicago Press.

    Google Scholar 

  • Lasswell, H. D. (1956). The decision process: Seven categories of functional analysis. College Park, MD: University of Maryland.

    Google Scholar 

  • Lasswell, H. D. (1971). A pre-view of policy sciences. New York: Elsevier.

    Google Scholar 

  • Leamer, E. E. (1985). Sensitivity analyses would help. American Economic Review, 75(3), 308.

    Google Scholar 

  • Leamer, E., & Leonard, H. (1983). Reporting the fragility of regression estimates. The Review of Economics and Statistics, 65(2), 306–317.

    Article  Google Scholar 

  • Levine, R., & Zervos, S. J. (1993). What have we learnt about policy and growth from cross country regressions? The American Economic Review, 82(2), 426–430.

    Google Scholar 

  • McCloskey, D. M. (1985). The rhetoric of economics. Madison, WI: University of Wisconsin Press.

    Google Scholar 

  • Nisbet, R. A. (1969). Social change and history: Aspects of the western theory of development. Oxford: Oxford University Press.

    Google Scholar 

  • Perl, A., Howlett, M., & Ramesh, M. (2018). Policy-making and truthiness: Can existing models cope with politicized evidence and willful ignorance in a post-fact world? Policy Sciences. https://doi.org/10.1007/s11077-018-9334-4.

    Google Scholar 

  • Rodgers, Y., & Cooley, J. (1999). Outstanding female economists in the analysis and practice of development economics. World Development, 27(8), 1397–1411.

    Article  Google Scholar 

  • Simon, H. A. (1986). Rationality in psychology and economics. Journal of Business, 59(4), S209–S224.

    Article  Google Scholar 

Download references

Acknowledgements

The author thanks Bob ‘RFI’ Smith for sustained support and discussion about the issues examined in this paper; mistakes remain entirely the author’s.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adam Fforde.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fforde, A. Yes, but what about the authority of policy analysts? A commentary and discussion of Perl et al., ‘Policy-making and truthiness: Can existing models cope with politicized evidence and willful ignorance in a post-fact world?’. Policy Sci 52, 153–169 (2019). https://doi.org/10.1007/s11077-018-9344-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11077-018-9344-2

Keywords

Navigation