Abstract
The paper discusses issues raised in Perl et al. (Policy Sci, 2018. https://doi.org/10.1007/s11077-018-9334-4), specifically the tensions between policy sciences’ search for ways reliably to link results to inputs and activities and the evidence they present for their conclusions that the three main approaches to analysis of policy processes do not focus upon this aspect of policy-making and are so well placed to cope with the ‘post-fact’ political era. The paper disagrees, arguing that this argument rather suggests that a main prop of modern political legitimacy—that policy can be reasonable and rational, and so not necessarily partisan—is under serious challenge, threatening a retreat to a politics of force and manipulation. The paper argues that this is best understood by appreciating that any empirical foundations of policy logics are not and never have been derived through a search for predictive power, no matter what is believed and taught, and told to those who pay for policy work, but are rather evidentially founded metaphors: theories. Correspondingly, in the norms of mainstream policy science, we cannot find any attempt to establish for a given empirical field whether there is or not adequate regularity to support assertions of (albeit with uncertainty) known outcomes. Given this, recent major failures of policy science’s expectations to bear fruit, despite assertions that evidentially based policy is reliably predictive, suggest that (amplified by academic interest in subjective aspects of knowledge construction) populist and popular shifts to reckless treatment of ‘facts’ appear as a not unreasonable reaction to ‘bad situations that policy-makers said would not happen’. The apparent success, reported by Perl et al. of the three main approaches to analysis of policy processes, then appear as somewhat irrelevant to those who pay for policy advice, if policy science is seen, not as a predictive science, but as one amongst many sources of political authority and so political order. The wise response by policy workers is then to reduce the ‘over-sell’ so as to restore their authority, in part by arguing that predictive power may be impossible (or would spend too much of the limited budget), so the mode of engagement should be non-instrumental action, and/or place far greater emphasis upon ensuring that those who are the objects of policy work are given adequate voice (lest they vote for Trump or Brexit). In that these options would imply less inconsistency, it would increase the authority of policy analysts, ceteris paribus.
Similar content being viewed by others
Notes
My first degree was in Engineering, and then I received M.Sc. and Ph.D. degrees in Economics. After a post-doc I then spent most of my career as a development consultant, close to policy work. My main scholarly field is contemporary Vietnam.
I am not suggesting that thorough efforts to ensure that analytical frameworks map well to empirics are not rigorous or fail to provide evidential basis for policy; rather, as I explain below, that a predictive criterion requires that theories contest ‘unto death’ as a matter of agreed normative procedure, so that the ‘multiple truths’ characteristic of many sciences that do not conventionally have that criterion are unacceptable. I read the quotation from Wilfred Owen’s poem above in this vein. My way of viewing prediction is I think innovative (Fforde 2017).
A wandering around the corridors of the ‘history of policy’ sections of any decent database can illuminate the wisdom of Kuhn’s remark that for an anomaly to matter it must be more than an anomaly, and often it is not (Kuhn 1970). For example, Levine and Zervos (1993) found that there were almost no robust relationships between policies and outcomes in the data space of international comparative economic growth analysis. Fforde (2005) looked at citations of this article and found that the great majority ignored this. Fforde (2018) reported that the pattern of structural change in developing countries globally since the early 1900 s has been, not industrialization, but servicisation, and that the faster the growth the more servicisation. Yet examination of all the World Bank’s World Development Reports showed that the term servicisation was not to be found, and a search of EBSCO Abstracts 1960–2015 recorded 6330 references to industrialization and 34 to servicisation (Fforde 2018: 4–5).
See, for example, a German military officer attempting to explain to a foreign audience the assumptions about knowledge inherent in the use of what he calls auftragstaktik, which assumes that combat is inherently chaotic, and so instrumental rationality at superior levels will tend to make you lose (Hoffman 1994). Contrast Freier et al. (2017) and see Simon (1986).
See Kenny and Williams (2001) for an argument that this assumption may be linked to assumptions that there is ontological and epistemological universalism—that things are the same, and to be understood in the same ways, everywhere. Statistically, this encourages assumptions that one is sampling from a single population; practically, it encourages belief that ‘what works there will work here’, and denial of local voice and ignorance of local specificities.
“Intermittent consulting work turned into a fulltime position at the World Bank during 1982–1986, as President of Economics and Research. Although the position allowed Krueger to play a direct role in the formulation and analysis of structural adjustment policies, the Bank had already made the shift toward the ``market-friendly'' approach before she arrived. At the Bank she made her largest impact on the research department, which went through a major transformation, possibly at the expense of driving away those who were not fully committed to the market-friendly approach.” (Rodgers and Cooley 1999: 1405).
Data comes from Hill et al. (2010) and are used in his exercise 5.13, p. 205. This data contains details about 1080 houses sold in Baton Rouge, Louisiana, during mid-2005. Again, full details are available from the author.
References
Cohen, J. (1994). The earth is round (p < 0.5). American Psychologist, 49(12), 997–1003.
Crombie, A. C. (1953). Robert Grosseteste and the origins of experimental science 1100–1700. Oxford: Clarendon Press.
Dunn, J. (2000). The cunning of unreason: Making sense of politics. New York, NY: Basic Books.
Fforde, A. (2005). Persuasion: Reflections on economics, data and the ‘homogeneity assumption’. Journal of Economic Methodology, 12(1), 63–91.
Fforde, A. (2013). Understanding development economics: Its challenge to development studies. London: Routledge.
Fforde, A. (2016). Confirmation bias: Methodological causes and a palliative response. Quality & Quantity. https://doi.org/10.1007/s11135-016-0389-z.
Fforde, A. (2017). The sceptical change agent: Reinventing development. New York: Palgrave.
Fforde, A. (2018). Yes, but what about services?—Is development doctrine changing? The Canadian Journal of Development Studies. https://doi.org/10.1080/02255189.2018.1410469.
Freier, N. P., et al. (2017). At our own peril: DOD risk assessment in a post-primacy world. Carlisle, PA: Strategic Studies Institute and US Army War College Press.
Held, D., et al. (1999). Global transformations (1st ed.). Cambridge: Polity Press.
Hill, R. C., Griffiths, W. E., & Lim, G. C. (2010). Principles of econometrics (4th ed.). Princeton: Princeton University Press.
Hoffman, K. (1994). Auftragstaktik: Mission-based leadership. Engineer, 24(4), 50–55.
Kenny, C., & Williams, D. (2001). What do we know about economic growth? Or, why don’t we know very much? World Development, 29(1), 1–22.
Kingdon, J. W. (1984). Agendas, alternatives and public policies. Boston: Little, Brown.
Kuhn, T. S. (1970). The structure of scientific revolutions. International encyclopedia of unified science (2nd enlarged ed.). Chicago: The University of Chicago Press.
Lasswell, H. D. (1956). The decision process: Seven categories of functional analysis. College Park, MD: University of Maryland.
Lasswell, H. D. (1971). A pre-view of policy sciences. New York: Elsevier.
Leamer, E. E. (1985). Sensitivity analyses would help. American Economic Review, 75(3), 308.
Leamer, E., & Leonard, H. (1983). Reporting the fragility of regression estimates. The Review of Economics and Statistics, 65(2), 306–317.
Levine, R., & Zervos, S. J. (1993). What have we learnt about policy and growth from cross country regressions? The American Economic Review, 82(2), 426–430.
McCloskey, D. M. (1985). The rhetoric of economics. Madison, WI: University of Wisconsin Press.
Nisbet, R. A. (1969). Social change and history: Aspects of the western theory of development. Oxford: Oxford University Press.
Perl, A., Howlett, M., & Ramesh, M. (2018). Policy-making and truthiness: Can existing models cope with politicized evidence and willful ignorance in a post-fact world? Policy Sciences. https://doi.org/10.1007/s11077-018-9334-4.
Rodgers, Y., & Cooley, J. (1999). Outstanding female economists in the analysis and practice of development economics. World Development, 27(8), 1397–1411.
Simon, H. A. (1986). Rationality in psychology and economics. Journal of Business, 59(4), S209–S224.
Acknowledgements
The author thanks Bob ‘RFI’ Smith for sustained support and discussion about the issues examined in this paper; mistakes remain entirely the author’s.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fforde, A. Yes, but what about the authority of policy analysts? A commentary and discussion of Perl et al., ‘Policy-making and truthiness: Can existing models cope with politicized evidence and willful ignorance in a post-fact world?’. Policy Sci 52, 153–169 (2019). https://doi.org/10.1007/s11077-018-9344-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11077-018-9344-2