Intelligent Evaluation and Performance Measurement in Public Health Policy and Public Service Systems



Healthcare organisations are not immortal, far from it. The service science approach highlights the role of service users in the service process. Their success and failure are dependent on performance measurement and evaluation. This chapter analyses the role of performance measurement and evaluation in the domain of intelligent policy making in the field of health. Traditionally, the role of evaluation and performance measurement has been central in the management doctrines and practices of public policy and public organisations. Performance measurement and evaluation are important features in the public policy process since they provide feedback information on how public authorities have attained the goals set for them. In this chapter we discuss the role and definition of key evaluation concepts within the framework of intelligent public policy and link these key evaluation concepts into a time-wise three-dimensional model of intelligent evaluative inquiry (ex ante, ex nunc and ex post). We aim to make a strong case for more intelligent public policy evaluation by addressing the simple fact that even though the world is practically full of information today, only part of it converts to knowledge for decision-makers and public policy makers. This chapter builds upon the evaluation practice related to ‘classical’ evidence-based medicine, addressing the role of evidence (albeit always contested to a certain extent) in fostering and nurturing decision-making in public policy and public organisations, but goes further towards public policy and public organisations. To this end, we stress the importance of deploying the best ideas of evidence-based medicine to strengthen policy- and organisation-based performance judgement. This chapter also explores the implications of this transformation for evaluation, performance monitoring and accountability, underlining that horizontal accountability referencing a wide democratic footprint is likely to become more explicit. Consequently, this chapter develops the idea of transformation of public sector performance management from the viewpoint of organisational intelligence. This chapter concludes that ‘hard nose’ rationalistic models of performance and evaluation are no longer fit for purpose in the health sector.


Public Policy Evaluation Public Sector Performance Management Public Healthcare Organizations Horizontal Accountability Vedung 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Aarons GA, Hurlburt M, McCue Horwitz S (2011) Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 38(1):4–23CrossRefPubMedGoogle Scholar
  2. Anderson RA, McDaniel RR Jr (2000) Managing healthcare organizations: where professionalism meets complexity science. Healthc Manag Rev 25(1):83–92CrossRefGoogle Scholar
  3. Argyris C (2005) Double-loop learning in organizations. A theory of action perspective. In: Smith KG, Hitt MA (eds) Great minds in management. The process of theory development. Oxford University Press, Oxford, pp 261–279Google Scholar
  4. Argyris C (2010) Organizational traps. Leadership, culture, organizational design. Oxford University Press, OxfordCrossRefGoogle Scholar
  5. Bauman AE, King L, Nutbeam D (2014) Rethinking the evaluation and measurement of health in all policies. Health Promot Int 29(1):143–151CrossRefGoogle Scholar
  6. Bendor J (2015) Incrementalism: dead yet flourishing. Public Adm Rev 75(2):194–205CrossRefGoogle Scholar
  7. Bianchi C (2015) Enhancing joined-up government and outcome-based performance management through system dynamics modelling to deal with wicked problems: the case of societal ageing. Syst Res Behav Sci 32:502–505. CrossRefGoogle Scholar
  8. Birkland TA (2016) An introduction to the policy process. Theories, concepts and models of public policy making. Routledge, New YorkGoogle Scholar
  9. Bort E, McAlpine R, Morgan G (2012) The silent crisis: failure and revival in local democracy in Scotland. The Jimmy Reid Foundation, GlasgowGoogle Scholar
  10. Bouckaert G, Halligan J (2008) Managing performance: international comparisons. Routledge, LondonGoogle Scholar
  11. Bouckaert G, Van Dooren W (2009) Performance measurement and management in public sector organizations. In: Bovaird T, Löffler E (eds) Public management and governance. Routledge, London, pp 151–164Google Scholar
  12. Bovens M (2005) Public accountability. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 182–209Google Scholar
  13. Bovens M (2010) Two concepts of accountability: accountability as a virtue and as a mechanism. West Eur Polit 33(5):946–967CrossRefGoogle Scholar
  14. Bovens M, Goodin RE, Schillemans T (eds) (2014) The Oxford handbook of public accountability. Oxford University Press, OxfordGoogle Scholar
  15. Cadilhac DA, Amatya B, Lalor E, Rudd A, Lindsay P, Asplund K (2012) Is there evidence that performance measurement in stroke has influenced health policy and changes to health systems? Stroke 43(12):3413–3420CrossRefPubMedGoogle Scholar
  16. Campbell DT, Russo MJ (1999) Social experimentation. Sage, Newbury ParkGoogle Scholar
  17. Clarke A (2006) Evidence-based evaluation in different professional domains: similarities, differences and challenges. In: Shaw IF et al (eds) The SAGE handbook of evaluation: policies, programs and practices. Sage, London, pp 559–581CrossRefGoogle Scholar
  18. Dahler-Larsen P (2006) Evaluation after disenchantment? Five issues shaping the role of evaluation in society. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Sage, Thousand Oaks, CA, pp 141–160Google Scholar
  19. Dahlgren D, Whitehead M (2007) European strategies for tackling social inequalities in health: levelling up part 2 [Online]. Accessed 23 Dec 2016
  20. De Jong GJ, Tesch Römer C (2012) Loneliness in old age in Eastern and Western European societies: theoretical perspectives. Eur J Ageing 9(4):285–295CrossRefGoogle Scholar
  21. Dunleavy P, Carrera L (2013) Growing the productivity of government services. Edward Elgar, CheltenhamCrossRefGoogle Scholar
  22. Ferlie EL, Fitzgerald G, McGivern DS, Bennett C (2011) Public policy networks and ‘wicked problems’: a nascent solution? Public Adm 89(2):307–324CrossRefGoogle Scholar
  23. Flynn N (2007) Public sector management. Sage, LondonGoogle Scholar
  24. Geuijen K, Moore M, Cederqvist A, Ronning R, van Twist M (2016) Creating public value in global wicked problems. Public Manag Rev, published online 25.8.2016. Accessed 27 Dec 2016
  25. Ghaffarzadegan D, Lyneisb J, Richardson GP (2011) How small system dynamics models can help the public policy process. Syst Dyn Rev 27(1):22–44Google Scholar
  26. Glennerster H (2002) United States poverty studies and poverty measurement: the past twenty-five years. Soc Serv Rev 76(1):83–107CrossRefGoogle Scholar
  27. Greene JC (2006) Evaluation, democracy and social change. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Policies, programs and practices. Sage, London, pp 118–140Google Scholar
  28. Greenhalgh T, Howick J, Maskrey N (2014) Evidence based medicine: a movement in crisis? BMJ 348:g3725 (published online 13.6.2014, Accessed 5 Nov 2016)Google Scholar
  29. Greve C (2015) Ideas in public management reform for the 2010s: digitalization, value creation and involvement. Public Organ Rev 15(1):49–65CrossRefGoogle Scholar
  30. Guba EG, Lincoln YS (1989) Fourth generation evaluation. Sage, Newbury ParkGoogle Scholar
  31. Harrison S, Smith C (2013) Neo-bureaucracy and public management: the case of medicine in the National Health Service. Compet Chang 7(4):243–254CrossRefGoogle Scholar
  32. Head BW (2008) Wicked policy problems. Public Policy 3(2):101–118Google Scholar
  33. Heclo H (1994) Poverty politics. In: Danziger SH et al (eds) Confronting poverty. Prescriptions for change. The Russell Sage Foundation/Harvard University Press, Cambridge, pp 396–437Google Scholar
  34. Hedström P, Swedberg R (eds) (1998) Social mechanisms: an analytical approach to social theory. Cambridge University Press, CambridgeGoogle Scholar
  35. Heinrich CJ (1999) Do government bureaucrats make effective use of performance management information? J Public Adm Res Theory 9(3):363–394CrossRefGoogle Scholar
  36. Hernes T (2014) A process theory of organization. Oxford University Press, OxfordCrossRefGoogle Scholar
  37. Hodges R (2012) Joined-up government and the challenges to accounting and accountability researchers. Financ Account Manag 28(1):26–51CrossRefGoogle Scholar
  38. Hood L, Flores M (2012) A personal view on systems medicine and the emergence of proactive P4 medicine: predictive, preventive, personalized and participatory. New Biotechnol 29(6):613–624CrossRefGoogle Scholar
  39. Howlett M, Migone A (2011) Charles Lindblom is alive and well and living in punctuated equilibrium land. Policy Soc 30(1):53–62CrossRefGoogle Scholar
  40. Hudson E, Nolan A (2015) Public healthcare eligibility and the utilisation of GP services by older people in Ireland. J Econ Ageing 6(4):24–43CrossRefGoogle Scholar
  41. Ikeda N, Saito E, Inoue M, Ikeda S, Satoh T, Wada K, Stickley A, Katanoda K, Mizoue T, Noda M, Iso H, Fujino Y, Sobue T, Tsugane S, Naghavi M, Ezatti M, Shibuya K (2011) What has made the population of Japan healthy? Lancet 378(9796):1094–1105CrossRefPubMedGoogle Scholar
  42. Innes JE, Booher DE (2016) Collaborative rationality as a strategy for working with wicked problems. Landsc Urban Plan 154(5):8–10CrossRefGoogle Scholar
  43. Kaivo-oja J, Virtanen P, Jalonen H, Stenvall J (2015) The effects of the internet of things and big data to organizations and their knowledge management practices. In: Luden L et al (eds) Knowledge management in organizations, vol 224. Springer, Heidelberg, pp 495–513CrossRefGoogle Scholar
  44. Kaivo-oja J, Virtanen P, Stenvall J, Jalonen H, Wallin J (2016) Future prospects for knowledge management in the field of health. KMO 2016. In: Proceedings of the 11th international knowledge management in organizations, Article no. 40., ACM, New York.
  45. Kaplan RS, Norton DP (1996) The balanced scorecard. Translating strategy into action. Harvard Business School Press, Boston, MAGoogle Scholar
  46. Karanikolos M, Mladovsky P, Cylus J, Thomson S, Basu S, Stuckler D, Mackenbach JP, McKee M (2013) Financial crisis, austerity and health in Europe. Lancet 381(9874):1323–1331CrossRefPubMedGoogle Scholar
  47. Kaufman H (1976) Are government organizations immortal? Brookings Institution Press, Washington, DCGoogle Scholar
  48. Kaufmann X-F (1987) Prevention and intervention in the analytical perspective of guidance. In: Hurrelmann K et al (eds) Social intervention: chances and constraints. Walter de Gruyter, New York, pp 3–20Google Scholar
  49. Koppell JGS (2005) Pathologies of accountability: ICANN and the challenge of “multiple accountabilities disorder”. Public Adm Rev 65(1):94–108CrossRefGoogle Scholar
  50. Kuhlmann E, Blank RH, Bourgeault IL, Wendt C (2015a) Healthcare policy and governance in international perspective. In: Kuhlmann E et al (eds) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New York, pp 3–19CrossRefGoogle Scholar
  51. Kuhlmann E, Blank RH, Bourgeault IL, Wendt C (eds) (2015b) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New YorkGoogle Scholar
  52. Kuhn T (2012) The structure of scientific revolutions. Chicago University Press, Chicago (originally published in 1962)Google Scholar
  53. Lane J-E (2005) Public administration and public management: the principal-agent perspective. Routledge, LondonGoogle Scholar
  54. Lee G, Park JY, Shin S-Y, Hwang JS, Jeong Ryu H, Lee JH, Bates DW (2015) Which users should be the focus of mobile personal health records? Analysis of user characteristics influencing usage of a tethered mobile personal health record. Telemed E-Health 22(5):419–428CrossRefGoogle Scholar
  55. Lewens T (2015) The meaning of science. Pelican Books, LondonGoogle Scholar
  56. Leyland AH, Groenewegen PP (2003) Multilevel modelling and public health policy. Scand J Public Health 31(4):267–274CrossRefPubMedGoogle Scholar
  57. Long A (2006) Evaluation of health services: reflections on practice. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Policies, programs and practices. Sage, London, pp 461–485Google Scholar
  58. March JG (1978) Bounded rationality, ambiguity and the engineering of choice. Bell J Econ 9(2):587–608CrossRefGoogle Scholar
  59. Mayne J (2008) Contribution analysis: an approach to exploring cause and effect. ILAC Brief 16 (May 2008).
  60. Michels A, Meijer A (2008) Safeguarding public accountability in horizontal government. Public Manag Rev 10(2):165–173CrossRefGoogle Scholar
  61. Mowery D, Smith HA, Cheney T, Bryan C, Conway M (2016) Identifying depression-related tweets from Twitter for public health monitoring. Online J Public Health Inf 8(1), Accessed 23 Dec 2016
  62. Naples NA (2014) Grassroots warriors. Activist mothering, community work, and the war on poverty. Routledge, LondonGoogle Scholar
  63. Neely A, Kennerly M, Walters A (2006) Performance measurement and management: public and private. Cranfield School of Management, CranfieldGoogle Scholar
  64. Newcomer KE, Hatry HP, Wholey JS (eds) (2015) Handbook of practical program evaluation. Jossey-Bass, New JerseyGoogle Scholar
  65. Niven PR (2011) Balanced scorecard. Step by step for government and non-profit agencies. Wiley, New JerseyGoogle Scholar
  66. Nørreklit H (2003) The balanced scorecard: what is the score? A rhetorical analysis of the balanced scorecard. Acc Organ Soc 28(6):591–619CrossRefGoogle Scholar
  67. North DC (2005) Understanding the process of economic change. Princeton University Press, Princetown, NJCrossRefGoogle Scholar
  68. Papanicolas I, Cylus J (2015) Comparison of healthcare systems performance. In: Kuhlmann E et al (eds) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New York, pp 116–132CrossRefGoogle Scholar
  69. Patton MQ (1997) Utilization-focused evaluation: the new century text. Sage, Thousand Oaks, CAGoogle Scholar
  70. Pawson R (2013) The science of evaluation. A realist manifesto. Sage, LondonCrossRefGoogle Scholar
  71. Pawson R, Tilley N (1997) Realistic evaluation. Sage, LondonGoogle Scholar
  72. Peters GB (2005) The problem of policy problems. J Comp Policy Anal Res Pract 7(4):349–370CrossRefGoogle Scholar
  73. Pfeffer J, Sutton RI (2006) Hard facts, dangerous half-truths and total nonsense. Profiting from evidence-based management. Harvard Business School Press, Boston, MAGoogle Scholar
  74. Porter ME, Olmsted Teisberg E (2006) Redefining health care. Creating value-based competition on results. Harvard Business School Press, Boston, MAGoogle Scholar
  75. Power M (1994) The audit explosion. Demos, LondonGoogle Scholar
  76. Power M (2005) The theory of audit explosion. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 326–344Google Scholar
  77. Robson C (2002) Real world research. Blackwell, OxfordGoogle Scholar
  78. Schillemans T (2011) Does horizontal accountability work? Evaluating potential remedies for the accountability deficit of agencies. Adm Soc 43(3):387–416CrossRefGoogle Scholar
  79. Scriven M (1991) Evaluation thesaurus. Sage, Newbury ParkGoogle Scholar
  80. Shadish WR, Luellen JK (2004) Donald Campbell: the accidental evaluator. In: Alkin MC (ed) Evaluation roots: tracing theorists’ views and influences. Sage, London, pp 80–87Google Scholar
  81. Shortell SM, Rundall TG, Hsu J (2007) Improving patient care by linking evidence based medicine and evidence-based management. JAMA 298(6):673–676CrossRefPubMedGoogle Scholar
  82. Silver H (1996) Culture, politics, and national discourses of the new urban poverty. In: Mingione E (ed) Urban poverty and the underclass. Blackwell, Oxford, pp 105–138CrossRefGoogle Scholar
  83. Stenvall J, Virtanen P (2015) Intelligent public organizations? Public Organ Rev, published on-line 2.12.2015.
  84. Talbot C (2005) Performance management. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 491–517Google Scholar
  85. Vakkuri J (2010) Struggling with ambiguity: public managers as users of NPM-oriented management instruments. Public Adm 88(4):999–1024CrossRefGoogle Scholar
  86. Vakkuri J (2013) Interpretive schemes in public sector performance: measurements creating managerial problems in local government. Int J Public Sect Perform Manag 2(2):156–174CrossRefGoogle Scholar
  87. Van Helden GJ, Johnsen Å, Vakkuri J (2012) Evaluating public sector performance management: the life cycle approach. Evaluation 18(2):159–175CrossRefGoogle Scholar
  88. Vedung E (1997) Public policy and program evaluation. Transaction, New BrunswickGoogle Scholar
  89. Vedung E (2010) Four waves of evaluation diffusion. Evaluation 16(3):263–277CrossRefGoogle Scholar
  90. Virtanen P, Laitinen I (2004) Beyond evaluation standards? Eur J Spat Dev (http:/ ISSN 1650-9544-Refereed articles Oct 2004 no 13
  91. Virtanen P, Stenvall J (2014) The evolution of public services from co-production to co-creation and beyond – an unfinished trajectory for the New Public Management? Int J Leadersh Public Serv 10(2):91–107CrossRefGoogle Scholar
  92. Virtanen P, Uusikylä P (2004) Exploring the missing links between cause and effect. A conceptual framework for understanding micro-macro conversions in programme evaluation. Evaluation 1(10):77–91CrossRefGoogle Scholar
  93. Virtanen P, Vakkuri J (2015) Searching for organizational intelligence in the evolution of public sector performance management. NISPAcee J Public Adm Policy 8(2):89–99Google Scholar
  94. Virtanen P, Vakkuri J (2016) Julkisen toiminnan tuloksellisuusarviointi [Public sector performance assessment and management, in Finnish]. Tietosanoma, HelsinkiGoogle Scholar
  95. Virtanen P, Kaivo-oja J, Ishino Y, Stenvall J, Jalonen H (2016a) Ubiquitous revolution, customer needs and business intelligence. Empirical evidence from Japanese healthcare sector. Int J Web Eng Technol 11(3):259–283CrossRefGoogle Scholar
  96. Virtanen P, Uusikylä P, Jalava J, Tiihonen S, Laitinen L, Noro K (2016c) Valtioneuvoston yhtenäisyys – kansainvälinen vertaileva tutkimus [available in only in Fiinish, The unity of government – an international comparative study]. Valtioneuvoston kanslia, Helsinki. Accessed at
  97. Walshe K, Rundall TG (2001) Evidence-based management: from theory to practice in health care. Milbank Q 79(3):429–457CrossRefPubMedPubMedCentralGoogle Scholar
  98. Weber EP, Khademian AN (2008) Wicked problems, knowledge challenges, and collaborative capacity builders in network settings. Public Adm Rev 68(2):334–349CrossRefGoogle Scholar
  99. Weiss C (1998) Evaluation. Prentice-Hall, New JerseyGoogle Scholar
  100. Wholey JS, Hatry HP (1992) The case for performance monitoring. Public Adm Rev 52(6):604–610CrossRefGoogle Scholar
  101. Winter S, Lasch R (2016) Recommendations for supplier innovation evaluation from literature and practice. Int J Oper Prod Manag 36(6):643–664CrossRefGoogle Scholar
  102. Yee AS (1996) The causal effects of ideas on policies. Int Organ 50(1):69–108CrossRefGoogle Scholar
  103. Zarefsky D (1986) President Johnson’s war on poverty. Rhetoric and history. The University of Alabama Press, TuscaloosaGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.The Finnish Innovation Fund SITRAHelsinkiFinland
  2. 2.School of ManagementUniversity of TampereTampereFinland

Personalised recommendations