Skip to main content
SpringerLink
Log in

Intelligent Evaluation and Performance Measurement in Public Health Policy and Public Service Systems

  • Chapter
  • First Online:
Intelligent Health Policy

  • 827 Accesses

Abstract

Healthcare organisations are not immortal, far from it. The service science approach highlights the role of service users in the service process. Their success and failure are dependent on performance measurement and evaluation. This chapter analyses the role of performance measurement and evaluation in the domain of intelligent policy making in the field of health. Traditionally, the role of evaluation and performance measurement has been central in the management doctrines and practices of public policy and public organisations. Performance measurement and evaluation are important features in the public policy process since they provide feedback information on how public authorities have attained the goals set for them. In this chapter we discuss the role and definition of key evaluation concepts within the framework of intelligent public policy and link these key evaluation concepts into a time-wise three-dimensional model of intelligent evaluative inquiry (ex ante, ex nunc and ex post). We aim to make a strong case for more intelligent public policy evaluation by addressing the simple fact that even though the world is practically full of information today, only part of it converts to knowledge for decision-makers and public policy makers. This chapter builds upon the evaluation practice related to ‘classical’ evidence-based medicine, addressing the role of evidence (albeit always contested to a certain extent) in fostering and nurturing decision-making in public policy and public organisations, but goes further towards public policy and public organisations. To this end, we stress the importance of deploying the best ideas of evidence-based medicine to strengthen policy- and organisation-based performance judgement. This chapter also explores the implications of this transformation for evaluation, performance monitoring and accountability, underlining that horizontal accountability referencing a wide democratic footprint is likely to become more explicit. Consequently, this chapter develops the idea of transformation of public sector performance management from the viewpoint of organisational intelligence. This chapter concludes that ‘hard nose’ rationalistic models of performance and evaluation are no longer fit for purpose in the health sector.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aarons GA, Hurlburt M, McCue Horwitz S (2011) Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 38(1):4–23

    Article  PubMed  Google Scholar 

  • Anderson RA, McDaniel RR Jr (2000) Managing healthcare organizations: where professionalism meets complexity science. Healthc Manag Rev 25(1):83–92

    Article  CAS  Google Scholar 

  • Argyris C (2005) Double-loop learning in organizations. A theory of action perspective. In: Smith KG, Hitt MA (eds) Great minds in management. The process of theory development. Oxford University Press, Oxford, pp 261–279

    Google Scholar 

  • Argyris C (2010) Organizational traps. Leadership, culture, organizational design. Oxford University Press, Oxford

    Book  Google Scholar 

  • Bauman AE, King L, Nutbeam D (2014) Rethinking the evaluation and measurement of health in all policies. Health Promot Int 29(1):143–151

    Article  Google Scholar 

  • Bendor J (2015) Incrementalism: dead yet flourishing. Public Adm Rev 75(2):194–205

    Article  Google Scholar 

  • Bianchi C (2015) Enhancing joined-up government and outcome-based performance management through system dynamics modelling to deal with wicked problems: the case of societal ageing. Syst Res Behav Sci 32:502–505. https://doi.org/10.1002/sres.2341

    Article  Google Scholar 

  • Birkland TA (2016) An introduction to the policy process. Theories, concepts and models of public policy making. Routledge, New York

    Google Scholar 

  • Bort E, McAlpine R, Morgan G (2012) The silent crisis: failure and revival in local democracy in Scotland. The Jimmy Reid Foundation, Glasgow

    Google Scholar 

  • Bouckaert G, Halligan J (2008) Managing performance: international comparisons. Routledge, London

    Google Scholar 

  • Bouckaert G, Van Dooren W (2009) Performance measurement and management in public sector organizations. In: Bovaird T, Löffler E (eds) Public management and governance. Routledge, London, pp 151–164

    Google Scholar 

  • Bovens M (2005) Public accountability. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 182–209

    Google Scholar 

  • Bovens M (2010) Two concepts of accountability: accountability as a virtue and as a mechanism. West Eur Polit 33(5):946–967

    Article  Google Scholar 

  • Bovens M, Goodin RE, Schillemans T (eds) (2014) The Oxford handbook of public accountability. Oxford University Press, Oxford

    Google Scholar 

  • Cadilhac DA, Amatya B, Lalor E, Rudd A, Lindsay P, Asplund K (2012) Is there evidence that performance measurement in stroke has influenced health policy and changes to health systems? Stroke 43(12):3413–3420

    Article  PubMed  Google Scholar 

  • Campbell DT, Russo MJ (1999) Social experimentation. Sage, Newbury Park

    Google Scholar 

  • Clarke A (2006) Evidence-based evaluation in different professional domains: similarities, differences and challenges. In: Shaw IF et al (eds) The SAGE handbook of evaluation: policies, programs and practices. Sage, London, pp 559–581

    Chapter  Google Scholar 

  • Dahler-Larsen P (2006) Evaluation after disenchantment? Five issues shaping the role of evaluation in society. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Sage, Thousand Oaks, CA, pp 141–160

    Google Scholar 

  • Dahlgren D, Whitehead M (2007) European strategies for tackling social inequalities in health: levelling up part 2 [Online]. http://www.thehealthwell.info/node/91930. Accessed 23 Dec 2016

  • De Jong GJ, Tesch Römer C (2012) Loneliness in old age in Eastern and Western European societies: theoretical perspectives. Eur J Ageing 9(4):285–295

    Article  Google Scholar 

  • Dunleavy P, Carrera L (2013) Growing the productivity of government services. Edward Elgar, Cheltenham

    Book  Google Scholar 

  • Ferlie EL, Fitzgerald G, McGivern DS, Bennett C (2011) Public policy networks and ‘wicked problems’: a nascent solution? Public Adm 89(2):307–324

    Article  Google Scholar 

  • Flynn N (2007) Public sector management. Sage, London

    Google Scholar 

  • Geuijen K, Moore M, Cederqvist A, Ronning R, van Twist M (2016) Creating public value in global wicked problems. Public Manag Rev, published online 25.8.2016. https://doi.org/10.1080/14719037.2016.1192163. Accessed 27 Dec 2016

  • Ghaffarzadegan D, Lyneisb J, Richardson GP (2011) How small system dynamics models can help the public policy process. Syst Dyn Rev 27(1):22–44

    Google Scholar 

  • Glennerster H (2002) United States poverty studies and poverty measurement: the past twenty-five years. Soc Serv Rev 76(1):83–107

    Article  Google Scholar 

  • Greene JC (2006) Evaluation, democracy and social change. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Policies, programs and practices. Sage, London, pp 118–140

    Google Scholar 

  • Greenhalgh T, Howick J, Maskrey N (2014) Evidence based medicine: a movement in crisis? BMJ 348:g3725 (published online 13.6.2014, Accessed 5 Nov 2016)

    Google Scholar 

  • Greve C (2015) Ideas in public management reform for the 2010s: digitalization, value creation and involvement. Public Organ Rev 15(1):49–65

    Article  Google Scholar 

  • Guba EG, Lincoln YS (1989) Fourth generation evaluation. Sage, Newbury Park

    Google Scholar 

  • Harrison S, Smith C (2013) Neo-bureaucracy and public management: the case of medicine in the National Health Service. Compet Chang 7(4):243–254

    Article  Google Scholar 

  • Head BW (2008) Wicked policy problems. Public Policy 3(2):101–118

    Google Scholar 

  • Heclo H (1994) Poverty politics. In: Danziger SH et al (eds) Confronting poverty. Prescriptions for change. The Russell Sage Foundation/Harvard University Press, Cambridge, pp 396–437

    Google Scholar 

  • Hedström P, Swedberg R (eds) (1998) Social mechanisms: an analytical approach to social theory. Cambridge University Press, Cambridge

    Google Scholar 

  • Heinrich CJ (1999) Do government bureaucrats make effective use of performance management information? J Public Adm Res Theory 9(3):363–394

    Article  Google Scholar 

  • Hernes T (2014) A process theory of organization. Oxford University Press, Oxford

    Book  Google Scholar 

  • Hodges R (2012) Joined-up government and the challenges to accounting and accountability researchers. Financ Account Manag 28(1):26–51

    Article  Google Scholar 

  • Hood L, Flores M (2012) A personal view on systems medicine and the emergence of proactive P4 medicine: predictive, preventive, personalized and participatory. New Biotechnol 29(6):613–624

    Article  CAS  Google Scholar 

  • Howlett M, Migone A (2011) Charles Lindblom is alive and well and living in punctuated equilibrium land. Policy Soc 30(1):53–62

    Article  Google Scholar 

  • Hudson E, Nolan A (2015) Public healthcare eligibility and the utilisation of GP services by older people in Ireland. J Econ Ageing 6(4):24–43

    Article  Google Scholar 

  • Ikeda N, Saito E, Inoue M, Ikeda S, Satoh T, Wada K, Stickley A, Katanoda K, Mizoue T, Noda M, Iso H, Fujino Y, Sobue T, Tsugane S, Naghavi M, Ezatti M, Shibuya K (2011) What has made the population of Japan healthy? Lancet 378(9796):1094–1105

    Article  PubMed  Google Scholar 

  • Innes JE, Booher DE (2016) Collaborative rationality as a strategy for working with wicked problems. Landsc Urban Plan 154(5):8–10

    Article  Google Scholar 

  • Kaivo-oja J, Virtanen P, Jalonen H, Stenvall J (2015) The effects of the internet of things and big data to organizations and their knowledge management practices. In: Luden L et al (eds) Knowledge management in organizations, vol 224. Springer, Heidelberg, pp 495–513

    Chapter  Google Scholar 

  • Kaivo-oja J, Virtanen P, Stenvall J, Jalonen H, Wallin J (2016) Future prospects for knowledge management in the field of health. KMO 2016. In: Proceedings of the 11th international knowledge management in organizations, Article no. 40. https://doi.org/10.1145/2925995.2926006, ACM, New York. http://dl.acm.org/icps.cfm

  • Kaplan RS, Norton DP (1996) The balanced scorecard. Translating strategy into action. Harvard Business School Press, Boston, MA

    Google Scholar 

  • Karanikolos M, Mladovsky P, Cylus J, Thomson S, Basu S, Stuckler D, Mackenbach JP, McKee M (2013) Financial crisis, austerity and health in Europe. Lancet 381(9874):1323–1331

    Article  PubMed  Google Scholar 

  • Kaufman H (1976) Are government organizations immortal? Brookings Institution Press, Washington, DC

    Google Scholar 

  • Kaufmann X-F (1987) Prevention and intervention in the analytical perspective of guidance. In: Hurrelmann K et al (eds) Social intervention: chances and constraints. Walter de Gruyter, New York, pp 3–20

    Google Scholar 

  • Koppell JGS (2005) Pathologies of accountability: ICANN and the challenge of “multiple accountabilities disorder”. Public Adm Rev 65(1):94–108

    Article  Google Scholar 

  • Kuhlmann E, Blank RH, Bourgeault IL, Wendt C (2015a) Healthcare policy and governance in international perspective. In: Kuhlmann E et al (eds) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New York, pp 3–19

    Chapter  Google Scholar 

  • Kuhlmann E, Blank RH, Bourgeault IL, Wendt C (eds) (2015b) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New York

    Google Scholar 

  • Kuhn T (2012) The structure of scientific revolutions. Chicago University Press, Chicago (originally published in 1962)

    Google Scholar 

  • Lane J-E (2005) Public administration and public management: the principal-agent perspective. Routledge, London

    Google Scholar 

  • Lee G, Park JY, Shin S-Y, Hwang JS, Jeong Ryu H, Lee JH, Bates DW (2015) Which users should be the focus of mobile personal health records? Analysis of user characteristics influencing usage of a tethered mobile personal health record. Telemed E-Health 22(5):419–428

    Article  Google Scholar 

  • Lewens T (2015) The meaning of science. Pelican Books, London

    Google Scholar 

  • Leyland AH, Groenewegen PP (2003) Multilevel modelling and public health policy. Scand J Public Health 31(4):267–274

    Article  PubMed  Google Scholar 

  • Long A (2006) Evaluation of health services: reflections on practice. In: Shaw IF et al (eds) The SAGE handbook of evaluation. Policies, programs and practices. Sage, London, pp 461–485

    Google Scholar 

  • March JG (1978) Bounded rationality, ambiguity and the engineering of choice. Bell J Econ 9(2):587–608

    Article  Google Scholar 

  • Mayne J (2008) Contribution analysis: an approach to exploring cause and effect. ILAC Brief 16 (May 2008). www.cgiar-ilac.org

  • Michels A, Meijer A (2008) Safeguarding public accountability in horizontal government. Public Manag Rev 10(2):165–173

    Article  Google Scholar 

  • Mowery D, Smith HA, Cheney T, Bryan C, Conway M (2016) Identifying depression-related tweets from Twitter for public health monitoring. Online J Public Health Inf 8(1), https://doi.org/10.5210/ojphi.v8i1.6561. Accessed 23 Dec 2016

  • Naples NA (2014) Grassroots warriors. Activist mothering, community work, and the war on poverty. Routledge, London

    Google Scholar 

  • Neely A, Kennerly M, Walters A (2006) Performance measurement and management: public and private. Cranfield School of Management, Cranfield

    Google Scholar 

  • Newcomer KE, Hatry HP, Wholey JS (eds) (2015) Handbook of practical program evaluation. Jossey-Bass, New Jersey

    Google Scholar 

  • Niven PR (2011) Balanced scorecard. Step by step for government and non-profit agencies. Wiley, New Jersey

    Google Scholar 

  • Nørreklit H (2003) The balanced scorecard: what is the score? A rhetorical analysis of the balanced scorecard. Acc Organ Soc 28(6):591–619

    Article  Google Scholar 

  • North DC (2005) Understanding the process of economic change. Princeton University Press, Princetown, NJ

    Book  Google Scholar 

  • Papanicolas I, Cylus J (2015) Comparison of healthcare systems performance. In: Kuhlmann E et al (eds) The Palgrave international handbook of healthcare policy and governance. Palgrave MacMillan, New York, pp 116–132

    Chapter  Google Scholar 

  • Patton MQ (1997) Utilization-focused evaluation: the new century text. Sage, Thousand Oaks, CA

    Google Scholar 

  • Pawson R (2013) The science of evaluation. A realist manifesto. Sage, London

    Book  Google Scholar 

  • Pawson R, Tilley N (1997) Realistic evaluation. Sage, London

    Google Scholar 

  • Peters GB (2005) The problem of policy problems. J Comp Policy Anal Res Pract 7(4):349–370

    Article  Google Scholar 

  • Pfeffer J, Sutton RI (2006) Hard facts, dangerous half-truths and total nonsense. Profiting from evidence-based management. Harvard Business School Press, Boston, MA

    Google Scholar 

  • Porter ME, Olmsted Teisberg E (2006) Redefining health care. Creating value-based competition on results. Harvard Business School Press, Boston, MA

    Google Scholar 

  • Power M (1994) The audit explosion. Demos, London

    Google Scholar 

  • Power M (2005) The theory of audit explosion. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 326–344

    Google Scholar 

  • Robson C (2002) Real world research. Blackwell, Oxford

    Google Scholar 

  • Schillemans T (2011) Does horizontal accountability work? Evaluating potential remedies for the accountability deficit of agencies. Adm Soc 43(3):387–416

    Article  Google Scholar 

  • Scriven M (1991) Evaluation thesaurus. Sage, Newbury Park

    Google Scholar 

  • Shadish WR, Luellen JK (2004) Donald Campbell: the accidental evaluator. In: Alkin MC (ed) Evaluation roots: tracing theorists’ views and influences. Sage, London, pp 80–87

    Google Scholar 

  • Shortell SM, Rundall TG, Hsu J (2007) Improving patient care by linking evidence based medicine and evidence-based management. JAMA 298(6):673–676

    Article  CAS  PubMed  Google Scholar 

  • Silver H (1996) Culture, politics, and national discourses of the new urban poverty. In: Mingione E (ed) Urban poverty and the underclass. Blackwell, Oxford, pp 105–138

    Chapter  Google Scholar 

  • Stenvall J, Virtanen P (2015) Intelligent public organizations? Public Organ Rev, published on-line 2.12.2015. https://doi.org/10.1007/s11115-015-0331-1

  • Talbot C (2005) Performance management. In: Ferlie E et al (eds) The Oxford handbook of public management. Oxford University Press, Oxford, pp 491–517

    Google Scholar 

  • Vakkuri J (2010) Struggling with ambiguity: public managers as users of NPM-oriented management instruments. Public Adm 88(4):999–1024

    Article  Google Scholar 

  • Vakkuri J (2013) Interpretive schemes in public sector performance: measurements creating managerial problems in local government. Int J Public Sect Perform Manag 2(2):156–174

    Article  Google Scholar 

  • Van Helden GJ, Johnsen Å, Vakkuri J (2012) Evaluating public sector performance management: the life cycle approach. Evaluation 18(2):159–175

    Article  Google Scholar 

  • Vedung E (1997) Public policy and program evaluation. Transaction, New Brunswick

    Google Scholar 

  • Vedung E (2010) Four waves of evaluation diffusion. Evaluation 16(3):263–277

    Article  Google Scholar 

  • Virtanen P, Laitinen I (2004) Beyond evaluation standards? Eur J Spat Dev (http:/www.nordregio.se/EJSD). ISSN 1650-9544-Refereed articles Oct 2004 no 13

  • Virtanen P, Stenvall J (2014) The evolution of public services from co-production to co-creation and beyond – an unfinished trajectory for the New Public Management? Int J Leadersh Public Serv 10(2):91–107

    Article  Google Scholar 

  • Virtanen P, Uusikylä P (2004) Exploring the missing links between cause and effect. A conceptual framework for understanding micro-macro conversions in programme evaluation. Evaluation 1(10):77–91

    Article  Google Scholar 

  • Virtanen P, Vakkuri J (2015) Searching for organizational intelligence in the evolution of public sector performance management. NISPAcee J Public Adm Policy 8(2):89–99

    Google Scholar 

  • Virtanen P, Vakkuri J (2016) Julkisen toiminnan tuloksellisuusarviointi [Public sector performance assessment and management, in Finnish]. Tietosanoma, Helsinki

    Google Scholar 

  • Virtanen P, Kaivo-oja J, Ishino Y, Stenvall J, Jalonen H (2016a) Ubiquitous revolution, customer needs and business intelligence. Empirical evidence from Japanese healthcare sector. Int J Web Eng Technol 11(3):259–283

    Article  Google Scholar 

  • Virtanen P, Uusikylä P, Jalava J, Tiihonen S, Laitinen L, Noro K (2016c) Valtioneuvoston yhtenäisyys – kansainvälinen vertaileva tutkimus [available in only in Fiinish, The unity of government – an international comparative study]. Valtioneuvoston kanslia, Helsinki. Accessed at www.tietokauttoon.fi

  • Walshe K, Rundall TG (2001) Evidence-based management: from theory to practice in health care. Milbank Q 79(3):429–457

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Weber EP, Khademian AN (2008) Wicked problems, knowledge challenges, and collaborative capacity builders in network settings. Public Adm Rev 68(2):334–349

    Article  Google Scholar 

  • Weiss C (1998) Evaluation. Prentice-Hall, New Jersey

    Google Scholar 

  • Wholey JS, Hatry HP (1992) The case for performance monitoring. Public Adm Rev 52(6):604–610

    Article  Google Scholar 

  • Winter S, Lasch R (2016) Recommendations for supplier innovation evaluation from literature and practice. Int J Oper Prod Manag 36(6):643–664

    Article  Google Scholar 

  • Yee AS (1996) The causal effects of ideas on policies. Int Organ 50(1):69–108

    Article  Google Scholar 

  • Zarefsky D (1986) President Johnson’s war on poverty. Rhetoric and history. The University of Alabama Press, Tuscaloosa

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Finnish Innovation Fund SITRA, Helsinki, Finland

    Petri Virtanen

  2. School of Management, University of Tampere, Tampere, Finland

    Jari Stenvall

Authors
  1. Petri Virtanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jari Stenvall
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Virtanen, P., Stenvall, J. (2018). Intelligent Evaluation and Performance Measurement in Public Health Policy and Public Service Systems. In: Intelligent Health Policy. Springer, Cham. https://doi.org/10.1007/978-3-319-69596-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-69596-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-69595-2

  • Online ISBN: 978-3-319-69596-9

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation