KeywordsAudit Report Performance Audit Audit Society Audit Office Audit Institution
Performance audit is an evaluation that is based on decisions made or goals established by the legislature; concerns the economy, efficiency, and effectiveness of public programs or organizations; and may be carried out throughout the whole public sector.
Performance audits examine the economy, efficiency, and effectiveness of governments’ programs, organizations, or activities for the purpose of improving transparency and accountability and making changes. Performance audit concerns both the local and central government, but central governmental audit institutions have often been dominant in the professional development of the performance audit, for example, by issuing standards. Performance audit spans activities in traditional financial auditing, inspection, and program evaluation. It can therefore be difficult to distinguish performance audit from traditional audit and program evaluation.
Performance audit is an independent examination of the economy, efficiency, and effectiveness of local and central governments’ programs, organizations, or activities for the purpose of enhancing transparency, facilitating accountability, and improving the outcomes. Economy, efficiency, and effectiveness (the 3Es) have several definitions (Metcalfe and Richards 1990), but some common conceptualizations will suffice here. Economy means making savings in actual resource inputs relative to planned resource inputs; efficiency means that it would not be possible to produce more outputs with the same inputs, or alternatively it would not be possible to reduce the inputs without also having to reduce the outputs; and effectiveness means, simply put, the matching between desired objectives and actual outputs. Some would also introduce equity as a fourth E, explicitly making clear that the distribution and financing of the outputs are also important for social welfare. An important issue in practice is that effectiveness requires both economy and efficiency.
The definition of effectiveness that assumes given objectives for all sectors and services is often an unrealistic assumption in practice, however. Instead of assuming given objectives and that organizations will pursue goals rationally, effectiveness is also conceptualized more broadly as innovativeness, adaptability, organizational learning, and a capacity to manage change.
Performance audit concerns both the local and central government, but central governmental audit institutions have often been dominant in the professional development of the performance audit, for example, by issuing standards, in particular the International Organization of Supreme Audit Institutions (INTOSAI). INTOSAI’s standards define performance audit as concerning the audit of economy, efficiency, and effectiveness embracing “(a) audit of the economy of administrative activities in accordance with sound administrative principles and practices, and management policies”; “(b) audit of the efficiency of utilization of human, nonfinancial and other resources, including examination of information systems, performance measures and monitoring arrangements, and procedures followed by audited entities for remedying identified deficiencies”; and “(c) audit of the effectiveness of performance in relation to achievement of the objectiveness of the audited entity, and audit of the actual impact of activities compared with the intended impact” (INTOSAI 2004, p. 11). INTOSAI also gives a shorter definition stating that: “Performance auditing is based on decisions made or goals established by the legislature, and it may be carried out throughout the whole public sector” (INTOSAI 2004, p. 11).
There are many types of performance audits, such as efficiency audit, program effectiveness audit, performance management capacity audit, performance information audit, risk assessment, best practice review, and general management review. Audit institutions may take on four typical auditor roles: the judge, the public accountant, the management consultant, and the evaluator. These auditor roles may affect the auditors’ identities as well as the organization of the performance auditing. The audits are often designed as projects. The issues or organizations that are selected for audit are chosen based on prior experience, political salience, economic importance, and risk assessments.
The impact of audit is a contested issue. The “audit society” proposition, based on observing the developments of an audit explosion in some countries, holds that organizations often use audits largely as rituals of verification that produce comfort (Power 1997). There is ongoing research on different dimensions of the impact of performance audit, including main effects on transparency, accountability, and improvement, as well as side effects and unintended consequences.
This article first discusses the difference between performance audit, traditional audit, and other evaluation. The article goes on to elaborate on different types of performance audits, the organization of performance auditing, performance audit products and the communication of these products, the impacts of performance audits, and challenges for performance audit. The last section concludes.
Separating Performance Audit from Traditional Audit and Program Evaluation
Performance audit is an evaluation activity of a government program or organization that spans activities in traditional financial auditing, inspection, and program evaluation. It can therefore often be difficult to distinguish performance audit from traditional audit and program evaluation. There are different images of government and good functioning, primary goals and dominant modes of review, as well as different roles of the reviewers between traditional audit, performance auditing, and program evaluation, which can be used in separating performance audits from traditional audits and program evaluations. Barzelay (1997) discusses three images: bureaucracies, value chains, and government interventions.
The image of the organization in traditional audit is typically a bureaucracy machine. The audit reviews the transactions of the organization with the environment and its task performance in order to see whether they are effectively regulated by systems. The primary goals of the review in the traditional audit are compliance to regulations and accountability to owners. The auditor’s role in traditional audit is verifying information, finding mismatches between the organizations’ practices and regulations, inferring consequences of the mismatches, and reporting the findings.
The image of the organization in performance auditing is the value chain. The value chain is a term often used by managers, where political scientist may use policy program, social scientists use social system, economists use production model, and evaluators use effect chain. The image of good functioning in performance auditing is that the organization and practices in converting inputs, through activities, into outputs and eventually outcomes, are optimal. The primary goals of the performance audit review are hence the performance of the value chain and its accountability. The dominant mode of the review is inspection, and the auditor’s role is evaluating different aspects of the value chain (program, system, production), including different organizations involved, and reporting the findings.
The image of the government in program evaluation is government-steered interventions in collective problems. The program evaluation therefore reviews whether the intervention, typically a public policy using different policy tools, achieves its goals, which are improving collective well-being and social welfare at large. The dominant mode of review in program evaluation is research. The auditor’s role in program evaluation is therefore the evaluation of the effectiveness of the interventions or measuring the impact of different interventions on the collective problems addressed as well as their side effects and unintended consequences.
Types of Performance Audits
Performance audit consists of many different types, largely compliance, activity, and performance-oriented types. A more detailed typology has seven different types: efficiency audit, program effectiveness audit, performance management capacity audit, performance information audit, risk assessment, best practice review, and general management review, all using different units of analysis, modes of review, scope of evaluation, and focus of effort (Barzelay 1997).
The efficiency audit analyzes organizational functions, processes, or program elements by inspecting different aspects of operations between a government or a public sector organization and “third parties.” It aims at identifying opportunities for lowering costs and reducing budgets in delivering program outputs. The program effectiveness audit inspects policies, programs, or major program elements that will affect performance of managerial functions. It aims at assessing the impact of public policies by evaluating the programs’ effectiveness. The performance management capacity audit inspects different governmental organizations that affect performance of managerial functions in the programs. The aim is assessing the organizations’ capacity to achieve the three generic goals of economy, efficiency, and effectiveness (3Es). The performance information audit audits a governmental organization’s nonfinancial information generated by the performance measurement and reporting system. Its aim is validating and attesting (verifying) the accuracy (reliability and validity) of the information provided by the organization’s performance management. The risk assessment inspects all aspects of a program’s design and operation. The aim is identifying major risks of program breakdown and their sources. The best practice review researches whole sectors, generic processes, or common functions in order to illuminate certain aspects of organizations and programs, such as their activities, processes, or outputs. It aims at formulating sector-specific standards of best practice and revealing relative performance of the participants in the review. Finally, the general management review inspects selected aspects of an organization’s structures, systems, and programs aiming for assessing the capacity of the organization to delivering upon its mission and policy mandates.
Hence, the different types of performance audits evaluate governmental programs and organizations’ processes and outputs with regard to economy, efficiency, and effectiveness; verifies information relating to the economy, efficiency, and effectiveness of these programs and organizations; as well as inspects the programs and organizations capacities to achieve goals, deliver on missions, and fulfill mandates with economy, efficiency, and effectiveness.
The Organization of Performance Audit
Besides the diversity of performance audit types, audit institutions may also take on several roles. There are four typical auditor roles (Pollitt et al. 1999): the judge/magistrate, the public accountant, the management consultant, and the researcher/scientist (evaluator). These roles will typically affect the auditors’ identities as well as the preferred organization of the performance auditing.
The organizational design of the performance audit may be conceived as having two dimensions: one dimension is related to how the production of the audit (the supply side) is organized, and the other dimension is related to how the users of the audits are organized (the demand side). A common typology uses four constitutionally different forms of Supreme Audit Institutions (SAIs): a court with a judicial function, a collegiate body without a judicial function, an audit office headed by an auditor-general within the executive government, and an audit office headed by an auditor-general outside the executive government, reporting to the parliament or a parliamentary subcommittee (Pollitt et al. 1999). The SAIs’ production forms of the performance audits may for simplicity be categorized into two groups: the courts of audit or collegiate body group and the national audit office form. The court of audit form is often used in Continental Europe in countries with a Rechtsstaat or Napoleonic law tradition, for example, in Belgium, France, Germany, the Netherlands, Portugal, and Spain. The national audit office form is often used in countries that were more active new public management reformers from the 1980s onward, such as Australia, the Baltic countries, New Zealand, the Nordic countries, the UK, and the USA. Some audit offices have been reorganized from being within the executive to being outside and reporting to the parliament in order to be more fully and formally independent from the executive.
The second dimension in the organizational design of the performance audits is related to how some of the main users of the audits, e.g., the parliaments, organize their handling of the audits. Also here two forms are discerned: the public accounts committee model (PAC model) and the non-PAC model. The PAC model means that the SAIs submit their audits to one specific, standing parliamentary committee, often termed the public accounts committee, as in the British parliament. The Westminster model in the UK has subsequently spread to many other countries, for example, to Norway. The PAC model results in powerful PACs and is effective for naming and shaming. Albeit seen as superior in pursuing accountability and holding ministers to account, the PAC model’s usefulness for enhancing organizational learning and contributing to improvement is uncertain. In the non-PAC model, the SAIs submit their audits to the parliament as such. With no apparent powerful “sponsor” within the parliament, the SAI may lose impact and seek alternative backers in order to disseminate their messages. In Sweden, using a non-PAC model, the relative newly formed SAI sought to use public media such as newspapers to disseminate their findings, a process that may have compromised the SAI’s legitimacy and perceived independence, at least temporarily (Bringselius 2014).
Considering that performance auditing span many types, the auditors may emphasize and perform different roles, the audit may be organized in different ways, and the auditors may have different professional backgrounds and identities. Traditional auditors used to be accountants, but the staff conducting performance auditing is often trained in social sciences, namely, political science, sociology, and anthropology, or in management and economics. Depending on training and departmental affiliation, the staff may take on different identities associated with the auditor roles, such as judges, accountants, management consultants, or evaluators. In addition, other stakeholders such as parliaments, ministries of finance, and private auditors may also have interests in the performance auditing in the public sector. There is therefore ongoing struggles or competition for jurisdictions, roles, and identities within the audit institutions and the government, as well as between auditors in the private and public sectors (Guthrie and Parker 1999; Jacobs 1998; Jeppesen 2012).
In order to institutionalize performance auditing in the established audit institutions’ practices, which often have a strong tradition of financial audit, many audit institutions have chosen to specialize by having some departments for financial audits and some departments for performance audits. There usually also are administrative departments and there may also be specific departments for methods and development. The audit institutions therefore have several strategic and organizational design choices when organizing their performance auditing. For example, a comparative study of four Nordic SAIs showed that among four strategic options – a performance auditing strategy, a financial auditing strategy, a portfolio strategy, and a hybrid strategy – three SAIs had adopted a portfolio strategy and one SAI had adopted a hybrid strategy (Jeppesen et al. 2017). Whether these configurations are stable or typical remains to be seen, and warrants more research, but there is considerable interest from consultants and accountants to overtake at least some of the performance auditing, not necessarily as traditional audits or as performance audits but as consulting.
Performance Audit’s Products and Communication
The performance audits are often decided by the auditors-general in the SAIs and by municipal control committees in the local government in order to upheld independence. Some audit institutions may allow other parties to suggest issues or organizations for audit, and some parliaments also reserve the right to be consulted on the choice of the audits. Typically, the audits are designed as projects, and the issues or organizations that are selected for auditing are chosen based on prior experience, political salience, economic importance, and risk assessments. If a preliminary analysis indicates an interesting issue, the performance audit goes forward using carefully designed methods. There may be several meetings and different methods of data collection, and the auditees are invited to comment on the draft reports in a “clearing process” before the final audit reports are submitted to the parliament (Keen 1999; Pollitt et al. 1999).
Politicians and policy-makers often experience information overload. In order to reach out to interested parties, as well as keeping the audit institutions’ independence, many auditors utilize media communication strategies. Such media communications strategies may seek to serve the SAI’s overall strategy, demonstrate the results of the SAI’s activities, facilitate the freedom to address different stakeholders, enhance impact of the audits, and enhance transparency and equal access to high-quality information. Audit institutions also complement their traditional product, the audit report, with different communication actions such as publishing annual reports, providing website archives, utilizing press offices, publishing press releases and – when there are especially important audits – holding press conferences, getting or participating in peer review reports (review of an audit institution by sister organizations), providing answers to frequently asked questions (FAQs), and utilizing social media such as Facebook, Twitter, YouTube, and Flickr, for disseminating information (González-Díaz et al. 2013).
Impacts of Performance Audits
The impacts of audit have been a contested issue since the mid-1990s. Echoing the development in new institutional sociological theory from the mid-1960s, the arguments are that many organizations adopt different and sometimes conflicting processes and structures that to some extent are rituals and symbols in order to conform to different and sometimes conflicting values and norms in the environment, more than heeding to effectiveness can explain. Such adaptations may result in decoupling between the technical operations of audit and its impacts and the image of the audit and its impacts. Studies of the US General Accounting Office (now the US Government Accountability Office) found complex relationships, but these relationships were more loose couplings than decoupling (Basu et al. 1999).
The audit society proposition, based on observing the developments in auditing in the UK, holds that organizations often use audits, but that the audits largely are rituals of verification that produce comfort (Power 1997). Part of this proposition is an effort to explain the large increase in the use of audits in different sectors of the society during the 1980s and 1990s, the so-called audit explosion. Some of this development can be explained by public management reforms that typically shifted emphasis from rules and hierarchy to outcomes and competition. These reforms often involved policies and programs for devolution of powers within the public sector and decentralization of authority and empowerment of managers within organizations. The increased emphasis on devolution, decentralization, and empowerment also unleashed a need for more control of outputs and outcomes in order to keep democratic control of public resources. Hence, policy tools for measuring and managing performance gained prominence in the public sector, including performance management, performance audit, and evaluation. Moreover, some argued that much of this development led to an increased use of accountants and consultants in audit in the public sector, but with largely symbolic outcomes. That the “audit explosion” has had tremendous impact is, however, little disputed. The audit society has forced many organizations and practices “to be made auditable” meaning that the entity needs to have clear formal boundaries, and the practices and processes often have to be codified and standardized, for example, to be amenable for scrutiny by ticking checklists in inspections, as well as changing the identities of the auditors from traditional accountants to performance auditors and modernizers (Skærbæk 2009). Much of this audit has produced comfort but has also produced discomfort when there has been discovered mismatch between practices and norms, performance below par, and misuse of public money.
Measuring and assessing the impacts of public policies and programs and their accompanying use of policy tools are notoriously difficult. This can partly explain why, approximately 20 years after the “audit explosion” in the mid-1990s, there are still few empirical studies of the impact of performance audit in the government (Van Loocke and Put 2011). Some of the few empirical studies that exist have, however, showed some positive but varying impacts of performance audit on public administration in different countries (see, e.g., Johnsen et al. 2001; Morin 2001, 2010, 2016; Raudla et al. 2015; Reichborn-Kjennerud 2013, 2014; Reichborn-Kjennerud and Johnsen 2011, 2015; Tillema and ter Bogt 2010). A comparison of performance audit practices in the Regional Audit Institutions (RAIs) and SAIs in Europe showed that there were institutional and maybe cultural factors determining some impacts. RAIs and SAIs that practiced performance audits in an Anglo-American model (the “UK-Nordic group”) or a Germanic model (the “Germanic group”) had positive impacts from the recommendations included in their performance audit reports. The Anglo-American model emphasizes following-up the audit processes and the implementation of the recommendations. The Germanic model has impacts based on parliaments’ promoting reforms. The RAIs and SAIs in the French group and in the “other” group had low performance audit activities, the audit reports were rarely presented in parliament, and there were little following-up of the audit recommendations. There was therefore little impact of the performance audits in these two groups (Torres et al. 2016).
Assessing the impacts of performance audits, defining which dimensions of impact one wants to assess, and measuring these dimensions with accurate data are conceptually and empirically challenging. The RAIs and SAIs themselves, however, regularly survey their auditees about the following-up of the performance audits sometime after the audits have been finished. These data are seldom made public, but would be an interesting data source for future research. The approach often used in the research literature is interviewing or surveying the auditees, but interviewing or surveying the auditors themselves about the impacts has also been done (Lapsley and Pong 2000).
Challenges for Performance Audits
There are some persistent challenges for many auditors. One challenge is the balancing act between independence from other stakeholders and responsiveness to their needs and inputs into the audit process (Lonsdale 2008). Some stakeholders may want to shape the focus or criteria for the audit, have valuable information for the audit, act as subcontractors, are involved in consultations during the audit process, or are important for disseminating the audit information (Jacobs 1998; Reichborn-Kjennerud and Johnsen 2011; Tillema and ter Bogt 2010). These issues may be perceived as having impacts for the auditors’ independence and the audit quality.
A second challenge is the need for being seen as independent in order to be able to be legitimate in publishing “bad news” or “naming and shaming” in a political and partisan context, while simultaneously having a mandate to contribute to improvements that effectually requires an ongoing and close relationship with the auditees (Barzelay 1997). Some research indicates that the impact on improvement varies between countries. How the auditors chose their auditor role, how they are organized, and how they design the audits may affect their perceived independence and impact of the performance audit on improvements.
A third challenge is to design and practice the audits in such a way that also corruption and financial misconduct are detected. Detecting corruption has been low on the auditors’ agenda, maybe because it is costly to detect corruption and requires investigative methods that traditionally have been seen as police work rather than audit. Corruption nevertheless affects economy, efficiency, and effectiveness, as well as infringes upon transparency and accountability. Therefore, detecting, if not investigating, corruption should fall within the remit of audit, including performance audit.
In the traditional audit, independent accountants verify organizations’ financial information in relation to standards in order for other parties, including owners, potential investors, governments, and the public at large, to be able to trust the information given by management. Performance audit is much more concerned with nonfinancial information and has often more ambiguous standards than traditional audit. Moreover, the purpose of the performance audit is often providing transparency in order for the parliament to be able to hold someone, typically the responsible minister, to account, as well as for facilitating organizational learning and improvement throughout the government. Despite hard to assess, all these purposes may have substantial impact on politics and public sector management, both through their direct effects and through their preventive effects. Therefore, maybe one of the most interesting as well as challenging issues in accounting and auditing is what would happen if there was no, or different, accounting and auditing regulations and practices.
- Guthrie JE, Parker LD (1999) A quarter of a century of performance auditing in the Australian federal public sector: a malleable masque. Abacus 35(3):302–332, http://onlinelibrary.wiley.com/doi/10.1111/1467-6281.00048/pdf CrossRefGoogle Scholar
- INTOSAI (2004) Standards and guidelines for performance auditing based on INTOSAI’s auditing standards and practical experience. ISSAI 3000. INTOSAI Professional Standards Committee, ViennaGoogle Scholar
- Jeppesen KK, Carrington T, Catasús B, Johnsen Å, Reichborn-Kjennerud K, Vakkuri J (2017) The strategic options of supreme audit institutions: the case of four Nordic countries. Finan Account Manage 32(2)Google Scholar
- Johnsen Å, Meklin P, Oulasvirta L, Vakkuri J (2001) Performance auditing in local government: an exploratory study of perceived efficiency of municipal value for money auditing in Finland and Norway. Eur Account Rev 10(3):583–599. doi.org/10.1080/09638180126803Google Scholar
- Lapsley I, Pong CKM (2000) Modernization versus problematization: value-for-money audit in public services. Eur Account Rev 9(4):541–567. doi:10.1080/713764876Google Scholar
- Metcalfe L, Richards S (1990) Improving public management, 2nd edn. Sage, LondonGoogle Scholar
- Power M (1997) The audit society: rituals of verification. Oxford University Press, OxfordGoogle Scholar
- Raudla R, Taro K, Agu C, Douglas JW (2015) The impact of performance audit on public sector organizations: the case of Estonia. Public Organ Rev. doi:10.1007/s11115-015-0308-0Google Scholar
- Reichborn-Kjennerud K (2013) Political accountability and performance audit: the case of the auditor general in Norway. Public Adm 91(3):680–695. doi:10.1111/padm.12025Google Scholar
- Reichborn-Kjennerud K, Johnsen Å (2015) Performance audit and supreme audit institutions’ impact on public administration: the case of the office of the auditor general in Norway. Adm Soc. doi:10.1177/0095399715623315Google Scholar
- Torres L, Yetano A, Pina V (2016) Are performance audits useful? A comparison of EU practices. Adm Soc. doi:10.1177/0095399716658500Google Scholar
- Van Loocke E, Put V (2011) The impact of performance audits: a review of the existing evidence. In: Lonsdale J, Wilkins P, Ling T (eds) Performance auditing: contributing to accountability in Democratic Government. Edward Elgar, Cheltenham, pp 175–208Google Scholar