1 Introduction

Information Technology projects fail, and the cost of these failures is staggering (for example; Engelbrecht et al. 2017; Hidding and Nicholas 2017; Hughes et al. 2017; Hughes et al. 2016a, 2016b; Standish Group 1994 to 2015). This concern has been highlighted and repeated for more than forty years (see; Davis 1974; Lucas 1981; Maddison et al. 1983; Avison and Fitzgerald 2003; Hoffer et al. 1998; Lauden and Lauden 1998; Hawryszkiewycz 2001; Nickerson 2001).

Research has proposed a host of different reasons to explain project failure (Prater et al. 2017; Ewusi-Mensah 1997; Baccarini et al. 2004; Al Neimat 2005; Al Ahmed et al. 2009). Recent research by the Standish Group (2017) has found that ‘development projects that exceed $100 million in labor costs, only 2% are successful, meaning on-time and within budget. Another 51% are considered challenged or over budget, behind schedule or didn’t meet user expectations. The rest, 47%, are seen as outright failures’ (Thibodeau 2017, para 5).

One of the reasons for explaining this high rate of failure has been assumed as due to shortcomings in generic project management capacity, rather than due to attributes of IT projects in particular. For example, according to Hidding and Nicholas (2017, p. 81), ‘most of the improvement efforts have focused on advancing variations of the traditional project management paradigm, such as (that which) is embodied by the Project Management Body of Knowledge’.

Two questions arise regarding IT project failure research. First, why is the success rate of IT projects so poor? And secondly, why, despite the efforts of many, the situation fails to improve? This problem is known as ‘Cobb’s Paradox’ (Bourne 2011). Cobb’s Paradox states: ‘We know why projects fail; we know how to prevent their failure—so why do they still fail?’. Cobb made the observation in 1995 while attending a presentation by the Standish Group (authors of the Chaos series of reports) while working at the Secretariat of the Treasury Board of Canada. Cobb’s observation that “we know why projects fail” should not be taken in a literal, completely black and white sense, rather it should be considered to be a reference to the collective body of expert commentary, opinion, research and project practitioners that have offered solutions. Despite the successful implementation of major IT projects, repeatable success continues to be elusive (Thibodeau 2017).

Cobb was not alone in observing that there is a great deal studied and written about project failure, and that consulting firms propose methodologies and remedies but little actual progress appears to have been made. The International Federation for Information Processing (IFIP) Working Party 8.6 ran a conference to address this specific issue asking ‘why our scholarship has not been more effective. Is the fault one of theory and inadequate understanding? Or is the problem one of knowledge transfer, the failure to embed research knowledge in the working practices of managers and policy-makers’ (Dwivedi et al. 2015a, 2015b).

This study reports on the Queensland Health payroll project. Queensland is a state of Australia, located on the north-east coast. Queensland has a population approaching five million persons and covers an area of almost two million square-kilometres. The most famous tourist attraction is the Great Barrier Reef. Queensland Health employs 65,000 persons, and has an operating budget of AUD$11 Billion annually. Queensland has more than two hundred hospitals and health care facilities.

The primary question of this research is why. Why despite all of the experience, the research, and the training that is available, the consultants and software companies focussing attention on IT projects and the billions upon billions of dollars spent, large scale IT projects continue to fail at a rate that appears little changed over the decades.

2 Findings

When examining the Queensland Health payroll project files there are clear and obvious factors, which can be identified as having either not occurred or had been executed poorly and could be considered the causes of project failure. Any objective assessment of the project would conclude that project management had failed, there was a lack of requirements definition even though it was the first contracted deliverable, and management across all layers of the project were in conflict. These are all of the issues that appear in the literature on failed projects, and appear to confirm previous research.

Of potential significance is that the evidence provided by witness statements mapped to the project chronology showed that issues related to the identified themes were raised by staff and consultants throughout the project phases, and yet they still they remained as issues that were not resolved nor remediated at the time they were raised. The evidence is that management was made aware of these failures. So it was not a lack of awareness or communication of the failure risks, and therefore highlighting these as the only contributory factors of project failure lacks explanatory completeness, as the issue related to the inability to act on the concerns suggests other contributing factors to project failure.

The incoming Executive Director who oversaw the commencement of the project and managed the first few years had the exit report from the immediately preceding whole-of-government project produced by the external consultants that provided stark warnings of how that project had failed and what was required to ensure the next project would not fail. The only conclusion that can be drawn is that this report was ignored in its totality.

To paraphrase Cobb’s Paradox (Bourne 2011) the management of the Queensland Health payroll project should have known why their project was certain to end in failure, yet they failed to act appropriately thereby ensuring that the project did in fact fail, and spectacularly. As was evident from the analysis of the witness statements in the conduct of the Queensland Health Payroll project - the management was regularly informed of what was going on with their project by both staff and external consultants (WS013). Management knew that the project was facing problems (or at least should have known). The reports on the 2005 whole-of-government initiative (WS039), the KPMG Report (WS003), the KJ Ross report on testing (PD103), the IBM and CorpTech report to ‘reconstruct’ the business requirements (PD063) and the 2009 Queensland Audit Office report (PD108) all provided clear statements identifying where the project was failing and what needed to be done to remedy the situation. Yet the problems persisted until the total project costs had blown out to beyond A$1 billion. Faced with the clear and certain statement that the project was performing badly, and with specific statements of where the project was failing, successive managements failed to act appropriately to stem the problems. The only conclusion that can be drawn from this failure to act is that senior executives of the Department, the Governance and steering committees, the Executive Director did not know what specific actions were available to them, or what they specifically needed to do in order to be effective. The management and oversight of this project were at a complete loss as to how to effectively manage an information technology project.

This research proposes that the following are the contributory factors that led to the Queensland Health Payroll project becoming a failure:

  • a lack of domain expertise by senior management responsible for the project as evidenced by the inability or unwillingness to adopt appropriate governance processes;

  • stakeholders remained in conflict throughout the life of the project;

  • there was a complete lack of accountability for failure evident throughout the project and especially when it came to vendor and contract management.

It is not immediately obvious why this situation was allowed to unfold in the manner in which it did. The project appeared to comply with all the appropriate governance structures and reporting requirements, yet an historical or retrospective view would allow that the project was never managed effectively.

Indeed, the findings of the Commission of Inquiry (WS122) state that ‘Its (Queensland Health payroll) failure, attended by enormous cost, damage to government and impact on workforce, may be the most spectacular example of all the unsuccessful attempts to impose a uniform solution on a highly complicated and individualised agency’ (WS122 p. 10). The Commissions conclusion was that there were two primary causes for the failure of the payroll project (1) ‘unwarranted urgency’ and (2) a ‘lack of diligence on behalf of State officials’. (WS122 p. 217). The Commissions Report elaborated further on lack of diligence, describing it as ‘poor decisions made in scoping the Interim Solution, in their Governance of the project, and in failing to hold IBM to account’ (WS122 p. 217). The Commissioner further reported that ‘the problems are systemic to government and to the natural commercial self-interest of vendors’ (WS122 p. 218) which supports the observation that Normalisation of Deviance (Vaughan 2016) was at play throughout the conduct of this project. However, these findings by the Commission do not explain what motivated senior management to ignore the lessons learned from immediately preceding projects, and to ignore the warnings and advice of their own personnel. It is unclear, from the Commissions’ report, what specific steps a subsequent project might implement to ensure that they too did not all into these traps.

2.1 Situational Incompetence

The question of most concern to this researcher has been to uncover why, despite all of the research, publications, education, training and certification that is available to individuals and organisations undertaking project management of an information technology solution, a project could still display all of the mistakes, errors and failings that have been identified in the literature.

The theme that was the most consistent throughout the project was that senior management was repeatably made aware of project risks and failings. Reports had been written about the whole-of-government project prior to the creation of the Queensland Health project that specifically enumerated the challenges and risks that needed to be kept front of mind to the QH project team (WS003, WS004). The literature provided no plausible explanation to describe the fact that senior executives responsible for the direct execution of the project, and departmental executives with governance and oversight accountability apparently ignored all of the advice that they were presented with.

What emerged from the data was that the executives in charge of the project, those executives that operated above the hands-on technical level, were manifestly incompetent when it came to issues of information systems project management. The executives simply did not understand the information that was being presented to them, and interpreted professional concerns raised by Queensland Health team members as “personality conflicts”. These executives were presented with several formal reports outlining risks and issues, and acted in a manner that, under conventional wisdom, would defy rational explanation - the witness statements and project documents provide no evidence of any action being taken to address the issues raised. On more than one occasion IBM complained that employees of Queensland Health were trying to hold IBM to its contract and make IBM meet its obligations. IBM convinced senior departmental management that these staff were interfering in the project and senior management subsequently ordered their removal from the project.

Engelbrecht et al. (2017) suggest that inexperienced managers will seek advice and guidance from inappropriate sources. Kruger and Dunning (2009) offer the observation that the unskilled and unaware (Ryvkin et al. 2012) are incapable of identifying their own failings, incapable of independently observing and learning from the competence of others, and incapable of identifying competence in others.

These findings have led this researcher to postulate a new theory: Situational Incompetence.

Situational Incompetence applies when an otherwise experienced executive is placed in a position of authority or accountability for which they lack experience, training or specific skills. In this new role they are effectively incompetent and incapable of providing reasoned advice, guidance or management.

Situational Incompetence has implication for how leaders are selected for complex tasks requiring specialist IT domain knowledge and technical competence, it may also apply to other disciplines requiring specific knowledge of unique technology in those domains (e.g.: science, technology, engineering, medicine, and maths).

3 Research Methods

The corpus of published literature on the subject of failed IT projects lacks evidence based research drawn from comprehensive case studies (Dwivedi et al. 2015b). This research addresses that gap, and aims to identify what occurred in a specific, very large project, and what led to failure in that instance. From this case study it is hoped that confirmation of previously identified contributory factors may emerge, or else that a new theory may be constructed leading to further research that might confirm these findings as being generally applicable. A single case study, even one as complex as the Queensland Health Payroll project, is still only a singular event and cannot produce outcomes which are generalisable. But this case is of ‘very special interest … and the study of (it’s) particularity and complexity … to understand its activity within important circumstances’ (Stake 1995, p xi) is worthy of being undertaken.

Thus, this research needs to follow an approach that will lead to formative ‘theory building’ rather than the more common ‘theory testing’ (Eisenhardt and Graebner 2007). Theory building is more suited to a comprehensive case study approach, which would subsequently lead to future research opportunities to test any emergent or confirmatory hypothesis. The goal of this research is to “understand more about the reasons why (project failure) occurs” (Keil 1995, p. 423) and has therefore employed an inductive case study approach.

The process of ‘theory building’ is undertaken by examining a case in detail by starting with little or no preconceived notion of the theory that will ultimately emerge from the data (Eisenhardt 1989b). ‘Induction is viewed as the key process, with the researcher moving from the data to empirical generalisation and on to theory’ (Heath and Cowley 2004, p. 144). Eisenhardt (Eisenhardt 1989b) refers to this method as ‘Inductive Case Oriented Research’.

For this study the observed phenomenon is the ongoing and continual failure of information technology projects where failure has been defined by the inability to deliver on time, to an agreed budget, and to meet the value and quality objectives of the enterprises that the systems are meant to serve.

The methodology being utilised to examine this case is Inductive Grounded Theory which follows the methods established by Glaser (2004), Grounded theory was designed with the intent of ensuring that ‘theories systematically emerge directly from data’ (Martin and Gynnild 2011, p. 20). The term ‘grounded’ is intended to imply that the emergent theories are grounded in the data and not generated a priori and then applied to surveys or examples. By investigating the social constructs that exist in and around the main concern, inductive case oriented research is looking to tease out answers to the question ‘why?’ (Charmaz 2008a).

Inductive case study methods start with collecting and analysing data for the purposes of developing theories (Charmaz 2008a). And while data analysis may be influenced by the beliefs, prior experiences, and readings of the researcher (Heath and Cowley 2004), any researcher held preconceptions as to the prevailing theories or contributory factors should be consciously suspended until theories emerge from the data (Baker et al. 1992). This does not mean that the researcher should ignore, forget, or deliberately exclude all prior knowledge and research. Ignoring everything that has gone before may lead the researcher to develop theories that are already fully exposed, or, worse, to trivialise the problem being addressed (Thornberg 2012).

For this project, the initial set of data was archival and drawn from the public records of the Queensland Royal Commission of Inquiry, supplemented by additional material requested through the Freedom-of-Information (FOI) process and comprised:

  • the published files of the Queensland Commission of Inquiry (http://www.healthpayrollinquiry.qld.gov.au, 2013) into the Queensland Health Payroll Project; and

  • documents obtained under freedom of information (FOI) requests to the Department of Health Queensland, and to the Queensland Treasury Department.

In total there were 355 files of which 116 were individual witness statements from the Commission of Inquiry, and the balance of 239 files have been sourced by repeated FOI requests. The documents sourced by FOI request contained multiple records in each file, bringing the sum total number of individual files and documents to be examined to approximately 1,000.

The total number of pages of witness statements amounted to 3,850. In addition there was the collection of project documentation that exceeded 5,000 pages of emails, reports, project plans and other data.

To examine the case from the perspective of a timeline of events, of data and advice that was available at the time, to the participants, the researcher must endeavour to reconstruct the project from the available information. Dekker (2014) refers to this method of investigation as being ‘inside the tunnel’.

Inside the tunnel ‘is the point of view of people in the unfolding situation. To them, the outcome was not known (or they would have done something else). They contributed to the direction of the sequence of events on the basis of what they saw on the inside of the unfolding situation. To understand human error, you need to attain this perspective’ (Dekker 2014, p. 18). Understanding the Queensland Health payroll project from a perspective that is reflective of the experience of the project executives and team members as events unfolded is critical to the inductive case study process. In order to emerge a theory or theories that may potentially be applied to working projects it is imperative that the actions and decisions that were taken throughout Queensland Health payroll project are understood in the context within which they were experienced at the time.

The files were loaded into NVIVO software for qualitative analysis, allowing the researcher to identify nodes of interest, and to collate and identify common behaviours occurring throughout the projects life. Every document was scanned into Nvivo where it was examined and tagged with topics. Nvivo also provided the main repository for memos. Some documents were unable to be scanned into NVIVO and these were analysed manually, with memo’s maintained using the same coding system as that used in NVIVO. A process of normalising the initial topics was conducted to reduce them down to a manageable data set of fifty topics. These fifty topics were then correlated to themes of which three primary themes emerged from the data.

4 Literature Review

The literature on information technology project management is vast, and stretches back over almost fifty years. ‘The History of Project Management’, (Kozak-Holland 2011) traces the same project management disciplines back to the time of the construction of the Great Pyramids of Giza and the Great Wall of China. In construction of the Great Wall of China, Kozak-Holland (2011) identifies the stages of planning, executing, controlling and monitoring, and closing as being evidenced in the ancient literature. When reviewing the construction of the Great Pyramids, the archeological evidence suggest the creation of an advanced sundial which divided time into 12 roughly equal segments during daylight hours and is evidence that ‘scheduling was done using the day as the basic unit of measure’ (Kozak-Holland 2011, p. 66).

Grenny et al. (2007, p. 2) referred to a phenomenon in project management that they called ‘fact-free planning’. ‘Project leaders under pressure from various stakeholders determine deadlines, scope, deliverables and budget with little or no regard for the hard facts about what will actually be required. At other times, they base their estimates on facts, only to have the estimates ignored. In either case, the result is a set of project parameters and goals that is unrealistic from the beginning’.

Jones (2004, p. 5) created a working hypothesis of the contributory factors of project failure as being ‘(1) poor quality control is the largest contributor to cost and schedule over-runs, and (2) poor project management is the most likely cause of inadequate quality control’.

A difference amongst studies about successful projects reported in the literature, is the criteria used in defining project success or failure. Many studies assess the success of a project as completion on time, on budget, and delivering the full scope of requirements (Andersen 2010; Baccarini et al. 2004). This is certainly the criteria that has been established by the previously discussed Chaos Reports from the Standish Group (1994 to 2015).

Part of the challenge of measuring project success or failure is the lack of consensus regarding what constitutes a successful project. The CHAOS studies measure the success of a project as on time, on budget, with the full scope of requirements (Andersen 2010; Baccarini et al. 2004). However, critical commentators find these criteria incomplete because ‘they do not consider, for example, usefulness, value or user satisfaction’ (Eveleens 2009, p. 7; deBakker et al. 2010; Munns and Bjeirmi 1996).

Most companies measure the success of IT projects as meeting implementation deadlines, budgets and agreed requirements. Yet, projects can be on-time and within budget and deliver no actual business value according to Marchand and Peppard (2008).

Failure, as commonly reported in the literature reviewed, is often defined by both timeliness and budget performance. In many instances success appears to be a function of finishing the project at any cost, even if some intended functionality is not delivered or is sacrificed in order to meet that deadline. On-time and on-budget are criteria, which may have little or nothing to do with whether or not the product of an information technology project will be deemed a success by the enterprise and the users of the system.

Whilst an investigation into success and failure measurement is a worthwhile endeavour, for the purposes of this research, the determination of success or failure will be the generally accepted on-time, on-budget with the agreed level of functionality - this was the criteria applied to the Queensland Health Payroll project by the Commission of Inquiry (Chesterman 2013).

Nasir and Sahbuddin (2011b) conducted a comprehensive analysis of the literature about factors contributing to IT project success. They collated data from 43 peer-reviewed papers from 1990 to 2010. They grouped by frequency of mention in order to construct a hierarchy that appears to imply that if a subject is mentioned most frequently then it must be the most important. Nasir and Sahbuddin (2011b, p. 1) claimed that ‘in a result unique to our study, we found that the factors of clear and frozen requirements, realistic estimation of the schedule and budget, along with a competent project manager are the five most critical success factors of software projects’.

In very large IT projects, the type which the Standish Group (2015) have identified as having the lowest success rate, the complexity inherent in the solution being built is very great. ‘Today, business processes are more complex, interconnected, interdependent and interrelated than ever before. Additionally, they reject traditional organisational structures in order to create complex communities comprised of alliances with strategic suppliers, outsourcing vendors, networks of customers and partnerships with key political groups, regulatory entities, and even competitors’ (Hass Hass 2007, p. 2).

It is this level of complexity which permeates every aspect of a project (Baccarini 1996), from the internal complexity of the business problem being solved (Al Neimat 2005), to organisational complexity that complicates what should have been relatively simple (Drummond 1998). When discussing complexity in this context most projects would be looking at the complexity of the business problem to be addressed, the complexity of the technology being deployed and inter- and intra-organisational complexity of dealing with competing demands (Thomas and Mengel 2008).

Beginning in 1995 Keil observed the escalating rate of IT project failure and its cost on business and government. The generic phrase “poor project management” (Keil 1995) is far too broad to provide clarity for what actually drives project escalation and ultimately failure. Keil (1995, p. 422) adopts the definition of escalation as being “continued commitment in the face of negative information about prior resource allocations, coupled with uncertainty surrounding the likelihood of goal attainment”.

According to Keil “projects are more prone to escalation when they involve a large potential payoff, when they are viewed as requiring a long-term investment in order to receive any substantial gain” (Keil 1995 p. 422). Keil touches on “psychological factors” which may impact a managers decision to continue with a project that appears doomed to failure, and suggests that “escalation is more likely to occur when managers make errors in processing information” but does not delve deeper into why managers make those errors in processing information, whether there are different outcomes associated with different “types” of managers, or whether or not there are underlying factors as to how managers process the information being presented to them.

Keil suggests (1995. p. 431) that certain psychological factors may contribute to escalation. These factors include:

  • prior history of success,

  • high degree of personal responsibility for the outcome of the project,

  • errors in information processing, and

  • emotional attachment to the project.

Prior history of success correlates to Vaughan’s (2016) observations as to the contributory factors of Normalisation of Deviance. Where an organisation has not previously experienced negative outcomes they will continue to assume that taking the same actions or decisions will not produce deleterious results. The fact that failure had not occurred previously is not proof that their decision making was sound, rather it may have been just “luck” that no disaster had previously befallen them. In the specific case of the NASA Challenger space shuttle, various other launches had been successful despite components such as the O-Rings operating beyond their specified tolerances, and so it was assumed that earlier decisions to launch were sound and this decision would also prove to be sound. The most likely description is however that previous launches prior to the Challenger explosion were “lucky” that components operating outside of tolerances had not caused a disaster to occur similar to what happened with the Challenger. A decision by NASA to implement processes to ensure that O-Rings were checked on future launches (the proximal cause) would do nothing to ameliorate the underlying cause (normalization of deviance).

Optimism bias in a project management environment (Prater et al. 2017) may also account for why project managers maintain a “continued commitment in the face of negative information” (Keil 1995, p. 422). But what is absent from the literature is why an experienced manager would suffer from what amounts to a delusional optimism bias in the face of hard evidence to the contrary. Does the project executive not understand the information being presented to them? Does the project executive somehow consider that they are immune from the risks and failures that the majority of projects face? What propels a project executive to operate under the assumption that their project will somehow be one of the very few to be successful? The fact that project executives ignore negative information about project escalation is supported by the evidence. Even the fact that project executives may suffer from optimism bias fails to clarify why an executive would act in this way? What conditions or conditioning lead the project executive to ignore clear evidence that their project is doomed to fail requires a deeper investigation.

5 The Case Study

In 2002, the Queensland Government (Chesterman 2013) decided to establish a ‘shared services initiative’ (SSI) to provide IT services as a shared electronic payroll resource amongst most Queensland Government departments and other statutory government agencies. As part of this initiative the SSI undertook the management of the existing Lattice Payroll System in independent use by several departments, Queensland Health (QH) amongst them.

By 1st of July 2003 (Chesterman 2013, p. 10) the SSI was underway and was called CorpTech. In August 2005 CorpTech was granted A$125 million to build and operate a whole-of-government human resources and finance IT software solution. Multiple vendors were commissioned to implement the solution and support CorpTech. There were smaller numbers of contractors engaged to build an integration between SAP ECC5 to WorkBrain for payroll rostering and time and attendance recording. These multiple related system developments by different vendors were intended to be inter-operable with no discernible separation to the end user.

In March of 2006 Queensland Health (QH) had transferred responsibility for the maintenance of human resource software and hardware to CorpTech. At this time, the provision of a new computerised payroll system for QH employees was thought to be urgent because the existing system, known as LATTICE, was nearing the end of its useful life (WS122 p. 11). By 2007, an independent review known as the ‘Kelliher Report’ (PD015) found that the whole of Government system was significantly behind schedule.

A series of reviews and tenders were undertaken to fix the project by introducing a Prime Contractor. IBM subsequently won that tender to commence in December 2007. ‘By October 2008 IBM had not achieved any of the contracted performance criteria; but it had been paid about $32 million of the revised contract price of $98 million; and it forecast that to complete what it had contracted to undertake would cost the State of Queensland $181 million’ (Chesterman 2013, para 2.13).

With the QH Payroll Project, IBM had agreed to undertake a project, at a fixed price, for which no statement of work existed and no detailed planning of any description had been undertaken.

The externally engaged legal firm (WS014), in preparing their advice with respect to each of the proposals from Accenture, IBM and Logica, stated that ‘we believe on balance that IBM’s Offer gives rise to a greater number of material issues and less thought has gone into IBM’s Offer regarding contractual mechanisms that will assist the customer or enhance the working relationship between the parties’ (WS014, p. 39).

At this stage of the Queensland Health Payroll project, the Queensland Government had accepted a contract to implement an IT project to a business problems for which no business case existed and no technical solutions architecture had been provided. The IT project was shown by the evidence tabled at the Commission (Chesterman 2013) and by the analysis of documents, to be a solution to fulfil an unknown set of requirements at a fixed price and timescale, and oddly one already in government use on an existing challenged project. Furthermore, senior management was acting against the advice of their technical experts (WS085) and external legal advisors (WS014).

On 14th of March 2010 the QH payroll system finally “went live” (operational) after ten failed prior attempts. The resulting system was reported to have 35,000 payroll anomalies or processing errors (WS053) and consequently required 1,000 clerical staff to manually process fortnightly pays that otherwise was intended as the most basic core function of the new system.

After the “go live” was achieved, the Queensland Government was facing a total expenditure in the range of AUD$1.2 billion for total cost of ownership of the project. The Executive Council of the Queensland Government ordered a Commission of Inquiry into the project.

6 Discussion

‘Organisational artefacts such as mission statements, goals and objectives, strategic plans and the like function as tools to reduce choice, not to guide it’ (Manning 2008, p. 677). In the same manner, the specification of requirements, the business case, the architecture and solution design of the project are all intended to constrain choice to deliver ‘order’. In the QH project ‘order’ should have been represented by a defined scope of work, a defined project plan which sets out not only what work will be done, but also what work will not be done, and by an agreed contract. None of these things existed on the QH payroll project, and any efforts to enforce them were resisted by the vendor with the support (tacit or otherwise) of departmental executives.

The issue of transparent flows of information between parties, of experts being able to make informed decisions utilising tacit information compared to less experienced people needing to ‘follow the script’ (Vo-Tran, 2014), of actors controlling the release of information, and of stakeholders presenting different versions of themselves across multiple stages becomes critical when one considers both the makeup of the governance and management of the QH project and the individuals involved. “The involvement of non-IT stakeholders can actually work detrimentally and confound and confuse proceedings, even causing error” (Engelbrecht et al. 2017, p. 995). Non-IT experienced management, placed in a position of authority “may be influenced by some suppliers or colleagues to whose IT knowledge they had access, and insist on a certain course of action” (ibid) which may result in confusion, delay or inappropriate decision making, and contribute to the risk of IT project failure.

An appropriate lens through which to view this performance construct is referred to as the Dunning-Kruger Effect. This effect is where the less competent an individual is with respect to a particular domain then the more they are likely to overstate their perceived knowledge and ability. This may be referred to as a ‘confidence/competence dissonance’. Individuals that lack competence in a particular domain (incompetent) but are not self-aware of their lack of competence, generally perceive their performance to be not significantly inferior to those who possess significant competence, training and ability (the experts).

This phenomena has also been described as the Unskilled and Unaware Problem (UUP) (Ryvkin et al. 2012). Essentially UUP argues that individuals that are unskilled in a particular domain overestimate their own competence in both absolute terms and relative terms. Top performers underestimate their absolute and relative performance. Kruger and Dunning (Kruger and Dunning 2009) found that an unskilled person was more likely to dramatically misstate their absolute and relative competence. Ehrlinger et al. (2008) have argued that UUP is a persistent feature of decision making. Furthermore, and potentially much more concerning for complex IT projects, Kruger and Dunning (2009) determined that the skills necessary to do the job, are the same skills necessary to identify competence in others. This facet of the UUP research is particularly important when an unskilled individual is placed in a position of decision making authority, in this case with respect to an IT Project. Where an unskilled individual possesses neither the skills necessary to do the job, nor the skills necessary to identify competence in others they are not in a position to make informed decisions on complex issues. The application of this principle to the Queensland Health Payroll project would suggest that the Executive Director, the Department Secretary, and the governance boards lacked the skills needed to identify competence in others, and to comprehend informed advice when it was provided.

Engelbrecht et al. (2017 p. 5) aimed to “identify whether a causal relationship exists between the various components of business managers’ IT competence and IT success”. What they found was that a “business managers’ IT competence can, and does, exert a substantial influence on project success” (ibid: p. 1002). They reported a ‘surprising’ finding where a lack of knowledge or competence was likely to have a negative impact on project outcomes, “although one would have expected a positive relationship and a positive impact, it has been reported that the involvement of non-IT stakeholders can actually work detrimentally and confound and confuse proceedings, even causing errors”.

Engelbrecht et al. (2017) also found that ‘business managers may be influenced by some suppliers or colleagues to whose IT knowledge they had access, and insist on a certain course of action. If that business manager is particularly influential in an organisation, then there could be similar confusions, delays, and even inappropriate decisions’. This finding is reflective of the behaviours referred to in the Witness Statements. The senior executives of Queensland Health deferred to the advice of the vendor, rather than their own staff. Having discounted the concept of “amoral actors” it is this lack of knowledge of information technology, and the executives inability to parse the information being presented that builds the foundations of a theory to explain how the Queensland Health payroll project became so dysfunctional and ended in failure.

Given the importance of information technologies to business success, and their presence in almost every endeavour, one would expect to see an increase in technically literate, skilled or experienced managements to provide effective oversight and governance. Coertze and vonSolms (2013) found that 10% of organisations had Chief Information Officer (CIO) or equivalent representation at board or executive level of organisational governing management. Only 15% of organisations had board members with any IT-related qualifications, and in their United Kingdom (UK) sample, no organisation exhibited board level oversight of organisational IT through qualified representation directly as a board member. A focus on general business competence over specific IT competence continues at the CIO level where less than 50% of CIOs in the United States of America (US) public sector had primary qualifications from technical or engineering backgrounds (Ionescu 2017).

Twenge and Foster (2010) found that ‘there has been a 30% tilt towards narcissistic attitudes in US students since 1979’, and that ‘The Narcissism Epidemic’ (Kremer 2013) breeds ‘the idea that being highly self-confident is the key to success’. Twenge and Campbell (2010) were at pains to point out that there is no correlation between confidence and successful outcomes. Kremer (2013) reported that ‘over 15,000 journal articles have examined the links between high self-esteem and measurable outcomes in real life, such as educational achievement, job opportunities, popularity, health, happiness and adherence to laws and social codes’ and found no correlation or causation.

‘Over the last 30 years confidence has replaced competence’ (Kremer 2013). Positive thinking has replaced knowledge. An increase in narcissism correlates with the unskilled and unaware problem (UUP) in that ‘individuals become so self-obsessed they cannot identify their own weaknesses or learn from others’ (Kruger and Dunning 2009). This narcissistic self-belief and confidence may go some way to explain why an executive with little knowledge of information technology and no formal training or experience in information technology would agree to take on the responsibility of running ‘the largest organisational reform undertaken within the State Government’ (WS122). When it comes to the QHP payroll project, it was stated very clearly by the Deputy-Secretary of the Department that the newly appointed Executive-Director was not skilled in information technology but was a very experienced people manager with greater than 30 years in the public sector (WS026). The Executive-Director described her education and work experience as mostly being in the human resources domain (WS024).

The potential risk that this lack of (Information Technology) domain expertise causes for Information Technology projects generally, and the Queensland Health project as a specific example is encapsulated by the Dunning-Kruger Effect (2009), ‘that incompetent individuals lack the metacognitive skills that enable them to tell how poorly they are performing, and as a result, they come to hold inflated views of their performance and ability’. They are therefore potentially prone to ignore mounting evidence of their contribution to project related issues, to over-estimate their own ability to diagnose and resolve issues, and to listen to and take advice from unreliable sources. All of which were evident in the witness statements.

Of even greater concern is the UUP findings (Ryvkin et al. 2012) that not only do the domain illiterate individuals tend to overestimate their own ability relative to their actual performance, they are also at risk of being deficient in identifying relevant domain competence in others, ‘participants who scored in the bottom quartile were less able to gauge the competence of others than were their top-quartile counterparts’ (Kruger and Dunning 2009). Furthermore, they found that ‘incompetent individuals fail to gain insight into their own incompetence by observing the behaviour of other people. Despite seeing the superior performances of their peers, bottom-quartile participants continued to hold the mistaken impression that they had performed just fine’ (Kruger and Dunning 2009).

A possible explanation contributing to the Queensland Health Payroll project failure is that where managers are not technically competent, but perceive themselves as managerially capable, not only are they potentially at risk of overestimating their own ability and underestimating the relative competence of the skilled workers on the project, they do not have the skills to discern the quality of advice being given to them. Essentially, the evidence suggests that they are at high risk of not being able to assess the difference between the veracity of a confident but incompetent colleague or vendor providing advice, in comparison to a competent but less-confident colleague.

These managerial perceptions about domain expertise, confidence and competence carry the risk of significant contribution to poor project management decision-making and governance with implications for overall project failure and success. The decision-making senior project manager with accountability, responsibility and authority needs to be able to assess the information provided to them in order to make well-informed decisions. It is contended in the interpretation of the QH project data presented in this study that the consequences of placing domain-challenged persons in positions of project-critical authority is likely to lead to unsatisfactory outcomes where:

  • managers who lack domain expertise will act the part that they perceive they need to adopt;

  • these managers tend to be incapable of identifying the skilled and competent individuals that can be trusted for expert advice;

  • these managers will not have the cognitive or experiential tools to determine an appropriate course of action when faced with a project related crisis; and

  • these managers are likely to confuse confidence with competence and may be subject to undue influence by other incompetent actors.

In summary, the Queensland Health Payroll project was potentially placed at significant risk by failing to appoint management, governance and oversight that comprised sufficient domain expertise appropriately matched to the size, complexity and nature of the project.

7 Testing Situational Incompetence

It has been argued in this paper that situational incompetence is allowed to persist because of normalisation of deviance (Vaughan 2016). Normalisation of deviance implies that incompetence is tolerated because it has not previously caused significant failures. It is known that smaller projects have much higher rates of success than larger projects (Standish Group 2015), and as a consequence the skills needed to effectively manage very large projects are rarely put to the test and competence deficiencies escape detection.

It is necessary therefore to provide a method of measuring the competence of leadership as it applies to a range of IT project situations. The situations being tested are those of increasing complexity and size, and the competence of leaders relative to those constructs.

figure a

Creating a measurement instrument requires the identification and creation of an effective scale. Scale development is well established in the literature and the ‘rules’ for creating an effective scale are well articulated (for example: Churchill 1979; Flynn et al. 1993; Rao et al. 1999; Kimberlin and Winterstein 2008)

The ‘key indicators of the quality of a measuring instrument are the reliability and validity of the measures’ (Kimberlin and Winterstein 2008, p. 2276). A measure is considered ‘valid’ when the differences in observed scores accurately reflect differences in the construct being examined (Churchill 1979, p. 64). ‘Validity is often defined as the extent to which an instrument measures what it purports to measure’ (Kimberlin and Winterstein 2008, p. 2278), but an instrument can be reliable without being valid. Reliability ensures that the instrument always generates a reproducible outcome, while validity ensures that the instrument measures what it is intended to measure. In this specific instance the instrument must validly test leadership competence in a given situation, and it must do so reliably under different inputs.

The first step in creating a measurement instrument (scale development) is to create an ‘item pool’. The goal is to develop a set of measures which might sample ‘all possible contents which might comprise the putative trait according to all knows theories of the trait’ (Flynn et al. 1993, p. 310). The domain of construct is determined by a literature search. This research has determined that the domain of construct is leadership competence in a given project situation. The instrument will be developed following the procedure outlined by Churchill (1979).

The factors which will be used for the initial version of the scale have been taken from prior research into project failure which focussed on factor analysis. In particular the leadership competence construct is been drawn from the work of Engelbrecht et al. (2017), while the software project complexity measures are being informed, principally, by the work of Fitsilis and Damasiotis (2015).

The item pool has been drawn from prior work as identified in the literature. However, the response format has been modified to ensure validity and reliability in the responses provided. The two dominant response types are dichotomous responses and scale responses based on some form of Likert-type measurement (Clark and Watson 1995, p. 312).

While some of the factors presented are of a general or generic nature, many are specific. Where the respondent is asked about their experience with and knowledge of technologies, these factors should be modified to reflect the specific project be examined. For the purposes of the initial presentation of the scale, the factors used have been framed in a generic style.

The framing of the questions has also been structured to be ‘forward looking’, with the intent of being able to predict how a project might be affected rather than looking backwards and analysing a previous project that has been completed.

The model being suggested is a simple X/Y plot. The ‘X’ scale refers to project complexity and the ‘Y’ scale to leadership competence in a technical domain.

8 Implications and Future Research

The implications for industry of this research is that more attention needs to be paid to the skills and competence of the individual that will have direct authority over an IT project. Specifically, the larger and more complex the project the more important that the leader be technically skilled and experienced. While an unskilled individual may not expose a small project to significant risk, the success rate of large and complex projects is so small (Standish Group 2015) that ensuring a positive project outcome for even the most skilled and experienced practitioners is challenging. Organisations cannot afford the increased risk of management not having the competence to provide effective oversight and governance.

This research reflects the findings of a single case study, albeit a very large case, but still just one instance of a failed project. The findings cannot be generalised to apply to all IT projects, however they do provide insight into what might be occurring on other projects and why research over the last thirty or more years has not resulted in a significant improvement in project outcomes. More work is required to confirm these findings on other projects. The instrument needs to be tested and applied more broadly to determine its validity to reproduce outcomes. As identified (Dwivedi et al. 2015b) more in-depth and detailed case studies are required of both failed and successful projects to identify what actually happened on these projects and what can be both avoided in the future, and what best practices can be generalised to ensure improved outcomes. The implications for future research from this study are that investigations need to go beyond factor analysis to look for underlying drivers of project failure.

9 Conclusion

This research identified that the leading cause of failure for the Queensland Health payroll project was that the project executives and governance bodies were ill-equipped to understand the complexity of an IT project. This lack of competence meant that the project executives did not have the experience to allow them to infer appropriate actions in the face of adverse circumstances. Project executives with little knowledge or skills in Information Technology were found to be unable to (i) recognise their own limitations, (ii) identify competence in others, (iii) learn from their mistakes (iv) learn from the example of others (v) and tended to favour inappropriate sources of advice and guidance. The final word on situational competence comes from the proceedings of the IFIP Conference on IT Project Failures: ’someone implementing IT needs to know which levers to pull, in which context, and at what time’ (Dwivedi et al. 2015bb, 149). IT project leaders do not need to be the technical expert, but they do need sufficient knowledge and experience to recognise expertise and to take appropriate actions when the situation demands.