The 2019 PBRF Review: What’s to be Done?
- 12 Downloads
The Minister of Education, Chris Hipkins, and the Associate Minister, Jenny Salesa, recently announced the panel appointed to review the Performance Based Research Fund (PBRF). The panel will follow the Terms of Reference (ToR) published in September 2018 (Ministry of Education 2018). This review follows previous reviews of PBRF completed in 2004, 2008, and 2012/2013. What are the problems with PBRF, and what changes can be made? We use the ToR to help consider these questions.
Reprise: The Problems with PBRF
The PBRF is inspired by the Public Choice principle of ‘provider capture’ which holds that university academics as government employees will necessarily, by ontological definition, exploit their employer unless they are restrained from doing so. At the same time, they can be ‘incentivised’ to work in the employer’s interests, rather than their own. Intellectual work is notoriously hard to judge: is a ‘knowledge worker’ putting in the requisite time and effort? Or are they leading a lazy ‘featherbedding’ life, perhaps claiming to need more time, office space, whiteboard pens and classrooms than they really do need?
So, the challenge when PBRF was being designed in 2002 was to convert intellectual output—often verbal, pedagogic, conversational, supportive—into measurable outputs, which could be used as the means of both surveillance and reward of academic labour. The difference from the classic economic example of running factories by paying the workers for their outputs is that in PBRF, while the inputs are individual, the rewards are collective: although enormous effort is required from each individual academic, the (rather paltry) financial recognition goes to the institution. Universities are not obliged to spend the money on promoting academics’ research—they can, for instance, spend it on student scholarships instead.
The PBRF has unintended consequences, more clearly seen in retrospect. The first iteration resulted in the wholesale purchase of old professors with long publishing lists—in some cases with disastrous consequences—and less employment of young academics who did not yet have a publishing record. The second iteration attempted to compensate for that, and for the failure to recognise outputs beyond the classic academic journal article, so the process became more and more complex. One of the aims of the 2019 review is to assess the improvements in outcome made by tweaks to the PBRF system in the last review.
A proclaimed purpose of PBRF from its inception is to recognise and support tertiary-level research excellence and thereby support quality research-led teaching. This is the origin of the emphasis on ‘the individual as the unit of measurement’; it is also a recognition of the link between research and the areas into which academics teach. The 2019 PBRF review aims to boost the government’s ‘support’ of ‘research excellence’ by improving the performance of the PBRF process, so that future research will have more national impact.
According to the ToR, “PBRF has supported the development of a stronger research culture across tertiary education organisations over the last 16 years” (Ministry of Education 2018, p. 1). Unfortunately, the PBRF has achieved this by instituting a surveillance culture. While the individual research assessment exercise is only one of the ways PBRF measures research excellence, the levels of scrutiny and stress related to the individual audit predominate in the minds of university academics and tertiary educators, country wide.
The stakes related to the individual assessment are significant for other reasons too. For those academics attaining an ‘A’ grading, their prospects of career advancement are considerably strengthened. For the organisations, while the individually-ranked portfolios attract on-going research funding, the aggregation of PBRF results overall enable one university to be named as the top-ranked university in the nation, significantly boosting that university’s ability to attract further research funds, and international students, who are far more significant in the funding of universities than PBRF itself.
The assumption built into the design of the PBRF system that competition between academics and between institutions would increase the quality of tertiary-level research in Aotearoa New Zealand is characteristically neoliberal thinking. Yet despite nearly 20 years of PBRF, we are arguably no better off; indeed, some of our most prestigious universities are relatively worse off, according to global league tables such as THE and QS (themselves a manifestation of neoliberal ideology).
Can the PBRF System be Improved?
The 2019 ToR do not guide the panel to address the neoliberal underpinnings of PBRF, but the panel is asked to check PBRF’s alignment with, for example, the National Research Charter, an initiative currently underway (Royal Society Te Apārangi 2019). It will also re-visit the concept of the ‘individual as the unit of measurement’, with, potentially, an increased focus on developing collaborative approaches to research.
The motive behind this shift is questionable: it may relate to the emphasis in the ToR to reduce individual workload during the audit period, or to finding better ways to gauge ‘industry impact’. While justified in relation to international trends, and practices in other local industries, the increasing emphasis on impact is problematic. Despite acknowledging the significance of upholding the legislative requirement that tertiary academics be the ‘critic and conscience’ of our society, that ideal may well become submerged beneath the waves of instrumentalist impact, determined purely by measures of successful performance.
A bid to ‘lower transaction costs’, should individual measurement be retained, includes reference to less frequent auditing, greater self-reporting, and reliance on digital measures, such as h-indices. These possible steps are unlikely to be warmly greeted by all. They depend, for example, on all scholars having and maintaining a digital footprint. Apart from the variability of this practice within the current community of scholars, there is the variability of digital results across the many measures available. Lowering transaction costs may reduce the time and cost associated with the current system, but is unlikely to provide the level of ‘objectivity’ of the existing system, and will encourage further ‘gaming’ of the system. The ToR also mention the possible intention to track researcher activity using the National Research Information System (Ministry of Business Innovation & Employment 2019), ultimately committing researchers to an increasingly instrumentalist, big-data driven surveillance of their research work.
The new ToR implicitly recognise the cultural bias inherent in standard research formats, and direct the panel to “consider how the PBRF can better support the research activity of all types of research, including basic, applied, creative, mātauranga Māori research, and Pacific or other cultural research perspectives” (p. 6). How a review can achieve that, within the parameters of a programme that is itself a function of precisely those biases, it does not explain.
While the intention to consider further steps by which to better acknowledge the work of new and emerging researchers, and a more diverse body of researchers, is welcome, we think the ToR glide over, and implicitly accept, the increasing casualisation of academic labour. To hide behind such notions as “the changing nature of work” (Ministry of Education 2018, p. 6) actually avoids a serious matter. While we agree that institutions should recognise the complex life arrangements of modern families, we resist the inclination to retain staff on part-time and/or temporary contracts. Furthermore, the ToR document in this one suggestion conflates several different notions: staffing diversity, ranges in researcher experience, including issues of mentoring, and the challenges of assessing people who find themselves (willingly or unwillingly) in various forms of casualised or part-time employment.
We assume the panellists will analyse the existing system in the context of all its shortcomings, recognising that not all the assumptions underpinning the ToR are necessarily sound. As the cliché suggests, there are no ‘silver bullet solutions’, but there may be ways to make the current system work better. With all the effort that has gone into it so far, there seems little support for ditching it altogether, as the Tertiary Education Union (TEU) has called for (Scoop Media 2019). Despite the current government’s wish to move away from neoliberalism in education it seems not to have recognised how deeply embedded these structures are. PBRF has become confounded with notions of ‘accountability’ as a sine qua non of educational management. Most likely the current review will (again) tweak, and not jettison, PBRF.
- Ministry of Business Innovation & Employment. (2019). New Zealand Research Information System. Retrieved from https://www.mbie.govt.nz/science-and-technology/science-and-innovation/research-and-data/nzris/.
- Ministry of Education. (2018). Terms of reference: Review of the performance-based research fund (PBRF) 2019. Retrieved from https://www.education.govt.nz/further-education/policies-and-strategies/review-of-the-performance-based-research-fund/.
- Royal Society Te Apārangi. (2019). Developing a national research charter for Aotearoa New Zealand. Retrieved from https://royalsociety.org.nz/what-we-do/research-practice/developing-a-national-research-charter-for-aotearoa-new-zealand.
- Scoop Media. (2019). It’s time for PBRF to go. Retrieved from http://www.scoop.co.nz/stories/ED1904/S00052/its-time-for-pbrf-to-go.htm.