Abstract
Researchers undertake a number of different research evaluation tasks, taking up a substantial part of their research time—estimated to about one work month per year for a professor. This paper addresses the various evaluator roles and tasks researchers take on, and the tensions they involve. How the research evaluator role may conflict with the researcher role and with societal expectations is discussed, as well as the intrinsic tensions in peer review; including expertise vs. impartiality, evaluators as neutral judges vs. exercise of power and influence, divergent peer assessments vs. the need for unanimous conclusions in peer panels, peer review vs. increase in quantitative indicators, and accountability to society vs. peer review as preserving the autonomy of science. The examination of these tensions provides insight in the political aspects of peer review, and a basis for discussing an agenda for future studies on the role of peer evaluators. Major future challenges for peer review concern how to meet demands for transparency and public accountability, and maintain academic autonomy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
AAUW. (2004). Tenure denied. Cases of sex discrimination in academia. Washington, DC: American Association of University Women Education Foundation and American Association of University Women Legal Advocacy Fund.
Aksnes, D. W., & Taxt, R. E. (2004). Peer review and bibliometric indicators: A comparative study at a Norwegian university. Research Evaluation, 13(1), 33–41.
Bornman, L., & Daniel, H. D. (2008). Reliability, fairness, and predictive validity of the peer review process for the selection of research fellowship recipients of the Boehringer Ingelheim fonds. In B. M. Kehm (Ed.), Hochschule im Wandel. Die Universität als Forschungsgegenstand. Festschrift für Ulrich Teichler (pp. 365–376). Frankfurt am Main: Campus.
Campanario, J. M. (1998a). Peer review for journals as it stands today—part 1. Science Communication, 19(3), 181–211.
Campanario, J. M. (1998b). Peer review for journals as it stands today—part 2. Science Communication, 19(4), 277–306.
Ceci, S. J., & Peters, D. P. (1982). Peer review: A study of reliability. Change, 14(6), 44–48.
Chubin, D. E., & Hackett, E. J. (1990). Peerless science. Peer review and US science policy. New York: State University of New York Press.
Cole, S. (1983). The hierarchy of the sciences? American Journal of Sociology, 89(1), 111–139.
Friedman, R. M. (2001). The politics of excellence. Behind the nobel prize in science. New York: Times Books.
Hackett, E. J. (1997). Peer review in science and science policy. In M. S. Frankel & J. Cave (Eds.), Evaluating science and scientists (pp. 51–60). Budapest: Central European University Press.
Hansen, H. F. (2009). Research evaluation: Methods, practice, and experience. Copenhagen: Danish Agency for Science, Technology and Innovation. Research: Analysis and Evaluation.
Hearn, J. C., & Anderson, M. S. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43(5), 503–529.
Henkel, M. (2007). Can academic autonomy survive in the knowledge society? A perspective from Britain. Higher education Research & Development, 26(1), 87–99.
Horrobin, D. F. (1982). Peer review: A philosophically faulty concept which is proving disastrous for science. Behavioral and Brain Sciences, 5(2), 217–218.
Irvine, J., & Martin, B. (1984). Foresight in science: Picking the winners. London: Pinter.
Jasanoff, S. (1990). The fifth branch, science advisers as policymakers. Cambridge, MA: Harvard University press.
Jasanoff, S. (2003). Technologies of humility: Citizen participation in governing science. Minerva, 41(3), 223–244.
Johnston, S. (1997). Examining the examiners: An analysis of examiners’ reports on doctoral theses. Studies in Higher Education, 22(3), 333–347.
Kyvik, S., & Langfeldt, L. (2004). Tidkrevende bedømmelsesarbeid. Forskningspolitikk, 27(2), 15.
Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, MA: Harvard University Press.
Langfeldt, L. (2001). The decision-making constraints and processes of grant peer review, and their effects on the review outcome. Social Studies of Science, 31(6), 820–841.
Langfeldt, L. (2002). Decision-making in expert panels evaluating research. Constraints, processes and bias. Oslo: Unipub. Accessed September 30, 2010, from http://www.statsvitenskap.uio.no/fag/polit/disputas/fulltxt/langfeldt.pdf.
Martin, B. (2000). Research grants: Problems and options. Australian Universities’ Review, 43(2), 17–22. Accessed September 23, 2010, from http://www.bmartin.cc/pubs/00aur.html.
Morley, L., Leonard, D., & David, M. (2002). Variations in vivas: Quality and equality in British PhD assessments. Studies in Higher Education, 27(3), 263–273.
Neave, G. (1998). The evaluative state reconsidered. European Journal of Education, 33(3), 265–284.
Nowotny, H., Scott, P., & Gibbons, M. (2003). ‘Mode 2’ revisited: The new production of knowledge—Introduction. Minerva, 41(3), 179–194.
Pöschl, U., & Koop, T. (2008). Interactive open access publishing and collaborative peer review for improved scientific communication and quality assurance. Information Services & Use, 28(2), 105–107.
Ravetz, J. R. (1971). Scientific knowledge and its social problems. Oxford: Clarendon Press.
Roy, R. (1984). Alternatives to review by peers: A contribution to the theory of scientific choice. Minerva, 22(3–4), 316–328.
Scott, A. (2007). Peer review and the relevance of science. Futures, 39(7), 827–845.
Sörlin, S. (2007). Funding diversity: Performance-based funding regimes as drivers of differentiation in higher education systems. Higher Education Policy, 20(4), 413–440.
Speck, B. W. (1993). Publication peer review. An annotated bibliography. Westport/London: Greenwood Press.
Tinkler, P., & Jackson, C. (2000). Examining the doctorate: Institutional policy and the PhD examination. Studies in Higher Education, 25(2), 167–180.
Travis, G. D. L., & Collins, H. M. (1991). New light on old boys: Cognitive and institutional particularism in the peer review system. Science, Technology & Human Values, 16(3), 322–341.
van den Besselaar, P., & Leydesdorf, L. (2009). Past performance, peer review and project selection: A case study in the social and behavioral sciences. Research Evaluation, 18(4), 272–288.
Van der Meulen, B. (1998). Science policies as principal-agent games. Institutionalization and path dependency in the relation between government and science. Research Policy, 27(4), 397–414.
Weller, A. C. (2001). Editorial peer review. Its strengths and weaknesses. Medford, NJ: Information Today (ASIS&T Monograph Series).
Woodhouse, E., & Sarewitz, D. (2007). Science policies for reducing societal inequities. Science and Public Policy, 34(3), 139–150.
Ziman, J. M. (1994). Prometheus bound: Science in a dynamic steady state. Cambridge: Cambridge University Press.
Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system. Minerva, 9(1), 66–100.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Langfeldt, L., Kyvik, S. Researchers as evaluators: tasks, tensions and politics. High Educ 62, 199–212 (2011). https://doi.org/10.1007/s10734-010-9382-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10734-010-9382-y