Peer Review and Evaluation of R&D Impacts

  • Barry Bozeman


Peer review is, arguably, the most important evaluation technique used in the sciences. The vast majority of public domain scientific papers are evaluated by peer review. Many government science and technology funding agencies use peer review as a primary means of selecting projects for funding. Peer review is also employed for the evaluation of R&D impacts, though this is a much less common application.


National Science Foundation Impact Evaluation Peer Review Project Selection Peer Review System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Atkinson, R. and W. Blanpied. 1985. “Peer Review and the Public Interest.” Issues in Science and Technology, 2: 101–114.Google Scholar
  2. Becker, Adam. 1990. “Peer Review and Support for Innovative Research.” Unpublished paper. Syracuse, N.Y.: Technology and Information Policy Program, Syracuse University.Google Scholar
  3. Boden, T. 1981. “Evaluation of Peer Review Draws Mixed Reactions.” Bioscience, 32, 1: 11.Google Scholar
  4. Bozeman, B. and M. Crow, U.S. R and D Laboratories and their Environments, Washington, DC: Report to the National Science Foundation, Science Resources Section, 1989.Google Scholar
  5. Bozeman, B. and M. Crow. In press. “Pork Barrel, Peer Review, and Congressional Science Policy.” Forum.Google Scholar
  6. Bozeman, B. 1979. “Straight Arrow Science and its Dangers.” Public Administration Review, 34: 116–121.CrossRefGoogle Scholar
  7. Bozeman, B. and S. Bretschneider. 1990. Assessing Electronic Merit Review at the National Science Foundation. Monograph prepared under contract to the Evaluation Office, National Science Foundation. Syracuse, N.Y.: Technology and Information Policy Program, Syracuse University.Google Scholar
  8. Ceci, S.J. and D. Peters. 1984. “How Blind is Blind Review.” American Psychologist, 39, 1491–1494.CrossRefGoogle Scholar
  9. Chubin, D. 1985. “Much Ado About Peer Review.” Bioscience, 36, 1: 18–24.CrossRefGoogle Scholar
  10. Chubin, D. and E.J. Hackett. 1990. Peerless Science: Peer Review and U.S. Science Policy. Albany, N.Y.: State University of New York Press.Google Scholar
  11. Committee of Federal Laboratories Task Force on Performance Measures for R and D. 1975. “Performance Measures for Research and Development.” Washington, D.C.: National Bureau of Standards.Google Scholar
  12. Cole, J. and S. Cole. 1981. Peer Review in the National Science Foundation. Washington, D.C.: National Academy Press.Google Scholar
  13. Cole, S., Rubin, L., and Cole, J.R., “Peer Review and the Support of Science,” Science, 237 (1977): 34–41.Google Scholar
  14. Cole, Stephen, Jonathan R. Cole, and Gary A. Simon. 1981. “Chance and Consensus in Peer Review.” Science, 214, 881–886.CrossRefGoogle Scholar
  15. Commins, W.D. Reviewer and Proposer Similarity and its Effect on Award Decision. Evaluation Staff Studies 76–1. ( Washington: National Science Foundation, 1976 ).Google Scholar
  16. Comptroller General of Canada. 1986. Evaluation of Research and Development Programs. Program Evaluation Branch, Office of the Comptroller General, Ottawa, Canada.Google Scholar
  17. Crane, D. 1972. Invisible Colleges: Diffusion of Knowledge in Scientific Communities. Chicago: University of Chicago Press.Google Scholar
  18. Gillespie, Gilbert W. Jr., Daryl E. Chubin, and George M. Kurzon. 1985. “Experience with NIH Peer Review: Researchers’ Cynicism and Desire for Change,” Science, Technology and Human Values, 10, 3, 44–53.CrossRefGoogle Scholar
  19. Hargens, Lowell. 1988. “Scholarly Consensus and Journal Rejection Rates.” American Sociological Review, 53, 139–151.CrossRefGoogle Scholar
  20. Hensler, D. Perceptions of the National Science Foundation Peer Review Process: A Report on a Survey of NSF Reviewers and Applicants. (Washington: National Science Foundation 77–33, December, 1976 ).Google Scholar
  21. Jasanoff, S. 1985. “Peer Review in the Regulatory Process.” Science Technology and Human Values, 10, 3: 20–33.CrossRefGoogle Scholar
  22. Klahr, David. 1985. “Insiders, Outsiders, and Efficiency in a National Science Foundation Panel.” American Psychologist, 40, 2, 148–54.CrossRefGoogle Scholar
  23. Koshland, D.E. 1985. “Peer Review of Peer Review.” Science, 228: 1387.Google Scholar
  24. Kruytbosch, Carlos. 1989. “The Role and Effectiveness of Peer Review.” Paper presented at the Ciba Foundation Conference on Research Evaluation.Google Scholar
  25. Leopold, A. Carl. 1988. “The Peer-Review System:Pique and Critique.” The Scientist, 11–12.Google Scholar
  26. Lindsey, Duncan. 1978. The Scientific Publication System in Social Science. San Francisco: Jossey Bass.Google Scholar
  27. Lindsey, Duncan. 1988. “Assessing Precision in the Manuscript Review Process: A Little Better than a Dice Roll.” Scientometrics, 14, 1,2, 75–82.Google Scholar
  28. Maddox, John. 1984. “Privacy and the Peer-Review System.” Nature 312, 497.CrossRefGoogle Scholar
  29. Mahoney, Michael. 1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy and Research, 1,2, 161–175.Google Scholar
  30. McCullough, Jim. 1989. “First Comprehensive Survey of NSF Applicants Focuses on their Concerns about Proposal Review.” Science, Technology, and Human Values, 14 (1), 78–88.CrossRefGoogle Scholar
  31. Meadows, A.J. 1974. Communication in Science. London: Butterworth Publishing.Google Scholar
  32. Mitroff, Ian I. and Daryl Chubin. 1979. “Peer Review at NSF: A Dialectical Policy Analysis.” Social Studies of Science, 9, 199–232.CrossRefGoogle Scholar
  33. Moment, Gardener. 1986. “Risk Money for Research and the Peer Review System.” Bioscience, 31, 6: 421.Google Scholar
  34. National Science Foundation, Evaluation Staff. Fairness of the NSF Award Decision Process: Fiscal Year 1982. ( Washington: National Science Foundation, 1984 ).Google Scholar
  35. National Science Foundation. Proposal Review at NSF: Perceptions of Principal Investigators. NSF Report 88–4. ( Washington: National Science Foundation, 1988 ).Google Scholar
  36. NIH Grants Peer Review Study Team. 1976. Grants Peer Review: Report to the Director, NIH Phase I. Washington, D.C.Google Scholar
  37. Noble, John H. 1974. “Peer Review: Quality Control of Applied Social Research.” Science, 185, 916–921.CrossRefGoogle Scholar
  38. Organization for Economic Co-operation and Development. 1986. Evaluation of Research: A Selection of Current Practices. Paris: OECD.Google Scholar
  39. Polanyi, M. 1962. “The Republic of Science: Its Political and Economic Theory.” Minerva, 1: 54–73.CrossRefGoogle Scholar
  40. Porter, A. and F. Rossini. 1985. “Peer Review of Interdisciplinary Proposals.” Science, Technology and Human Values, 10, 3, 34–42.CrossRefGoogle Scholar
  41. Proceedings of the 1989 Meeting of the Chairpersons of DRG Initial Review Groups.“ 1989. Washington, D.C.: U.S. Department of Health and Human Services, National Institutes of Health.Google Scholar
  42. Roy, Rustom. 1985. “Funding Science: The Real Defects of Peer Review and an Alternative to It.” Science, Technology and Human Values, 10, 3, 47–54.CrossRefGoogle Scholar
  43. Roy, Rustom. 1982. “Peer Review of Proposals-Rationale, Practice, and Performance.” Bulletin of Science, Technology, and Society, 2, 402–422.Google Scholar
  44. Roy, Rustom. 1984. “Alternatives to Review by Peers: A Contribution to the Theory of Scientific Choice.” Minerva, 22 (3,4) 316–328.CrossRefGoogle Scholar
  45. Salasin, J., L. Hattery, and T. Ramsay. 1980. The Evaluation of Federal Research Programs. McClean, Va.: The MITRE Corporation.Google Scholar
  46. Silber, John. 1985. Testimony before the Congressional Science Policy Task Force, Washington, D.C., June 26, 1985.Google Scholar
  47. Southgate, M.T. 1982. “Conflict of Interest and the Peer Review Process.” Journal of the American Medical Association, 258: 1375.CrossRefGoogle Scholar
  48. Stahl, M.J. and J.A. Steger. 1977. “Measuring Innovation and Productivity: A Peer Rating Approach.” Research Management, 20, 1: 35–38.Google Scholar
  49. U.S. Department of Energy. 1990. Procedures for Peer Review Assessments. Washington, D.C.: Office of Energy Research, Office of Program Analysis, U.S. Department of Energy.Google Scholar
  50. U. S. General Accounting Office. 1987. University Funding: Information on the Role of Peer Review at NIH and NSF. Washington, D.C.: USGAO, 5.Google Scholar
  51. Weinberg, Alvin. 1966. Reflections on Big Science. Cambridge, Mass.: MIT Press.Google Scholar
  52. Zuckerman, H. and R. Merton. 1973. “Institutionalized Patterns of Evaluation in Science,” in R. Merton (ed.) The Sociology of Science. Chicago: University of Chicago Press.Google Scholar

Copyright information

© Springer Science+Business Media New York 1993

Authors and Affiliations

  • Barry Bozeman
    • 1
  1. 1.Center for Technology and Information Policy The Maxwell School of Citizenship Public Affairs and The L.C. Smith College of EngineeringSyracuse UniversityUSA

Personalised recommendations