Advertisement

Peer Review: From “Sacred Ideals” to “Profane Realities”

  • David R. JohnsonEmail author
  • Joseph C. Hermanowicz
Chapter
Part of the Higher Education: Handbook of Theory and Research book series (HATR, volume 32)

Abstract

Peer review, a socially structured process of evaluating scholarly and scientific performance, is a ubiquitous condition of role performance in the professoriate and central to the production of knowledge. Focusing on the evaluation of publication, this chapter directs attention to three features of peer review: its functional ideals and relationship with the academic reward system; the social organizational basis of peer review and trends that constrain it; and dysfunctions that arise related to reliability, bias, and violation of anonymity. The discussion underscores structural and cultural characteristics of peer review that seemingly shatter its idealized image. While all faculty who conduct research subject themselves to the evaluation of their peers, extant research on peer review is disproportionately based on studies of relatively prestigious journals in the social sciences and medicine. The review thus identifies promising paths for future empirical studies of peer review that would examine understudied disciplines and publication venues, ideally through comparative frameworks.

Keywords

Articles Publication Evaluation Faculty Scientists Academic careers Work Sociology of Higher Education Sociology of knowledge Productivity Bias Stratification 

References

  1. Aarssen, L. W., Lortie, C. J., Budden, A. E., Koricheva, J., Leimu, R., & Tregenza, T. (2009). Does publication in top-tier journals affect reviewer behavior? Plos One, 4(7), e6283. doi: 10.1371/journal.pone.0006283.CrossRefGoogle Scholar
  2. Abbott, A. (2011). Personal communication to second author, annual board meeting of the American Journal of Sociology, Atlanta, Georgia.Google Scholar
  3. Abramowitz, S. I., Gomes, B., & Abramowitz, C. V. (1975). Publish or politic: Referee bias in manuscript review. Journal of Applied Social Psychology, 5(3), 187–2000.CrossRefGoogle Scholar
  4. Adair, R. K., Carlon, H. R., & Sherman, C. (1981). Anonymous refereeing. Physics Today, 34(6), 13–15.CrossRefGoogle Scholar
  5. Alam, M., Kim, N.A., Havey, J., Rademaker, A., Ratner, D., Tregre, B.,…Coleman, W.P. (2011). Blinded vs. unblended peer review of manuscripts submitted to a dermatology journal: A randomized multi-rater study. British Journal of Dermatology, 165(3), 563–567.Google Scholar
  6. Alperts, B. (2013). Impact factor distortions. Science, 340(6134), 787.CrossRefGoogle Scholar
  7. Amrein, K., Langmann, A., Fahrleitner-Pammer, A., Pieber, T. R., & Zollner-Schwetz, I. (2011). Women underrepresented on editorial boards of 60 major medical journals. Gender Medicine, 8(6), 378–387.CrossRefGoogle Scholar
  8. Aper, J. P., & Fry, J. E. (2003). Post-tenure review at graduate institutions in the United States. Journal of Higher Education, 74(3), 241–260.CrossRefGoogle Scholar
  9. Appel, C. S. (1994). University press editing and publishing. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 179–194). Lanham, MD: Rowman & Littlefield Publishers.Google Scholar
  10. Bacon, F. (1620). Novum Organum: Or, true suggestions for the interpretation of nature. London, UK: Routledge and Sons.Google Scholar
  11. Bazerman, C. (1988). Shaping written knowledge: The genre and activity of the experimental article in science. Madison, WI: University of Wisconsin Press.Google Scholar
  12. Becher, T., & Trowler, P. R. (2001). Academic tribes and territories. Buckingham, UK: Open University Press.Google Scholar
  13. Becker, H. S. (1984). Art worlds. Berkeley, CA: University of California Press.Google Scholar
  14. Ben-David, J. (1965). The scientific role: The conditions of its establishment in Europe. Minerva, 4(1), 15–54.CrossRefGoogle Scholar
  15. Bernard, H. R. (1980). Report from the editor. Human Organization, 39(4), 366–369.CrossRefGoogle Scholar
  16. Bess, J. L. (1988). Collegiality and bureaucracy in the modern university: The influence of information and power on decision-making structures. New York: Teachers College Press.Google Scholar
  17. Beveridge, W. I. B. (1950). The art of scientific investigation. New York: Vintage Books.Google Scholar
  18. Biagioli, M. (2002). From book censorship to academic peer review. Emergences: Journal for the Study of Media & Composite Cultures, 12(1), 11–45.CrossRefGoogle Scholar
  19. Björk, B. C., Roos, A., & Lauri, M. (2009). Scientific journal publishing: Yearly volume and open access availability. Information Research, 14(1). http://InformationR.net/ir/14-1/paper391.html
  20. Björk, B., Welling, P., Laakso, M., Majlender, P., Hedlund, T., & Guõnason, G. (2010). Open access to the scientific literature: Situation 2009. PLoS ONE, 5(6), e11273. doi:10.137/journal.pone.0011273.Google Scholar
  21. Björk, B. C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Infometrics, 7(4), 914–923.CrossRefGoogle Scholar
  22. Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review. American Economic Review, 81(5), 1041–1067.Google Scholar
  23. Bohannon, J. (2013). Who’s afraid of peer review? Science, 342(6154), 60–65.CrossRefGoogle Scholar
  24. Bornmann, L., & Daniel, H. (2005). Selection of research fellowship recipients by committee peer review: Reliability, fairness and predictive validity of board of trustees’ decisions. Scientometrics, 63(2), 297–320.CrossRefGoogle Scholar
  25. Bornmann, L., Mutz, R., & Daniel, H. (2010a). A reliability-generalization study of journal peer reviews: A multi-level meta-analysis of inter-rater-reliability and its determinants. PloS One, 5(12), e14331. doi: 10.1371/journalpone.0014331.CrossRefGoogle Scholar
  26. Bourdieu, P. (1984). Distinction: A social critique of the judgement of taste. Cambridge, MA: Harvard University Press.Google Scholar
  27. Bradley, J. V. (1981). Pernicious publication practices. Bulletin of the Psychonomic Society, 18(1), 31–34.CrossRefGoogle Scholar
  28. Braxton, J. M., & Hargens, L. L. (1996). Variation among academic disciplines: Analytical frameworks and research. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. XI, pp. 1–46). New York, NY: Agathon.Google Scholar
  29. Budden, A. E., Tregenza, T., Aarssen, L. W., Koricheva, J., Leimu, R., & Lortie, C. J. (2008). Double-blind review favours increased representation of female authors. TRENDS in Ecology and Evolution, 23(1), 4–6.CrossRefGoogle Scholar
  30. Burnham, J. C. (1990). The evolution of editorial peer review. Journal of the American Medical Association, 263(10), 1323–1329.CrossRefGoogle Scholar
  31. Calhoun, C. (2010). On Merton’s legacy and contemporary sociology. In R. K. Merton (Ed.), Sociology of science and sociology as science (pp. 1–32). New York, NY: Columbia University Press.CrossRefGoogle Scholar
  32. Campanario, J. M. (1993). Consolation for the scientist: Sometimes it is hard to publish papers that are later highly-cited. Social Studies of Science, 23, 342–362.CrossRefGoogle Scholar
  33. Campanario, J. M. (1998). Peer review for journals as it stands today—Part 1. Science Communication, 19(1), 181–211.CrossRefGoogle Scholar
  34. Ceci, S. J., & Peters, D. (1984). How blind is blind peer review? American Psychologist, 39(12), 1491–1492.CrossRefGoogle Scholar
  35. Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women’s underrepresentation in science. Proceedings of the National Academy of Sciences, 108(8), 3157–3162.Google Scholar
  36. Chase, J. M. (1970). Normative criteria for scientific publication. American Sociologist, 5(3), 262–265.Google Scholar
  37. Cho, A. H., Johnson, S. A., Schuman, C. E., Adler, J. M., Gonzalez, O., Graves, S. J., et al. (2014). Women are underrepresented on the editorial boards of journals in environmental biology and natural resource management. PeerJ, 2, e542. doi: 10.7717/peerj.542.CrossRefGoogle Scholar
  38. Chubin, D. E., & Hackett, E. J. (1990). Peerless science: Peer review and U.S. science policy. Stony Brook, NY: State University Press of New York.Google Scholar
  39. Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14(1), 119–186.CrossRefGoogle Scholar
  40. Cicchetti, D. V., & Eron, L. D. (1979). The reliability of manuscript reviewing for the Journal of Abnormal Psychology. Proceedings of the American Statistical Association, 22, 596–600.Google Scholar
  41. Cleary, F. R., & Edwards, D. J. (1960). The origins of the contributors to the A.E.R. during the ‘fifties. The American Economic Review, 50(5), 1011–1014.Google Scholar
  42. Cole, S. (1992). Making science: Between nature and reality. Cambridge, MA: Harvard University Press.Google Scholar
  43. Cole, S., & Cole, J. R. (1973). Social Stratification in Science. Chicago, IL: University of Chicago Press.Google Scholar
  44. Cole, S., Cole, J. R., & Simon, G. (1981). Chance and consensus in peer review. Science, 214(4523), 881–886.CrossRefGoogle Scholar
  45. Cole, S., Rubin, L., & Cole, J. R. (1978). Peer review in the National Science Foundation: Phase one of a study. Washington, DC: National Academy of Sciences.Google Scholar
  46. Coser, L. A., Kadushin, C., & Powell, W. W. (1982). Books: The culture and commerce of publishing. Chicago, IL: University of Chicago Press.Google Scholar
  47. Coursol, A., & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17(2), 136–137.CrossRefGoogle Scholar
  48. Crane, D. (1965). Scientists at major and minor universities: A study of productivity and recognition. American Sociological Review, 30, 699–714.CrossRefGoogle Scholar
  49. Crane, D. (1967). The gatekeepers of science: Some factors affecting the selection of articles for scientific journals. The American Sociologist, 2(4), 195–201.Google Scholar
  50. Cummings, L., Frost, P. J., & Vakil, T. F. (1985). The manuscript review process: A view from the inside on coaches, critics, and special cases. In L. L. Cummings & P. J. Frost (Eds.), Publishing in the organizational sciences (pp. 469–508). Homewood, IL: Irwin.Google Scholar
  51. Dar, R. (1987). Another look at Meehl, Lakatos, and the scientific practices of psychologists. American Psychologist, 42(2), 145–151.CrossRefGoogle Scholar
  52. De Vries, R., Anderson, M. S., & Martinson, B. C. (2006). Normal misbehavior: Scientists talk about the ethics of research. Journal of Empirical Research on Human Research Ethics, 1(1), 43–50.CrossRefGoogle Scholar
  53. Dewitt, N., & Turner, R. (2001). Bad peer reviewers. Nature, 413(6852), 93.CrossRefGoogle Scholar
  54. Dressler, A. J. (1970). Nobel prizes: 1970 awards—Physics. Science, 170, 604–606.CrossRefGoogle Scholar
  55. Edwards, L.N., & Ferber, M.A. (1986). Journal reviewing practices and the progress of women in the economics profession: Is there a relationship? Newsletter of the Committee on the Status of Women in the Economics Profession, 1–7.Google Scholar
  56. Emerson, G. B., Winston, J. W., Wolf, F., Heckman, J., Brand, R., & Leopold, S. S. (2010). Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170(21), 1934–1939.CrossRefGoogle Scholar
  57. Epstein, W. (1990). Confirmational response bias among social work journals. Science, Technology, and Human Values, 15(1), 9–38.Google Scholar
  58. Ernst, E., & Kienbacher, T. (1991). Chauvinism. Nature, 353(6336), 560.CrossRefGoogle Scholar
  59. Ernst, E., Resch, K. L., & Uher, E. M. (1992). Reviewer bias. Annals of Internal Medicine, 116(1), 958.CrossRefGoogle Scholar
  60. Fairweather, J. S. (2002). The ultimate faculty evaluation: Promotion and tenure decisions. New Directions for Institutional Research, 114, 97–108.CrossRefGoogle Scholar
  61. Fanelli, D. (2010). “Positive” results increase down the hierarchy of the sciences. PLos One, 5(4), e10068. doi: 10.1371/journal.pone.0010068.CrossRefGoogle Scholar
  62. Ferber, M. A., & Teiman, M. (1980). Are women economists at a disadvantage in publishing journal articles? Eastern Economic Journal, 6(3–4), 189–193.Google Scholar
  63. Finke, R. A. (1990). Recommendations for contemporary editorial practices. American Psychologist, 45(5), 669–670.CrossRefGoogle Scholar
  64. Fisher, M., Freidman, S. B., & Strauss, B. (1994). The effects of blinding on acceptance of research papers by peer review. JAMA, 272(2), 143–148.CrossRefGoogle Scholar
  65. Fox, M. F. (1994). Scientific misconduct and editorial peer review processes. Journal of Higher Education, 65(3), 298–309.CrossRefGoogle Scholar
  66. Friedkin, N. (1983). Horizons of observability and limits of informal control in organizations. Social Forces, 62(1), 54–77.CrossRefGoogle Scholar
  67. Fyfe, J. J. (1994). Cops and robbers in academe: Editing Justice Quarterly. In R. Simons & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 59–72). Boston, MA: Rowman and Littlefield.Google Scholar
  68. Garfield, E. (1955). Citation indexes to science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.CrossRefGoogle Scholar
  69. Garfunkel, J. M., Ulshen, M. H., Hamrick, H. J., & Lawson, E. (1994). Effect of institutional prestige on reviewers’ recommendations and editorial decisions. JAMA, 272(2), 137–138.CrossRefGoogle Scholar
  70. General Accounting Office. (1994). Peer review: Reforms needed to ensure fairness in federal agency grant selection. Washington, DC: General Accounting Office.Google Scholar
  71. Geiger, R. L. (1986). To advance knowledge: The growth of American research universities, 1900–1940. New York, NY: Oxford University Press.Google Scholar
  72. Gilbert, J. R., Williams, E. S., & Lundberg, G. D. (1994). Is there a gender bias in JAMA’s review process? JAMA, 272(2), 137–138.CrossRefGoogle Scholar
  73. Gillespie Jr., G. W., Chubin, D. E., & Kurzon, G. M. (1985). Experience with NIH peer review: Researchers’ cynicism and desire for change. Science, Technology, and Human Values, 10(3), 44–54.CrossRefGoogle Scholar
  74. Goldberg, P. (1968). Are women prejudiced against women? Trans-Action, 5(5), 28–30.Google Scholar
  75. Goodrich, D. W. (1945). An analysis of manuscripts received by the editors of the American Sociological Review from May 1, 1944 to September 1, 1945. American Sociological Review, 10(6), 716–725.CrossRefGoogle Scholar
  76. Greaves, S., Scott, J., Clarke, M., Miller, L., Hannay, T., Thomas, A., & Campbell, P. (2006). Overview: Nature’s trial of open peer review. Nature. doi: 10.1038/nature05535.Google Scholar
  77. Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20.CrossRefGoogle Scholar
  78. Grouse, L. D. (1981). The Ingelfinger rule. Journal of the American Medical Association, 245(4), 375–376.CrossRefGoogle Scholar
  79. Guest, A. M. (1994). Gatekeeping among the demographers. In R. Simons & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 85–106). Boston, MA: Rowman and Littlefield.Google Scholar
  80. Guetzkow, J., Lamont, M., & Mallard, G. (2004). What is originality in the humanities and social sciences? American Sociological Review, 69(2), 190–212.CrossRefGoogle Scholar
  81. Gustin, B. H. (1973). Charisma, recognition, and the motivation of scientists. American Journal of Sociology, 78(5), 1119–1134.Google Scholar
  82. Hall, R. A., & Hall, M. B. (Eds.) (1966). The correspondence of Henry Oldenburg, Vol. 2. Madison, WI: University of Wisconsin Press.Google Scholar
  83. Hamermesh, D. S. (1994). Facts and myths about refereeing. The Journal of Economic Perspectives, 8(1), 153–163.CrossRefGoogle Scholar
  84. Hargens, L. L. (1988). Scholarly consensus and journal rejection rates. American Sociological Review, 53, 139–151.CrossRefGoogle Scholar
  85. Hargens, L. L., & Herting, J. R. (1990). Neglected considerations in the analysis of agreement among journal referees. Scientometrics, 19, 91–106.Google Scholar
  86. Hersen, M., & Miller, D. J. (1992). Future directions: A modest proposal. In D. J. Miller & M. Hersen (Eds.), Research fraud in the behavioral and biomedical sciences (pp. 225–244). New York, NY: John Wiley.Google Scholar
  87. Harnad, S. (1982). Peer commentary on peer review: A case study in scientific quality control. New York, NY: Cambridge University Press.Google Scholar
  88. Hearn, J. C., & Anderson, M. S. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43(5), 503–529.CrossRefGoogle Scholar
  89. Hermanowicz, J. C. (1998). The stars are not enough: Scientists—Their passions and professions. Chicago, IL: University of Chicago Press.Google Scholar
  90. Hermanowicz, J. C. (2009). Lives in science: How institutions shape academic careers. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
  91. Hermanowicz, J. C. (2016a). The proliferation of publishing: Economic rationality and ritualized productivity in a neoliberal era. American Sociologist, 47, 174–191.CrossRefGoogle Scholar
  92. Hermanowicz, J. C. (2016b). Universities, academic careers, and the valorization of ‘shiny things.’. Research in the Sociology of Organizations, 46, 303–328.CrossRefGoogle Scholar
  93. Herrera, A. J. (1999). Language bias discredits the peer-review system. Nature, 297(6719), 467.CrossRefGoogle Scholar
  94. Hess, D. J. (1997). Science studies: An advanced introduction. New York, NY: New York University Press.Google Scholar
  95. Hirschauer, S. (2010). Editorial judgments: A praxeology of ‘voting’ in peer review. Social Studies of Science, 40(1), 71–103.CrossRefGoogle Scholar
  96. Hood, J. (1985). The lone scholar myth. In M. F. Fox (Ed.), Scholarly writing and publishing (pp. 111–124). Boulder, CO: Westview Press.Google Scholar
  97. Horton, R. (2000). Genetically modified food: consternation, confusion, and crack-up. Medical Journal of Australia, 177, 148–149.Google Scholar
  98. Jacobs, J. A. (2013). In defense of disciplines: Interdisciplinarity and specialization in the research university. Chicago,IL: University of Chicago Press.Google Scholar
  99. Jacobs, J. A., & Winslow, S. E. (2004). Overworked faculty: Job stresses and family demands. The Annals of the American Academy of Political and Social Science, 596(1), 104–129.CrossRefGoogle Scholar
  100. Jauch, L. R., & Wall, J. L. (1989). What they do when they get your manuscript: A survey of Academy of Management reviewer practices. Academy of Management Journal, 32(1), 157–173.CrossRefGoogle Scholar
  101. Judson, H. F. (1994). Structural transformations of the sciences and the end of peer review. Journal of the American Medical Association, 272(2), 92–94.CrossRefGoogle Scholar
  102. Juhasz, S., Calvert, E., Jackson, T., Kronick, D., & Shipman, J. (1975). Acceptance and rejection of manuscripts. IEEE Transactions on Professional Communication, 18(3), 177–185.CrossRefGoogle Scholar
  103. Katz, D. S., Proto, A. V., & Olmsted, W. W. (2002). Incidence and nature of unblinding by authors: Our experience at two radiology journals with double-blinded peer review policies. American Journal of Roentgenology, 179(6), 1415–1417.CrossRefGoogle Scholar
  104. Kerr, S., Tolliver, J., & Petree, D. (1977). Manuscript characteristics which influence acceptance for management and social science journals. Academy of Management Journal, 20(1), 132–141.CrossRefGoogle Scholar
  105. Knight, J. (2003). Negative results: Null and void. Nature, 422(6932), 554–555.CrossRefGoogle Scholar
  106. Knorr-Cetina, K. (1981). The manufacture of knowledge: An essay on the constructivist and contextual nature of science. Oxford,UK: Pergamon Press.Google Scholar
  107. Kravitz, D. J., & Baker, C. I. (2011). Toward a new model of scientific publishing: Discussion and a proposal. Frontiers in Computational Neuroscience, 5(55), 1–12.Google Scholar
  108. Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.Google Scholar
  109. Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
  110. Langfeldt, L. (2001). The decision-making constraints and processes of grant peer review, and their effects on the review outcome. Social Studies of Science, 31(6), 820–841.CrossRefGoogle Scholar
  111. Lawrence, J. H., Celis, S., & Ott, M. (2014). Is the tenure process fair? What faculty think. Journal of Higher Education, 85(2), 155–192.CrossRefGoogle Scholar
  112. Leahey, E., & Cain, C. L. (2013). Straight from the source: Accounting for scientific success. Social Studies of Science, 43(6), 927–951.CrossRefGoogle Scholar
  113. Lee, C. J., & Schunn, C. D. (2011). Social biases and solutions for procedural objectivity. Hypatia: A Journal of Feminist Philosophy, 26(2), 352–373.CrossRefGoogle Scholar
  114. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.CrossRefGoogle Scholar
  115. Levenson, H., Burford, B., Bonno, B., & Davis, L. (1975). Are women still prejudiced against women? A replication and extension of Goldberg’s study. Journal of Psychology, 89, 67–71.CrossRefGoogle Scholar
  116. Liebert, R. J. (1976). Productivity, favor, and grants among scholars. American Journal of Sociology, 82(3), 664–673.CrossRefGoogle Scholar
  117. Link, A. M. (1998). US and non-US submissions. Journal of the American Medical Association, 280(3), 246–247.Google Scholar
  118. Lloyd, M. E. (1990). Gender factors in reviewer recommendations for manuscript publication. Journal of Applied Behavior Analysis, 23(4), 539–543.CrossRefGoogle Scholar
  119. Long, J. S., Allison, P. D., & McGinnis, R. (1993). Rank advancement in academic careers: Sex differences and the effects of productivity. American Sociological Review, 58, 703–722.CrossRefGoogle Scholar
  120. Long, J. S., & Fox, M. F. (1995). Scientific careers: Universalism and particularism. Annual Review of Sociology, 21, 45–71.CrossRefGoogle Scholar
  121. Loonen, M. P. J., Hage, J. J., & Kon, M. (2005). Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery. Plastic and Reconstructive Surgery, 116(5), 1461–1472.CrossRefGoogle Scholar
  122. Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1(2), 161–175.CrossRefGoogle Scholar
  123. McAfee, R. P. (2014). Edifying editors. In M. Szenberg & L. Ramrattan (Eds.), Secrets of Economics Editors (pp. 33–44). Cambridge, MA: MIT Press.Google Scholar
  124. McLachlan, J. C. (2010). Integrative medicine and the point of credulity. BMJ, 341, c6979.CrossRefGoogle Scholar
  125. McNutt, R. A., Evans, A. T., Fletcher, R. H., & Fletcher, S. W. (1990). The effects of blinding on the quality of review. JAMA, 263(10), 1371–1376.CrossRefGoogle Scholar
  126. Merton, R. K. (1973a). The normative structure of science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 267–278). Chicago, IL: University of Chicago Press (Article originally published in 1942).Google Scholar
  127. Merton, R. K. (1973b). Priorities in scientific discovery. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 286–324). Chicago, IL: University of Chicago Press (Article originally published in 1957).Google Scholar
  128. Merton, R. K. (1973c). The Matthew effect in science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 439–459). Chicago, IL: University of Chicago Press (Article originally published in 1968).Google Scholar
  129. Merton, R. K., & Zuckerman, H. (1973). Age, aging, and the age structure in science. In N. W. Storer (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 497–559). Chicago, IL: University of Chicago Press (Article originally published in 1972).Google Scholar
  130. Moossy, J., & Moossy, Y. R. (1985). Anonymous authors, anonymous referees: An editorial explanation. Journal of Neuropathology and Experimental Neurology, 44(3), 225–228.CrossRefGoogle Scholar
  131. Morton, H. C., & Price, A. J. (1986). The ACLS survey of scholars: Views on publications, computers, libraries. Scholarly Communication, 5, 1–16.Google Scholar
  132. Mulkay, M. (1980). Interpretation and the use of rules: The case of the norms of science. In T. F. Gieryn (Ed.), Science and social structure: A Festschrift for Robert K. Merton (pp. 111–125). New York, NY: New York Academy of Sciences.Google Scholar
  133. National Science Board. (2010). Science and engineering indicators 2010. Arlington, VA: National Science Foundation.Google Scholar
  134. Olson, C. M., Rennie, D., Cook, D., Dickersin, K., Flanagin, A., Hogan, J., … Pace, B. (2002). Publication bias in editorial decision making. Journal of the American Medical Association, 287(21), 2825–2828.Google Scholar
  135. O’Meara, K. A. (2004). Beliefs about post-tenure review: The influence of autonomy, collegiality, career stage, and institutional context. Journal of Higher Education, 75(2), 178–202.CrossRefGoogle Scholar
  136. Opthof, T., Coronel, R., & Janse, M. (2002). The significance of the peer review process against the background of bias: Priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias. Cardiovascular Research, 56(3), 339–346.CrossRefGoogle Scholar
  137. Orr, R. & Kassab, J. (1965). Peer group judgments on scientific merit: Editorial refereeing. Presentation to the Congress of the International Federation for Documentation, Washington, DC. October 15.Google Scholar
  138. Paludi, M. A., & Bauer, W. D. (1983). Goldberg revisited: What’s in an author’s name? Sex Roles, 9(3), 387–390.CrossRefGoogle Scholar
  139. Patriquin, L., Bensimon, E. M., Polkinghorne, D. E., Bauman, G., Bleza, M. G., Oliverez, P. M., & Soto, M. (2003). Posttenure review: The disparity between intent and implementation. Review of Higher Education, 26(3), 275–297.CrossRefGoogle Scholar
  140. Perna, L. W. (2001). Sex and race differences in faculty tenure and promotion. Research in Higher Education, 42(5), 541–567.CrossRefGoogle Scholar
  141. Perna, L. W. (2003). Studying faculty salary equity: A review of theoretical and methodological approaches. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 18, pp. 323–388).CrossRefGoogle Scholar
  142. Perna, L. W. (2005). Sex differences in faculty tenure and promotion: The contribution of family ties. Research in Higher Education, 46(3), 277–307.CrossRefGoogle Scholar
  143. Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of accepted, published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195.Google Scholar
  144. Polanyi, M. (1946). Science, faith, and society. Chicago, IL: University of Chicago Press.Google Scholar
  145. Powell, W. W. (1985). Getting into print: The decision-making process in scholarly publishing. Chicago, IL: University of Chicago Press.Google Scholar
  146. Power, M. (1997). The audit society: Rituals of verification. Oxford, UK: Oxford University Press.Google Scholar
  147. Price, D. J. (1975). Science since Babylon. New Haven, CT: Yale University Press.Google Scholar
  148. Primack, R. B., Ellwood, E., Miller-Rushing, A. J., Marrs, R., & Mulligan, A. (2009). Do gender, nationality, or academic age affect review decisions? An analysis of submission to the journal Biological Conservation. Biological Conservation, 142, 2415–2418.CrossRefGoogle Scholar
  149. Primack, R. B., & Marrs, R. (2008). Bias in the review process. Biological Conservation, 141(12), 2919–2920.CrossRefGoogle Scholar
  150. Ramos-Alvarez, M. M., Moreno-Fernandez, M. M., Valdes-Conroy, B., & Catena, A. (2008). Criteria of the peer review process for publication of experimental and quasi-experimental research in psychology: A guide for creating research papers. International Journal of Clinical and Health Psychology, 8(3), 751–764.Google Scholar
  151. Ripp, A. (1985). Peer review is alive and well in the United States. Science, Technology, and Human Values, 10(3), 82–86.CrossRefGoogle Scholar
  152. Rosenblatt, A., & Kirk, S. A. (1981). Recognition of authors in blind review of manuscripts. Journal of Social Science Research, 3(4), 383–394.Google Scholar
  153. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.CrossRefGoogle Scholar
  154. Rowson, R. C. (1994). A formula for successful scholarly publishing: Policy-oriented research and the humanities. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 195–208). Lanham, MD: Rowman & Littlefield Publishers.Google Scholar
  155. Roy, R. (1985). Funding science: The real defects of peer review and an alternative to it. Science, Technology, and Human Values, 10(3), 73–81.CrossRefGoogle Scholar
  156. Schuster, J. H., & Finkelstein, M. J. (2006). The American faculty: The restructuring of academic work and careers. Baltimore, MD: Johns Hopkins University Press.Google Scholar
  157. Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 498–502.CrossRefGoogle Scholar
  158. Shadish Jr., W. R. (1989). The perception and evaluation of quality in science. In B. Gholson, W. R. Shadish Jr., R. A. Neimeyer, & A. C. Houts (Eds.), Psychology of science: Contributions to metascience (pp. 383–426). Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  159. Shadish Jr., W. R., Doherty, M., & Montgomery, L. M. (1989). How many studies in the file drawer? An estimate from the family/marital psychotherapy literature. Clinical Psychology Review, 9(5), 589–603.CrossRefGoogle Scholar
  160. Shatz, D. (2004). Peer review: A critical inquiry. Lanham, MD: Rowman & Littlefield.Google Scholar
  161. Shils, E. (1997). The order of learning: Essays on the contemporary university. New Brunswick, NJ: Transaction.Google Scholar
  162. Siler, K., Lee, K., & Bero, L. (2015). Measuring the effectiveness of scientific gatekeeping. Proceedings of the National Academy of Sciences, 112(2), 360–365.CrossRefGoogle Scholar
  163. Silverman, R. (1988). Peer judgment: An ideal typification. Knowledge: Creation, Diffusion, Utilization, 9, 362–382.Google Scholar
  164. Simon, R. J., Bakanic, V., & McPhail, C. (1986). Who complains to journal editors and what happens. Sociological Inquiry, 56(2), 259–271.CrossRefGoogle Scholar
  165. Smart, R. G. (1964). The importance of negative results in psychological research. Canadian Psychology, 5, 225–232.CrossRefGoogle Scholar
  166. Smigel, E. O., & Ross, H. L. (1970). Factors in the editorial decision. The American Sociologist, 5(1), 19–21.Google Scholar
  167. Smith, J. A., Nixon, R., Bueschen, A. J., Venable, D. D., & Henry, H. H. (2002). The impact of blinded versus unblended abstract review on scientific program content. Journal of Urology, 168(5), 2123–2125.CrossRefGoogle Scholar
  168. Sokal, A. D. (1996). A physicists experiments with cultural studies. Lingua Franca, 6(4), 62–64.Google Scholar
  169. Starbuck, W. H. (2003). Turning lessons into lemonade: Where is the value in peer reviews? Journal of Management Inquiry, 12(4), 344–351.CrossRefGoogle Scholar
  170. Sterling, T. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—Or vice versa. Journal of the American Statistical Association, 54(285), 30–34.Google Scholar
  171. Stossel, T. P. (1985). Refinement in biomedical communication: A case study. Science, Technology, & Human Values, 10(3), 39–43.CrossRefGoogle Scholar
  172. Strang, D., & Siler, K. (2015). Revising as reframing: Original submissions versus published papers in Administrative Science Quarterly, 2005–2009. Sociological Theory, 33(1), 71–96.CrossRefGoogle Scholar
  173. Sugimoto, C. R., Larivière, V., Ni, C., & Cronin, B. (2013). Journal acceptance rates: a cross-disciplinary analysis of variability and relationships with journal measures. Journal of Informatics, 7(4), 897–906.Google Scholar
  174. Tenopir, C., & King, D. W. (2009). The growth of journals publishing. In B. Cope & A. Phillips (Eds.), The future of the academic journal (pp. 105–124). Oxford, UK: Chandos Publishing.CrossRefGoogle Scholar
  175. Teplitskiy, M. (2015). Frame search and re-search: How quantitative sociological articles change during peer review. American Sociologist. doi: 10.1008/s12108-015-9288-3.Google Scholar
  176. Thatcher, S. G. (1994). Listbuilding at university presses. In R. J. Simon & J. J. Fyfe (Eds.), Editors as gatekeepers: Getting published in the social sciences (pp. 209–258). Lanham, MD: Rowman & Littlefield Publishers.Google Scholar
  177. Travis, G., & Collins, H. M. (1991). New light on old boys: cognitive and institutional particularism in the peer review system. Science, Technology, & Human Values, 16(3), 322–341.CrossRefGoogle Scholar
  178. Tuchman, G. (2009). Wannabe U: Inside the Corporate University. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
  179. Valkonen, L., & Brooks, J. (2011). Gender balance in Cortex acceptance rates. Cortex, 47, 763–770.CrossRefGoogle Scholar
  180. Wanderer, J. J. (1966). Academic origins of contributors to the “American Sociological Review”, 1955–1965. The American Sociologist, 1(5), 241–243.Google Scholar
  181. Ward, C. (1981). Prejudice against women: Who, when, why? Sex Roles, 7(2), 163–171.CrossRefGoogle Scholar
  182. Ware, M. & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community—An international study. Publishing Research Consortium.Google Scholar
  183. Waters, M. (1989). Collegiality, bureaucratization, and professionalization: A Weberian Analysis. American Journal of Sociology, 94(5), 945–972.CrossRefGoogle Scholar
  184. Webb, T. J., O’Hara, B., & Freckleton, R. P. (2008). Does double-blind review benefit female authors? Trends in Ecology and Evolution, 23(7), 351–353.CrossRefGoogle Scholar
  185. Weisse, A. B. (1986). Say it isn’t no: Positive thinking and the publication of medical research. Hospital Practice, 21(3), 23–25.CrossRefGoogle Scholar
  186. Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. Medford, NJ: American Society for Information Science and Technology.Google Scholar
  187. Wessely, S. (1996). What do we know about peer review? Psychological Medicine, 26, 883–886.CrossRefGoogle Scholar
  188. Wessely, S., Brugha, T., Cowen, P., Smith, L., & Paykel, E. (1996). Do authors know who refereed their paper? A questionnaire survey. BMJ, 313(7066), 1185.CrossRefGoogle Scholar
  189. Whittaker, R. J. (2008). Journal review and gender equality: A critical comment on Budden et al. Trends in Ecology and Evolution, 23(9), 478–479.CrossRefGoogle Scholar
  190. Wilhite, A. W., & Fong, E. A. (2012). Coercive citation in academic publishing. Science, 335(6068), 542–543.CrossRefGoogle Scholar
  191. Wood, M., & Johnsrud, L. (2005). Post-tenure review: What matters to faculty. Review of Higher Education, 28(3), 393–420.CrossRefGoogle Scholar
  192. Yankauer, A. (1990). Who are the peer reviewers and how much do they review? JAMA, 263(10), 1338–1340.CrossRefGoogle Scholar
  193. Yankauer, A. (1991). How blind is blind review? American Journal of Public Health, 81(7), 843–845.CrossRefGoogle Scholar
  194. Youn, T. I. K., & Price, T. M. (2009). Learning from the experience of others: The evolution of faculty tenure and promotion rules in comprehensive institutions. Journal of Higher Education, 80(2), 204–237.CrossRefGoogle Scholar
  195. Ziman, J. M. (1968). Public knowledge: An essay concerning the social dimension of science. Cambridge, UK: Cambridge University Press.Google Scholar
  196. Zuckerman, H. (1977). Scientific elite: Nobel laureates in the United States. New York, NY: The Free Press.Google Scholar
  197. Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: Institutionalization, structure and the functions of the referee system. Minerva, 9, 66–100.CrossRefGoogle Scholar
  198. Zuckerman, H. (1988). Sociology of science. In N. J. Smelser (Ed.), Handbook of sociology (pp. 511–574). Newbury Park, CA: Sage.Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.College of EducationThe University of Nevada – RenoRenoUSA
  2. 2.Department of SociologyThe University of GeorgiaAthensUSA

Personalised recommendations