Abstract
More recent advancements in digital technologies have significantly alleviated the dissemination of new scientific ideas as well as the storing, searching and retrieval of large amounts of published research findings. While not denying the benefits of this novel ‘economy of memory,’ this paper endeavors to shed light on the ways in which the use of digital technologies may be linked to a distortion of the system of formal publications that facilitates the effective dissemination and collaborative building of scientific knowledge. Through combining three different strands of discussion that are often left separate – those pertaining to the cognitive effects of new technological memory systems, those pertaining to citation and publishing practices, and those regarding the effects of formalizing modes of research governance – it is also shown that this distortion is not merely a consequence of technological developments alone. Rather, such a distortion is inseparable from and potentially aggravated by the spreading of increasingly dysfunctional, formalizing research governance mechanisms. It is argued that these mechanisms run the risk of fostering the proliferation of knowledge practices that are characterized by an increasing degree of superficiality as well as the strategic publication of research that is of a decreasing degree of originality. If left unaddressed, this may pose a serious threat to the efficiency and effectiveness of the formal record of scientific knowledge as a tool for the dissemination of original research. By extension, this may in the long run seriously undermine the capacity of the publicly funded research system more generally.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
My use of the concepts ‘science’ and ‘scientific’ throughout this paper tends to be inclusive rather than exclusive, unless otherwise noted. Reflecting this more inclusive notion, some attention will be devoted to the social sciences (see, e.g., section “Citation Practices”). At the same time, the majority of examples used in this paper refer to the natural and technical sciences, for these are the domains where some of the developments to be discussed play out most strongly.
The concept of ‘scientific culture’ is generally meant here to signify a specialized mode of collaborative cognitive practice aiming for the systematic development and critical examination of more or less rational forms knowledge – and which at the same time is a product of specific cultural and historical processes. For a highly instructive account of the emergence of scientific culture and that of its distinctive set of cognitive values in early modern Europe, see Gaukroger (2006). For a range of instructive case studies pertaining to the formation and historical development of scientific culture from the perspective of the sociology of scientific knowledge, see Barnes and Shapin (1979). In the context of such sociological investigations, considerable attention has been paid to the role that writing and the related practices, both epistemic and rhetoric, have played in the historical formation of early modern scientific culture (Bazerman 1981; Bazerman 1988, Part II; Yearly 1981). At the same time, focusing on more contemporary contexts of the production and dissemination of scientific knowledge, researchers have also extensively studied the epistemic and rhetorical use and usefulness of formal and informal textual practices and representations (e.g., Gusfield 1976; Gilbert 1977; Latour 1979; Knorr and Knorr 1978).
As Eisenstein observes in this regard, the use of print “made food for thought much more abundant and allowed mental energies to be more efficiently used” (1983: 259). Eisenstein (1979) goes on to argue that, from a historical perspective, this process of rationalization played a crucial role in the process that led to the emergence of modern scientific culture and thinking. This, she claims, is because the use of print led to a standardization not only of languages but also of knowledge, which in turn helped foster an epistemic culture that effectively relies on mutual criticism and collaboration.
Shaffer and Kaput (1999) even go as far as proposing that the rapid diffusion of such digital media and technologies is synonymous with the evolutionary transition to a new stage of cognitive activity that they refer to as ‘virtual culture.’ Fundamental to this new cognitive stage is the accomplishment of the technological externalization of symbol-processing operations, which requires computational media that are inherently dynamic, that is, capable of changing their state, without the ongoing guidance from the operator (Shaffer and Clinton 2006: 287).
What is meant by this is that the cognitive processes at play are functionally built upon and involve specific types of coordination between various cognitively efficacious elements such as minds, living bodies and a range of environmental structures, technological artifacts, for instance. Importantly, theories and approaches working with a distributed cognition focus are specifically interested in exploring the nature of the relations and dynamic interactions taking place between elements rather than these elements’ inherent properties. This interest in relations and interactions explains their alignment with ecological approaches and perspectives (Hutchins 2010a).
To clarify, my discussion concerning the distortion of the formal record of scientific knowledge is generally neutral with regard to the question concerning the validity of the “categorical distinction between genuine and distorted scientific knowledge” (Miller 2009: 261). This is in the sense that my discussion is primarily focused on elucidating the nature of the real-world processes that facilitate, shape and constrain the constitution, dissemination and reproduction of scientific knowledge, while refraining from making sweeping claims about the intrinsic nature and epistemological status of this knowledge itself.
More recently, there has also been a trend emerging in some countries such as the UK toward developing and implementing more complex variants of impact that take into account the broader social and economic benefits of research. It is still too early to evaluate the feedback effect these new regimes of research evaluation have on researchers and their institutions.
It has been shown that such strategic authorship is particularly prevalent in those domains of scientific research that are not merely highly competitive but that also a have strong potential for commercialization. Striking examples are the medical and pharmaceutical sciences, where (academic) publication activities are in many instances organized by professional publication planners, and where the production and dissemination of research findings in academic journals are “ghost-managed” by commercial sponsors (Sismondo 2009).
It has been recommended that such editorial practices ought to include the systematic use of already available technologies and forensic methods that assist in identifying redundant publications and reproduced or manipulated sets of data (see Errami et al. 2010; Marris 2006; Sun et al. 2010). An unintended effect of the use of such methods and technologies may, however, be a narrowing of the scope for legitimate and contained forms of repeating in a publication. Such forms may comprise, for example, publications by authors that have the purpose of situating their novel research findings in the context of their past research (and which thus involve some degree of repetition), or publications that serve the purpose of communicating research findings to different audiences.
The problems associated with using, in various forms, (number of) citations as a measure of research impact, and even more so, of research quality are legion. For instance, it seems obvious that there cannot be any straight form of correlation between the number of citations and quality of the paper published, for the simple reason that, in some instances, and for some time after publication, a paper of a lower quality, one with a catchy title but little substance for instance, or, a paper that offers nothing new but is published by an authority in the field, or even one that is just ‘marketed’ most effectively, are cited most often (see Andras 2011: 98). For a comprehensive discussion of (quantitative) forms of citation analysis, the construction of indicators, and of issues such as proper use of citation data in policy contexts as well as potential limitations of such forms of analysis, see Moed (2005). For a comprehensive overview of the state-of-the-art of citation analysis, including qualitative approaches, see Bornmann and Daniel (2006).
See Bornmann and Daniel (2006, 50–58) for a discussion of various influential content- and context-sensitive citation studies and the taxonomies they employ.
These findings to some extent align with those of a similar study conducted by Eichhorn and Yankauer (1987) examining quotation and citation accuracy in public health articles. The authors of this study found that out of the 150 references checked, 46 (31%) contained some sort of citation error, and 45 (30%) some minor or major error of quotation.
For a relatively recent study exploring the scope of the problem of accuracy in referencing in the (medical-)scientific literature, see Sieber and Holt (2000).
For an overview of how these developments play out in a number of countries, including the UK and Australia, see Geuna and Martin (2003). For more recent accounts of the situation in the UK and Australia and its perceived effects, see Adams (2009), Barker (2007), Gläser and Laudel (2007), and Lewis and Ross (2011). An instructive typological discussion of research governance systems and associated mechanisms along criteria such as degree of formalization, exogeneity of authority, strength and extent of enforcement is provided by Whitley (2011).
Butler’s findings are undeniably telling, however, one should note that she, due to the large-scale quantitative nature of her studies, has to rely on a rather superficial measure of quality, namely citation impact.
One should not forget here that ultimately, the genuine quality of published research can only be reasonably determined if taking into account the content and characteristics of the publication itself, rather than mainly focusing on the journal where it is published, or on the number of citations the publication receives. As Brenner aptly comments in this regard: “Before we develop a pseudo-science of citation analysis, we should remind ourselves that what matters absolutely is the scientific content of a paper and that nothing will substitute for either knowing or reading it” (Brenner 1995: 568).
References
Abbott, Andrew. 1988. The system of professions: An essay on the division of expert labor. Chicago: University of Chicago Press.
Abbott, Andrew. 2010. Varieties of ignorance. American Sociologist 41: 174–189.
Adams, Jonathan. 2009. The use of bibliometrics to measure research quality in UK higher education institutions. Archivum Immunologiae et Therapiae Experimentalis 57: 19–32.
Andras, Peter. 2011. Research: Metrics, quality, and management implications. Research Evaluation 20: 90–106.
Barker, Katherine. 2007. The UK Research Assessment Exercise: The evolution of a national research evaluation system. Research Evaluation 16: 3–12.
Barnes, Barry, and Steven Shapin (eds.). 1979. The natural order: Historical studies of scientific culture. London: Sage.
Bazerman, Charles. 1981. What written knowledge does: Three examples of academic discourse. Philosophy of the Social Sciences 11: 361–387.
Bazerman, Charles. 1988. Shaping written knowledge: The genre and activity of the experimental article in science. Madison, WI: University of Wisconsin Press.
Bornmann, Lutz. 2011. Mimicry in science? Scientometrics 86: 173–177.
Bornmann, Lutz, and Hans-Dieter Daniel. 2006. What do citations measure? A review of studies on citing behavior. Journal of Documentation 64: 45–80.
Brenner, Sydney. 1995. Loose ends. Current Biology 5: 568.
Butler, Linda. 2003. Explaining Australia’s increased share of ISI publications—The effects of a funding formula based on publication counts. Research Policy 32: 143–155.
Butler, Linda. 2004. What happens when funding is linked to publication counts? In Handbook of quantitative science and technology research: The use of publication and patent statistics in studies of S&T systems, eds. Henk F. Moed, Wolfgang Glänzel, and Ulrich Schmoch, 389–405. Dordrecht: Kluwer.
Cozzens, Susan E. 1989. What do citations count? The rhetoric-first model. Scientometrics 15: 437–447.
de Solla-Price, Derek J. 1986. Little science, big science …and beyond. New York: Columbia University Press.
Donald, Merlin. 1991. Origins of the modern mind: Three stages in the evolution of culture and cognition. Cambridge, MA: Harvard University Press.
Eichorn, Philip, and Alfred Yankauer. 1987. Do authors check their references? A Survey of accuracy of references in three public health journals. American Journal of Public Health 77: 1011–1012.
Eisenstein, Elizabeth L. 1979. The printing press as an agent of change: Communications and cultural transformations in early modern Europe. Cambridge: Cambridge University Press.
Eisenstein, Elizabeth L. 1983. The printing revolution in early modern Europe. Cambridge: Cambridge University Press.
Enders, Walter, and Gary A. Hoover. 2004. Whose line is it? Plagiarism in economics. Journal of Economic Literature 42: 487–493.
Errami, Mounir, Zhaohui Sun, Angela C. George, Tara C. Long, Michael A. Skinner, Jonathan D. Wren, and Harold R. Garner. 2010. Identifying duplicate content using statistically improbable phrases. Bioinformatics 26: 1453–1457.
Evans, James A. 2008. Electronic publication and the narrowing of science and scholarship. Science 321: 395–399.
Evans, James A., and Jacob Reimer. 2009. Open access and global participation in science. Science 323: 1025.
Evans, James A., and Andrey Rzhetsky. 2010. Machine science. Science 329: 399–400.
Evans, James T., Howard I. Nadjari, and Sherry A. Burchell. 1990. Quotational and reference accuracy in surgical journals. A continuing peer review problem. JAMA: Journal of the American Medical Association 263: 1353–1354.
Gaukroger, Stephen. 2006. The emergence of a scientific culture: Science and the shaping of modernity 1210–1685. Oxford: Oxford University Press.
Geuna, Aldo, and Ben R. Martin. 2003. University research evaluation and funding: An international comparison. Minerva 41: 277–304.
Gilbert, G. Nigel. 1977. Referencing as persuasion. Social Studies of Science 7: 113–122.
Gläser, Jochen, and Grit Laudel. 2007. Evaluation without evaluators: The impact of funding formulae on Australian University research. In The Changing governance of the sciences: The advent of research evaluation systems, eds. Richard Whitley, and Jochen Gläser, 127–151. Dordrecht: Springer.
Goody, Jack. 1977. The domestication of the savage mind. Cambridge: Cambridge University Press.
Gusfield, Joseph. 1976. The literary rhetoric of science: Comedy and pathos in drinking driver research. American Sociological Review 41: 16–34.
Harris, Roy. 1989. How does writing restructure thought? Language & Communication 9: 99–106.
Havelock, Eric A. 1982. The literate revolution in Greece and its cultural consequences. Princeton, NJ: Princeton University Press.
Hutchins, Edwin. 1995. Cognition in the wild. Cambridge, MA: MIT Press.
Hutchins, Edwin. 2001. Cognitive artifact. In The MIT encyclopedia of the cognitive sciences, eds. Robert A. Wilson, and Frank C. Keil, 126–128. Cambridge, MA: MIT Press.
Hutchins, Edwin. 2010a. Cognitive ecology. Topics in Cognitive Science 2: 705–715.
Hutchins, Edwin. 2010b. Distributed cognition. In The international encyclopedia of the social and behavioral sciences, eds. Neil J. Smelser, and Paul B. Baltes, 2068–2072. Amsterdam: Elsevier.
Kaplan, Norman. 1965. The norms of citation behavior: Prolegomena to the footnote. American Documentation 16: 179–184.
Kirsh, David. 1995. The intelligent use of space. Artificial Intelligence 73: 31–68.
Kirsh, David. 2000. A few thoughts on cognitive overload. Intellectia 30: 19–51.
Kirsh, David. 2006. Distributed cognition: A methodological note. Pragmatics & Cognition 14: 249–262.
Knorr, Karin D., and Dietrich W. Knorr. 1978. From scenes to scripts: On the relationship between laboratory research and published paper in science. Vienna: Institute for Advanced Studies; Ithaca, NY: Cornell University.
Kostoff, Ronald N., J. Dustin Johnson, Antonio Del Rio, Louis A. Bloomfield, Michael F. Shlesinger, Guido Malpohl, and Hector D. Cortes. 2006. Duplicate publication and ‘paper inflation’ in the fractals literature. Science and Engineering Ethics 12: 543–554.
Lariviere, Vincent, and Yves Gingras. 2010. On the prevalence and scientific impact of duplicate publications in different scientific fields (1980–2007). Journal of Documentation 66: 179–190.
Latour, Bruno, and Steve Woolgar. 1979. Laboratory life: The social construction of a scientific fact. Beverly Hills: Sage.
Lewis, Jenny, and Sandy Ross. 2011. Research funding systems in Australia, New Zealand and the UK: Policy settings and perceived effects. Policy & Politics 39: 379–398.
Long, Tara C., Mounir Errami, Angela C. George, Zhaohui Sun, and Harold R. Garner. 2009. Responding to possible plagiarism. Science 323: 1293–1294.
Marris, Emma. 2006. Should journals police scientific fraud? Nature 439: 520–521.
Merton, Robert K. 1973. The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.
Miller, Boaz. 2009. What does it mean that PRIMES is in P? Popularization and distortion revisited. Social Studies of Science 39: 257–288.
Moed, Henk F. 2005. Citation analysis in research evaluation. Dordrecht: Springer.
Mojon-Azzi, Stefania M., Xiaoyi Jiang, Ulrich Wagner, and Daniel S. Mojon. 2004. Redundant publications in scientific ophthalmologic journals—The tip of the iceberg? Ophthalmology 111: 863–866.
Norman, Donald A. 1991. Cognitive artifacts. In Designing interaction: Psychology at the human–computer interface, ed. John M. Carroll, 17–38. Cambridge: Cambridge University Press.
Norman, Donald A. 1993. Things that make us smart: Defending human attributes in the age of the machine. New York: Basic Books.
Persson, Olle, Wolfgang Glänzel, and Rickard Danell. 2004. Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics 60: 421–432.
Power, Michael. 1997. The audit society: Rituals of verification. Oxford: Oxford University Press.
Power, Michael. 2004. Counting, control and calculation: Reflections on measuring and management. Human Relations 57: 765–783.
Power, Michael. 2005. The theory of the audit explosion. In The Oxford handbook of public management, eds. Ewan Ferlie, Laurence E. Lynn, and Christopher Pollitt, 326–344. Oxford: Oxford University Press.
Roig, Miguel. 2005. Re-using text from one’s own previously published papers: An exploratory study of potential self-plagiarism. Psychological Reports 97: 43–49.
Schein, Moshe, and Rameh Paladugu. 2001. Redundant surgical publications: Tip of the iceberg? Surgery 129: 655–661.
Schimank, Uwe. 2005. “New Public Management” and the academic profession: Reflections on the German situation. Minerva 43: 361–376.
Shaffer, David W., and Katherine A. Clinton. 2006. Toolforthoughts: Reexamining thinking in the digital age. Mind, Culture, and Activity 13: 283–300.
Shaffer, David W., and James J. Kaput. 1999. Mathematics and virtual culture: An evolutionary perspective on technology and mathematics education. Educational Studies in Mathematics 37: 97–119.
Shannon, Claude E., and Warren Weaver. 1949. The mathematical theory of communication. Urbana, IL: University of Illinois Press.
Sieber, Robert W., and Shaun Holt. 2000. Accuracy of references in five leading medical journals. Lancet 356: 1445.
Simkin, Mikhail, and Vwani P. Roychowdhury. 2005. Stochastic modeling of citation slips. Scientometrics 62: 367–384.
Sismondo, Sergio. 2009. Ghosts in the machine: Publication planning in the medical sciences. Social Studies of Science 39: 171–198.
Stichweh, Rudolf. 1992. The sociology of scientific disciplines: On the genesis and stability of the disciplinary structure of modern science. Science in Context 5: 3–15.
Sun, Zhaohui, Mounir Errami, Tara Long, Chris Renard, Nishant Choradia, and Harold Garner. 2010. Systematic characterizations of text similarity in full text biomedical publications. PloS ONE 5: e12704. doi:10.1371/journal.pone.0012704.
Sutton, John. 2006. Distributed cognition: Domains and dimensions. Pragmatics & Cognition 14: 235–247.
Tenopir, Carol, Donald W. King, Sheri Edwards, and Wu Lei. 2009. Electronic journals and changes in scholarly article seeking and reading patterns. Aslib Proceedings 61: 5–32.
von Elm, Erik, Greta Poglia, Bernhard Walder, and Martin R. Tramer. 2004. Different patterns of duplicate publication: An analysis of articles used in systematic reviews. JAMA: Journal of the American Medical Association 291: 974–980.
Weingart, Peter. 2005. Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics 62: 117–131.
Whitley, Richard. 2011. Changing governance and authority relations in the public sciences. Minerva 49(4): 359–385.
Yearly, Steven. 1981. Textual persuasion: The role of social accounting in the construction of scientific arguments. Philosophy of the Social Sciences 11: 409–435.
Acknowledgments
I wish to acknowledge Australian Research Council funding support for the research project ‘Knowledge Building in Schooling and Higher Education: Policy strategies and effects’ (ARC Discovery Project 2011–13, DP110102466, Chief Investigator Prof. Lyn Yates). I would also like to thank Lyn Yates as well as the anonymous reviewers for their helpful feedback and suggestions.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Woelert, P. The ‘Economy of Memory’: Publications, Citations, and the Paradox of Effective Research Governance. Minerva 51, 341–362 (2013). https://doi.org/10.1007/s11024-013-9232-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11024-013-9232-z