Skip to main content
SpringerLink
Log in

Measuring Scholarly Impact

  • Chapter
  • First Online:
Representing Scientific Knowledge

Abstract

The ability to measure scholarly impact, ranging from individual scientists to an institution of researchers, is crucial to both research assessment and the advance of science itself. In this chapter, we summarize an array of fundamental and widely used concepts and computational methods for measuring scholarly impact as well as identifying more generic properties such as semantic relatedness, burstness, clumping, and centrality. Most of these common ideas are applicable to a wide variety of needs as long as we can identify the profound issues that are in common across distinct phenomena. Normalizations of metrics across scientific fields and the year of publication are discussed with concrete examples.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 74.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://code.google.com/p/ws4j/.

  2. 2.

    https://aclweb.org/aclwiki/RG-65_Test_Collection_(State_of_the_art).

  3. 3.

    Fletcher, Jack McKay and Wennekers, Thomas (2017).  From Structure to Activity: Using Centrality Measures to Predict Neuronal Activity. International Journal of Neural Systems. 0 (0): 1750013. doi:10.1142/S0129065717500137.

References

  • Banerjee S, Pedersen T (2002) An adapted Lesk algorithm for word sense disambiguation using WordNet. In: Gelbukh A (ed) CICLing 2002, LNCS 2276. Springer, Heidelberg, pp 136–145

    Google Scholar 

  • Bonacich P (1972) Factoring and weighting approaches to status scores and clique identification. J Math Sociol 2(1):113–120

    Article  Google Scholar 

  • Bookstein A, Klein ST, Raita T (1998) Clumping properties of content-bearing words. JASIS 49(2):102–114

    Google Scholar 

  • Chen C (2006) CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. J AM SOC INF SCI TEC 57(3):359–377. doi:10.1002/asi.20317

  • Chen C (2012) Predictive effects of structural variation on citation counts. J Am Soc Inform Sci Technol 63(3):431–449

    Article  Google Scholar 

  • Chen C (2014) The fitness of information: quantitative assessments of critical evidence. Wiley, Hoboken

    Google Scholar 

  • Chen C (2017) Science mapping: a systematic review of the literature. J Data Inf Sci 2(2):1–40

    MathSciNet  Google Scholar 

  • Chen C, Leydesdorff L (2014) Patterns of connections and movements in dual-map overlays: A new method of publication portfolio analysis. J Assoc Inf Sci Technol 65(2): 334–351

    Google Scholar 

  • Deerwester S, Dumais T, Landauer T, Furnas G, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6): 391–407

    Google Scholar 

  • Ding Y, Rousseau R, Wolfram D (eds) (2014) Measuring scholarly impact: methods and practice. Springer, Heidelberg. doi:10.1007/978-3-319-10377-8

  • Dunning T (1993) Accurate methods for the statistics of surprise and coincidence. Comput Linguist 19(1):61–74

    Google Scholar 

  • Egghe L (2006) Theory and practise of the g-index. Scientometrics 69(1):131–152. doi:10.1007/s11192-006-0144-7

    Article  Google Scholar 

  • Freeman LC (1977) A set of measuring centrality based on betweenness. Sociometry 40:35–41

    Article  Google Scholar 

  • Fries CC (1952) The structure of English. Harcourt Brace, New York

    Google Scholar 

  • Hicks D, Wouters P, Waltman L, Rijcke Sd, Rafols I (2015) Bibliometrics: The Leiden Manifesto for research metrics. Nature 520(7548):429–431. doi:10.1038/520429a

    Article  Google Scholar 

  • Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 102(46):16569–16572. doi:10.1073/pnas.0507655102

    Article  MATH  Google Scholar 

  • Hirst G, St-Onge D (1998) Lexical chains as representations of context for the detection and correction of malapropisms. In: Fellbaum C (ed) WordNet: an electronic lexical database. The MIT Press, Cambridge, MA

    Google Scholar 

  • Jiang JJ, Conrath DW (1997) Semantic similarity based on corpus statistics and lexical taxonomy. In: Proceedings of international conference research on computational linguistics (ROCLING X), Taiwan

    Google Scholar 

  • Kleinberg J (2002) Bursty and hierarchical structure in streams. In: Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining, pp 91–101

    Google Scholar 

  • Leacock C, Chodorow M (1998) Combining local context and WordNet similarity for word sense identification. In: Fellbaum C (ed) WordNet: an electronic lexical database, Chapter 11. The MIT Press, Cambridge, MA

    Google Scholar 

  • Leontief WW (1941) The structure of American economy, 1919–1929. Harvard University Press

    Google Scholar 

  • Lesk M (1986) Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In: Proceedings of the 5th annual international conference on systems documentation (SIGDOC’86). Toronto, Ontario, Canada, pp 24–26

    Google Scholar 

  • Leydesdorff L (2012) Alternatives to the journal impact factor: I3 and the top-10% (or otp-25%) of the most-highly cited papers. Scientometrics 92:355–365

    Article  Google Scholar 

  • Leydesdorff L, Bornmann L, Mutz R, Opthof T (2011) Turning the tables on citation analysis one more time: principles for comparing sets of documents. J Am Soc Inform Sci Technol 62(7):1370–1381. doi:10.1002/asi.21534

    Article  Google Scholar 

  • Lin D (1998) An information-theoretic definition of similarity. In: ICML ’98 Proceedings of the fifteenth international conference on machine learning, pp 296–304, 24–27 July 1998

    Google Scholar 

  • Miller GA, Charles WG (1991) Contextual correlates of semantic similarity. Lang Cogn Process 6(1):1–28

    Article  MathSciNet  Google Scholar 

  • Onodera N, Yoshikane F (2015) Factors affecting citation rates of research articles. J Assoc Inf Sci Technol 66(4):739–764

    Article  Google Scholar 

  • Piffer D (2012) Can creativity be measured? An attempt to clarify the notion of creativity and general directions for future research. Thinking Skills Creativity 7(3):258–264. doi:10.1016/j.tsc.2012.04.009

    Article  Google Scholar 

  • Rada R, Bicknell E (1989) Ranking documents with a thesaurus. JASIS 40(5): 304–310

    Google Scholar 

  • Radicchi F, Fortunato S, Castellano C (2008) Universality of citation distributions: toward an objective measure of scientific impact. Proc Natl Acad Sci 105(45):17268–17272

    Article  Google Scholar 

  • Resnik P (1995) Using information content to evaluate semantic similarity in a taxonomy

    Google Scholar 

  • Seeley JR (1949) The net of reciprocal influence: a problem in treating sociometric data. Can J Psychol 3:234–240

    Article  Google Scholar 

  • Seglen PO (1992) The skewness of science. J Am Soc Inf Sci 43(9):628–638

    Article  Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:3379–3423

    Article  MathSciNet  Google Scholar 

  • Shwed U, Bearman PS (2010) The temporal structure of scientific consensus formation. American Sociological Review 75(6):817–840

    Google Scholar 

  • Tversky A (1977) Features of similarity. Psychol Rev 84(4):327–352

    Article  Google Scholar 

  • Wang D, Song C, Barabási A-L (2013) Quantifying long-term scientific impact. Science 342(6154):127–132

    Article  Google Scholar 

  • Wu Z, Palmer M (1994) Verbs semantics and lexical selection. In: Proceedings of the 32nd annual meeting on association for computational linguistics. Las Cruces, New Mexciao, pp 133–138, 27–30 June 1994

    Google Scholar 

  • Zafarani R, Abbasi MA, Liu H (2014) Chapter 3, Network measures. In: Social media mining: an introduction. Cambridge University Press. http://dmml.asu.edu/smm/chapters/SMM-ch3.pdf

  • Zhu X, Turney P, Lemire D, Vellino A (2015) Measuring academic influence: not all citations are equal. J Assoc Inf Sci Technol 66(2):408–427

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computing and Informatics, Drexel University, Philadelphia, PA, USA

    Chaomei Chen

  2. Department of Library and Information Science, Yonsei University, Seoul, South Korea

    Min Song

Authors
  1. Chaomei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, C., Song, M. (2017). Measuring Scholarly Impact. In: Representing Scientific Knowledge. Springer, Cham. https://doi.org/10.1007/978-3-319-62543-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62543-0_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-62541-6

  • Online ISBN: 978-3-319-62543-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation