Assessing the Societal Impact of Scientific Research

  • Pali U. K. De SilvaEmail author
  • Candace K. Vance
Part of the Fascinating Life Sciences book series (FLS)


Peer reviewing and citation metrics are traditionally used in assessing scientific research, with the emphasis on measuring the scientific quality and impact. Consensus among stakeholders of scientific research was building toward the need for assessing the societal benefits of scientific research, in addition to scientific quality. With the recognition of these needs by different governments and research funding agencies, formulating policies and guidelines to incorporate societal impact assessment in grant funding requirements and national science policies began. The most critical and challenging aspect of measuring the societal benefits is identifying assessment tools that efficiently and effectively measure these impacts. With the computer and communication technological advances and fast evolving social networking environment, use of the alternative metrics or altmetrics in assessing the societal impact of research gained attention. In this chapter, these developments are discussed by reviewing literature on the topic. The potential of altmetrics in assessing societal benefits of scientific research, and their strengths and limitation as assessment metrics, the empirical evidence of the correlation between altmetrics and traditional citation metrics, and efforts that are needed and in progress to improve the quality and standards of altmetrics are examined.


Societal impact Alternative metrics Altmetrics Citation metrics 


  1. Bloch, H. (2010). Research evaluation down under: An outsider’s view from the inside of the Australian approach. American Journal of Economics and Sociology, 69(5), 1530–1552.CrossRefGoogle Scholar
  2. Bornmann, L. (2012). Measuring the societal impact of research. EMBO Reports, 13(8), 673–676. doi: 10.1038/embor.2012.99 CrossRefPubMedPubMedCentralGoogle Scholar
  3. Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123–1144. doi: 10.1007/s11192-015-1565-y CrossRefGoogle Scholar
  4. Bornmann, L., & Marx, W. (2014). How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons. Scientometrics, 98(1), 211–219.CrossRefGoogle Scholar
  5. Bozeman, B., & Sarewitz, D. (2011). Public value mapping and science policy evaluation. Minerva, 49(1), 1–23.CrossRefGoogle Scholar
  6. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019. doi: 10.1002/asi.23309 CrossRefGoogle Scholar
  7. Darling, E. S., Shiffman, D., Côté, I. M., & Drew, J. A. (2013). The role of Twitter in the life cycle of a scientific publication. arXiv preprint arXiv:1305.0435
  8. De Winter, J. C. F. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102(2), 1773–1779.CrossRefGoogle Scholar
  9. Donovan, C. (2005). The governance of social science and everyday epistemology. Public Administration, 83(3), 597–615.CrossRefGoogle Scholar
  10. Donovan, C. (2007). The qualitative future of research evaluation. Science and Public Policy, 34(8), 585–597.CrossRefGoogle Scholar
  11. Donovan, C. (2008). The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. New Directions for Evaluation, 2008(118), 47–60.CrossRefGoogle Scholar
  12. Fenner, M. (2014). Altmetrics and other novel measures for scientific impact. In S. Bartling & S. Friesike (Eds.), Opening science: The evolving guide on how the internet is changing research, collaboration and scholarly publishing (pp. 179–189). Cham: Springer.CrossRefGoogle Scholar
  13. Greenhalgh, T., Raftery, J., Hanney, S., & Glover, M. (2016). Research impact: A narrative review. BMC Medicine, 14(1), 1.CrossRefGoogle Scholar
  14. Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics, 1–11. doi: 10.1007/s11192-016-1910-9
  15. Haustein, S., Costas, R., & Larivière, V. (2015a). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Haustein, S., Sugimoto, C. R., Larivière, V., Costas, R., Zahedi, Z., & Wouters, P. (2015b). The thematic orientation of publications mentioned on social media: Large-scale disciplinary comparison of social media metrics with citations. Aslib Journal of Information Management, 67(3), 260–288.CrossRefGoogle Scholar
  17. Holbrook, J. B., & Frodeman, R. (2011). Peer review and the ex ante assessment of societal impacts. Research Evaluation, 20(3), 239–246.CrossRefGoogle Scholar
  18. Jobmann, A., Hoffmann, C. P., Künne, S., Peters, I., Schmitz, J., & Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: Comparison of current tools. Bibliometrie-praxis und forschung, 3.Google Scholar
  19. Kousha, K., & Thelwall, M. (2007). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology, 58(7), 1055–1065.CrossRefGoogle Scholar
  20. Lewis, J. M., & Ross, S. (2011). Research funding systems in Australia, New Zealand and the UK: Policy settings and perceived effects. Policy & Politics, 39(3), 379–398.CrossRefGoogle Scholar
  21. Li, X., & Thelwall, M. (2012, September). F1000, Mendeley and traditional bibliometric indicators. Paper presented at the International conference on science and technology indicators.Google Scholar
  22. Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471. doi: 10.1007/s11192-011-0580-x CrossRefGoogle Scholar
  23. Lin, J., & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20.CrossRefGoogle Scholar
  24. Mohammadi, E., Thelwall, M., Haustein, S., & Lariviere, V. (2015). Who reads research articles? An altmetrics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology, 66(9), 1832–1846. doi: 10.1002/asi.23286 CrossRefGoogle Scholar
  25. O’Reilly, T. (2007). What is Web 2.0: Design patterns and business models for the next generation of software. Communications & Strategies, 1, 17.Google Scholar
  26. Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(7431), 159–159.Google Scholar
  27. Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7).Google Scholar
  28. Rousseau, R., & Ye, F. Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(26), 3288–3290.Google Scholar
  29. Shema, H., Bar-Ilan, J., & Thelwall, M. (2014). Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology, 65(5), 1018–1027.CrossRefGoogle Scholar
  30. Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: Article downloads, Twitter mentions, and citations. PloS One, 7(11). doi: 10.1371/journal.pone.0047523
  31. Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143. doi: 10.1007/s11192-013-1117-2 CrossRefGoogle Scholar
  32. Taraborelli, D. (2008, 20–23 May). Soft peer review: Social software and distributed scientific evaluation. Paper presented at the Proceedings of the Eighth International Conference on the Design of Cooperative Systems, Carry-Le-Rouet.Google Scholar
  33. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PloS One, 8(5), e64841.Google Scholar
  34. Torres, D., Cabezas, A., & Jimenez, E. (2013). Altmetrics: New indicators for scientific communication in web 2.0. Comunicar 41, 53–60. doi: 10.3916/c41-2013-05
  35. Van der Meulen, B., & Rip, A. (2000). Evaluation of societal quality of public sector research in the Netherlands. Research Evaluation, 9(1), 11–25.CrossRefGoogle Scholar
  36. Wouters, P., & Costas, R. (2012). Users, narcissism and control: Tracking the impact of scholarly publications in the 21st century. SURFfoundation Utrecht.Google Scholar
  37. Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513. doi: 10.1007/s11192-014-1264-0 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Murray State UniversityMurrayUSA

Personalised recommendations