, Volume 79, Issue 1, pp 147–156 | Cite as

Open access scientometrics and the UK Research Assessment Exercise



Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Assessment Exercise (RAE) — together with the growing movement toward making the full-texts of research articles freely available on the web — offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, etc. can all be tested and validated jointly, discipline by discipline, against their RAE panel rankings in the forthcoming parallel panel-based and metric RAE in 2008. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.


Citation Count Citation Impact Journal Impact Factor External Criterion Aptitude Test 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Brody, T. (2003), Citebase search: Autonomous citation database for e-print archives. ECS Technical Report.
  2. Brody, T., Harnad, S., Carr, L. (2006), Earlier web usage statistics as predictors of later citation impact. Journal of the American Association for Information Science and Technology (JASIST), 57(8): 1060–1072. Scholar
  3. Garfield, E., (1955) Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122: 108–111, Scholar
  4. Giles, C. L., Bollacker, K., Lawrence, S. (1998), CiteSeer: An automatic citation indexing system. Digital Libraries 98: Third ACM Conf. on Digital Libraries, ACM Press, New York, 1998, pp. 89–98. Scholar
  5. Guédon, J.-C. (2002), In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing. Washington, DC: The Association of Research Libraries. Scholar
  6. Hajjem, C., Harnad, S., Gingras, Y. (2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4) 39-47. Scholar
  7. Harnad, S. (2006), Online, continuous, metrics-based research assessment. Technical Report, ECS, University of Southampton.
  8. Harnad, S., Brody, T. (2004), Comparing the impact of Open Access (OA) vs. non-OA articles in the same journals, D-Lib Magazine, 10(6) June.
  9. Harnad, S., Carr, L., Brody, T., Oppenheim, C. (2003), Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne, 35 (April 2003).
  10. Hirsch, J. E. (2005), An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46): 16569–16572, Scholar
  11. Kleinberg, J. M. (1999), Hubs, authorities, and communities. ACM Computing Surveys, 31(4),
  12. Kline, P. (2000), The New Psychometrics: Science, Psychology and Measurement. Routledge.Google Scholar
  13. Lawrence, S. (2001), Online or invisible? Nature, 411(6837): 521, Scholar
  14. Landauer, T. K., Foltz, P. W., Laham, D. (1998), Introduction to latent semantic analysis. Discourse Processes, 25: 259–284.CrossRefGoogle Scholar
  15. Moed, H. F. (2005), Citation Analysis in Research Evaluation. NY Springer.Google Scholar
  16. Oppenheim, Charles (1996), Do citations count? Citation indexing and the research assessment exercise, Serials, 9: 155–61. Scholar
  17. Page, L., Brin, S., Motwani, R., Winograd, T. (1999), The PageRank Citation Ranking: Bringing Order to the Web.
  18. Shadbolt, N., Brody, T., Carr, L., Harnad, S. (2006), The open research web: a preview of the optimal and the inevitable. In: Jacobs, N. (Ed.), Open Access: Key Strategic, Technical and Economic Aspects, chapter 21. Chandos.

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  1. 1.Chaire de recherche du Canada, Institut des sciences cognitivesUniversité du Québec à MontréalMontréalCanada

Personalised recommendations