Skip to main content
SpringerLink
Log in
SpringerLink
Log in

An Evaluation of Evaluators: Multivariate Statistical Analysis of Journal Evaluation Indicators

  • Conference paper
  • First Online:
Proceedings of the Seventh International Conference on Management Science and Engineering Management

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 241))

  • 2691 Accesses

Abstract

Journal evaluation has blossomed over the past four decades. Not only the bibliometric indicators provided by the Journal Citation Reports (JCR), but alternatives such as Google Scholar, Scopus and many other recently-introduced indicators are becoming popular. However, high correlations between journal evaluation indicators indicate that the development of new variants of the indicators has resulted in hardly any additional empirical contribution, so the application of existing indicators is a more promising direction for future research. This paper attempts to show how to evaluate and classify journals using current journal evaluation indicators by various multivariate statistical analysis methods. Data were collected from all journals in the Operations Research and Management Science category of JCR [10] and SCImago Journal and Country Rank (scimagojr.com). Analysis in other scholarly fields can be conducted using the same method but with different category data in JCR and SCImago Journal and Country Rank.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Garfield E (2006) The history and meaning of the journal impact factor. Journal of the American Medical Association 295:90–93

    Google Scholar 

  2. Garfield E, Merton RK (1979) Citation indexing, its theory and application in science, technology, and humanities. Wiley, New York

    Google Scholar 

  3. Pinski G, Narin F (1976) Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing & Management 12:297–312

    Google Scholar 

  4. Moussa S, Touzani M (2010) Ranking marketing journals using the Google Scholar-based hg-index. Journal of Informetrics 4:107–117

    Google Scholar 

  5. Neuhaus C, Daniel HD (2008) Data sources for performing citation analysis: An overview. Journal of Documentation 64:193–210

    Google Scholar 

  6. Braun T, Glanzel W, Schubert A (2006) A Hirsch-type index for journals. Scientometrics 69:169–173

    Google Scholar 

  7. Gonzalez-Pereira B, Guerrero-Bote VP,Moya-Anegón F (2010) A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of Informetrics 4:379–391

    Google Scholar 

  8. Braun T, Glanzel W, Schubert A (2006) A Hirsch-type index for journals. Scientometrics 69:169–173

    Google Scholar 

  9. Garfield E (2006) Citation indexes to science: A new dimension in documentation through association of ideas. International Journal of Epidemiology 35:1123–1127

    Google Scholar 

  10. Journal Citation Reports (2012), Journal Citation Reports Help: Immediacy Index, available at: http://admin-apps.isiknowledge.com/JCR/help.htm (accessed April 12, 2013)

  11. Lehmann S, Jackson AD, Lautrup BE (2008) A quantitative analysis of indicators of scientific performance. Scientometrics 76:369–390

    Google Scholar 

  12. Bordons M, Fernández MT, Gomez I (2002) Advantages and limitations in the use of impact factor measures for the assessment of research performance. Scientometrics 53:195–206

    Google Scholar 

  13. Bornmann L, Mutz R, Daniel HD (2008) Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology 59:830–837

    Google Scholar 

  14. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 1–38

    Google Scholar 

  15. Schafer JL, Olsen MK (1998) Multiple imputation for multivariate missing-data problems: A data analyst’s perspective. Multivariate Behavioral Research 33:545–571

    Google Scholar 

  16. Kaiser HF (1974) An index of factorial simplicity. Psychometrika 39:31–36

    Google Scholar 

  17. Bergstrom CT, West JD, Wiseman MA (2008) The EigenfactorTMmetrics. The Journal of Neuroscience 28: 11433–11434

    Google Scholar 

  18. Xu JP, Li ZM, Shen WJ et al (2013) Multi-attribute comprehensive evaluation of individual research output based on published research papers. Knowledge-Based Systems 43:135–142

    Google Scholar 

  19. Liu ZL, Zhang JL, Zou LF (2012) The Evaluation Index System for Enterprise IT Service Quality. In: The Sixth International Conference of Management Science and Engineering Management 219–225

    Google Scholar 

  20. Liu ZL, Zhang JL, Zou LF. (2012) A comprehensive assessment of commercial health & fitness clubs’ service quality. International Journal of Management Science and Engineering Management 7:263–267

    Google Scholar 

  21. Smith SD (2004) Is an article in a top journal a top article? Financial Management 133–149

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Decision Sciences Department, LeBow College of Business, Drexel University, Philadelphia, 19104, USA

    Neil Desnoyers & Zongmin Li

  2. Uncertainty Decision-Making Laboratory, Sichuan University, Chengdu, 610064, People’s Republic of China

    Zongmin Li

Authors
  1. Neil Desnoyers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zongmin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neil Desnoyers .

Editor information

Editors and Affiliations

  1. Business School, Sichuan University, Chengdu, China, People's Republic

    Jiuping Xu

  2. Drexel University, Philadelphia, USA

    John A. Fry

  3. LeBow College of Business Department of Decision Sciences, Drexel University, Philadelphia, Pennsylvania, USA

    Benjamin Lev

  4. Azerbaijan National Academy of Sciences, Institute of Cybernetics, Baku, Azerbaijan

    Asaf Hajiyev

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Desnoyers, N., Li, Z. (2014). An Evaluation of Evaluators: Multivariate Statistical Analysis of Journal Evaluation Indicators. In: Xu, J., Fry, J., Lev, B., Hajiyev, A. (eds) Proceedings of the Seventh International Conference on Management Science and Engineering Management. Lecture Notes in Electrical Engineering, vol 241. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40078-0_35

Download citation

Publish with us

Policies and ethics

Search

Navigation