, Volume 116, Issue 1, pp 147–160 | Cite as

Rainbow ranking: an adaptable, multidimensional ranking method for publication sets

  • Georgios StoupasEmail author
  • Antonis Sidiropoulos
  • Antonia Gogoglou
  • Dimitrios Katsaros
  • Yannis Manolopoulos


Various scientometric indices have been proposed in an attempt to express the quantitative and qualitative characteristics of scientific output. However, fully capturing the performance and impact of a scientific entity (author, journal, institution, conference, etc.) still remains an open research issue, as each proposed index focuses only on particular aspects of scientific performance. Therefore, scientific evaluation can be viewed as a multi-dimensional ranking problem, where dimensions represent the assorted scientometric indices. To address this problem, the skyline operator has been proposed in Sidiropoulos et al. (J Informetr 10(3):789–813, 2016) with multiple combinations of dimensions. In the present work, we introduce a new index derived from the utilization of the skyline operator, called Rainbow Ranking or RR-index that assigns a category score to each scientific entity instead of producing a strict ordering of the ranked entities. Our RR-index allows the combination of any known indices depending on the purposes of the evaluation and outputs a single number metric expressing multi-criteria relative ranking and can be applied to any scientific entity such as authors and journals. The proposed methodology was experimentally evaluated using a dataset of over 105,000 scientists from the Computer Science field.


Scientometrics Ranking h-Index Skyline 


  1. Alguliev, R. M., Aliguliyev, R. M., Fataliyev, T. K., & Hasanova, R. S. (2014). Weighted consensus index for assessment of the scientific performance of researchers. Collnet Journal of Scientometrics and Information Management, 8(2), 371–400.CrossRefGoogle Scholar
  2. Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2010). hg-index: A new index to characterize the scientific output of researchers based on h- and g-index. Scientometrics, 82, 391–400.CrossRefGoogle Scholar
  3. Anderson, T. R., Hankin, R. K., & Killworth, P. D. (2008). Beyond the Durfee square: Enhancing the h-index to score total publication output. Scientometrics, 76(3), 577–588.CrossRefGoogle Scholar
  4. Bollen, J., van de Sompel, H., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE, 4(6), e6022.CrossRefGoogle Scholar
  5. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.CrossRefGoogle Scholar
  6. Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. P. (2014). h-index research in scientometrics: A summary. Journal of Informetrics, 8(3), 749–750.CrossRefGoogle Scholar
  7. Bornmann, L., Mutz, R., Ηug, S. E., & Daniel, H. (2011). A multilevel meta-analysis of studies reporting correlations between the h-index and 37 different h-index variants. Journal of Informetrics, 5(3), 346–359.CrossRefGoogle Scholar
  8. Börzsönyi, S., Kossmann, D., & Stocker, K. (2001). The skyline operator. In Proceedings of the IEEE international conference on data engineering (ICDE) (pp. 421–430).Google Scholar
  9. Brzezinski, M. (2015). Power laws in citation distributions: Evidence from scopus. Scientometrics, 103(1), 213–228.CrossRefGoogle Scholar
  10. Chomicki, J., Godfrey, P., Gryz, J., & Liang, D. (2003). Skyline with presorting. In Proceedings of the IEEE international conference on data engineering (ICDE) (pp. 717–719).Google Scholar
  11. Egghe, L. (2006). Theory and practice of g-index. Scientometrics, 69(1), 131–152.CrossRefGoogle Scholar
  12. Franceschini, F. (2014). The citer-success-index: A citer-based indicator to select a subset of elite papers. Scientometrics, 101(2), 963–983.CrossRefGoogle Scholar
  13. Glanzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.CrossRefGoogle Scholar
  14. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefzbMATHGoogle Scholar
  15. Hutchins, B., & Yuan, X. A. (2016). Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biology, 14(9), e1002541.CrossRefGoogle Scholar
  16. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The r- and ar-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.CrossRefGoogle Scholar
  17. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.Google Scholar
  18. Ruane, F., & Tol, R. (2008). Rational (successive) h-indices: An application to economics in the Republic of Ireland. Scientometrics, 75(2), 395–405.CrossRefGoogle Scholar
  19. Schreiber, M., Malesios, C., & Psarakis, S. (2012). Exploratory factor analysis for the hirsch index, 17 h-typevariants, and some traditional bibliometric indicators. Journal of Informetrics, 6(3), 347–358.CrossRefGoogle Scholar
  20. Sidiropoulos, A., Gogoglou, A., Katsaros, D., & Manolopoulos, Y. (2016). Gazing at the skyline for star scientists. Journal of Informetrics, 10(3), 789–813.CrossRefGoogle Scholar
  21. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007). Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253–280.CrossRefGoogle Scholar
  22. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2015). Ranking and identifying influential scientists versus mass producers by the Perfectionism Index. Scientometrics, 103(1), 1–31.CrossRefGoogle Scholar
  23. Stoupas, G. (2017). Citation graph analysis and application of prototype evaluation and ranking methods. Master Thesis. Alexander Technological Institute of Thessaloniki.Google Scholar
  24. Tamar, V. L., & Tim, S. E. (2015). Ranking journals using altmetrics. CoRR. abs/1507.00451.Google Scholar
  25. Tsai, C. F. (2014). Citation impact analysis of top ranked computer science journals and their rankings. Journal of Informetrics, 8(2), 318–328.CrossRefGoogle Scholar
  26. Van Raan, A. F. J. (2004). Sleeping beauties in science. Scientometrics, 59(3), 467–472.CrossRefGoogle Scholar
  27. Voorneveld, M. (2003). Characterization of Pareto dominance. Operations Research Letters, 31(1), 7–11.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101(1), 125–158.CrossRefGoogle Scholar
  29. Wohlin, C. (2009). A new index for the citation curve of researchers. Scientometrics, 81(2), 521–533.CrossRefGoogle Scholar
  30. Wolcott, H. N., Fouch, M. J., Hsu, E., Bernaciak, C., Corrigan, J., & Williams, D. (2015). Modeling time-dependent and -indpendent indicators to facilitate identification of breakthrough research papers. In Proceedings of the international society of scientometrics & informetrics conference (ISSI) (pp. 403–408).Google Scholar
  31. Zhang, C. (2009). The e-index, complementingthe h-index for excess citations. PLoS ONE, 4(5), e5429.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Department of InformaticsAristotle UniversityThessalonikiGreece
  2. 2.Department of Information TechnologyAlexander Technological Educational Institute of ThessalonikiThessalonikiGreece
  3. 3.Department of Electrical and Computer EngineeringUniversity of ThessalyVolosGreece

Personalised recommendations