Introduction

Assessing scientific publications by scientometric methods has two main aspects: impact and quantity. Scientometric assessments are performed through indicators. The indicators selected for evaluating individuals, teams, countries or journals can characterize impact or quantity or both (Vinkler 2013). The two sides of the assessment, i.e. impact and quantity cannot be separated completely. The total number of publications or the specific index of the publication productivity (publications by scientist) reflects not only the amount but also the impact of the published information. And, the total number of citations received or the specific index: citations per publication depend not only on impact but also on the amount of published information. There are some combined traditional impact and quantity indicators available in the literature (Vinkler 2010a, 2013).

For a long time, the mentioned indices were preferably calculated from data referring to whole publication sets. With the publication of the Hirsch index (Hirsch 2005) however; it has begun the preferred use of indicators referring to papers in the “elite” (or core) subsets within the total. This trend has resulted in a plethora of scientometric impact indices (Schreiber 2010; Schreiber et al. 2012; Todeschini and Baccini 2016).

The indices referring to the elite subsets are derived from a relatively small, most frequently cited part (“core” or “elite” subset) of total publications. The central idea behind the use of this method is the assumption that a relatively small part of all may represent the total publication impact. Even, the indices derived from the exclusive part of publications may characterize the relevant publication impact more appropriately than the mean indices referring to the corresponding total set.

The idea of applying core journal papers for assessment finds support also by the fact that the distribution of citations over papers is skewed (Seglen 1992). It is well-known that scientific progress is made primarily through information in publications acknowledged by relatively high number of citations (Plomp 1990; Aksnes 2003; Vinkler 2010a, 2017a). Consequently, publications with high influence may be revealed by determining relatively highly cited journal papers.

Elite subsets can be obtained by different statistics: h-statistics, g-statistics, percentage statistics, π-statistics, πv-statistics, etc. The core or elite part of publication sets are termed as h-core, g-core, π-core, or 1%, 10%, etc. of total, according to the method applied. Naturally, the publications in the set studied are ranked according to the decreasing citation frequency by each method.

The impact indices derived through the mentioned statistics, e.g.: h-index (Hirsch 2005) and h-related indices (Bornmann et al. 2011), g-index (Egghe 2006), π-index (Vinkler 2009), πv-index (Vinkler 2010b), I3-index, indices of percentiles and percentile classes, PR(2, 10) or PR(2, 50) or PR(6) (Leydesdorff 2012; Bornmann et al. 2013; Bornmann and Marx 2014) etc. strongly depend on the bibliometric features of the corresponding field. There are some methods, however applying special outside standards for obtaining elite subsets. It may be supposed, these indices are (at least partly) independent of the bibliometric features of the field. One of these methods for selecting core papers applies the publications with more citations than the mean (or multiplied or squared mean) number of citations, not in the publication set analysed but, in the corresponding field (Table 1).

Table 1 Name and calculation method for selecting elite subsets
Full size table

The publications in elite subsets in a scientific field may reveal hot topics or relevant, core information of the field depending on the time periods applied. Science is developing permanently. Therefore, the dynamic study of the publications in the elite subsets is highly relevant (Egghe 2007). Through comparing the topics of frequently cited papers in the elite sets in consecutive time periods, the development of research directions may be followed.

The size of the core sets depends on the calculation method selected, and it may bring about discrepancies during evaluations. The h-statistics, e.g. generally used may produce controversial results.

J. Hirsch, who introduced the h-index in 2005, is a university professor of physics. He published altogether 271 journal papers up to 03. 06. 2019. Until that date their papers received a total of 19,304 citations. The mean citation frequency of his papers is equal to 71.23. Taking into account all journal papers of Hirsch, independent of their topic, his h-index is: 58 (see WoS). The mentioned author initiated a revolution in the development of scientometric impact indices by suggesting the h-index.

His first scientometric paper (2005) obtained 3633 citations up to the mentioned date. Accordingly, we would think, he is one of the most influental scientists in the field. However, I have surveyed the publication list of the mentioned author in WoS. Among his 271 papers only 6 articles may be classified as scientometric publications. The citation rate of these papers is as follows: 1/(published in 2005): 3633, 8/(published in 2007): 434, 31/(published in 2010): 123, 145/(published in 2014): 25, and 249/(published in 2019): 0. (The cursive numbers are the rank numbers of the papers by citations taking into account all papers (271) of the mentioned author.) Accordingly, the h-index, i.e. the scientometric impact of J. Hirsch would be equal to 4. It is because the value of h-index cannot exceed the number of publications in the set, and it does not regard the number of citations obtained (Vinkler 2007).

However, calculating the π-index of the scientometric papers of Hirsch, a relatively high value is obtained. The number of Hirsch’s scientometric papers = 6, accordingly the number of π-core papers: √2 = 2.46, see Table 1. In calculating the π-index, we have to sum up the number of citations to π-core papers (here 2.46 rounded as 2), and the sum should be divided with 100. Accordingly: π-index = 0.01 (3633 + 434) = 40.67. This value is rather high compared to that for well-known scientometricians (Vinkler 2017b).

The given example makes it clear that the π-index prefers scientists with relatively high number of citations to the journal papers in the π-core against the total number of citations. This is because the distribution of citations is skewed, in general (Seglen 1992), and the π-method calculates with the citations to √P papers, where the total number of papers equals P, see Table 1. Therefore, it may happen that a PhD student shows a relatively high π-index although he or she published, say only four papers. Let us assume, e.g. that two of the papers were published together with an internationally well-known professor. Let it be the number of citations to the individual papers as follows: 1: 400, 2: 200, 3: 10, and 4: 0. Accordingly, the size of the π-core: P(π) = √4 = 2.00. This way, the young fellow may offer a π-index = 0.01(400 + 200) = 6.0 which counts as a relatively high π-index for junior scientists (Vinkler 2009, 2017b). On the other side, the π-index may not be favourable for scientists with a relatively short publication list. This is because the size of the π-set strongly depends on P.

The above examples may demonstrate, the automatic evaluation of publications of persons or teams (or similarly: journals) through the mentioned and not mentioned elite subset indices without further scientometric considerations, may cause errors. Therefore, it is advisable to apply several indices. And, provided they converge, we may accept the outcomes. The mentioned discrepancies may occur at evaluating individuals, primarily. To assess greater sets of papers, the impact of the errors is weeker.

The number of assessment processes using elite set indicators is growing permanently (Todeschini and Baccini 2016) although, the methodology and the validation of the indices is far from complete. Therefore the study of features of elite subsets calculated by different methods seems to be relevant.

In the present paper I try to compare some characteristics of different elite publication subsets in the field of scientometrics. The first step of the study was the search for the core publication channels in scientometrics.

The present study aims primarily at

  • revealing the leading journals publishing majority of journal papers in scientometrics and related fields, through calculating the frequency of journals in the h-core and π-core of the publications of Price medallists,

  • determining the share of the individual journals in the elit subsets of a combined set of papers in the leading scientometric journals,

  • determining the share of the individual journals in the elit subsets of the combined set of papers in the field obtained by key words: scientometr*, bibliometr*, informetr*, and webometr*,

  • comparing the share of papers in the individual elite subsets published by Price medallists.

Data and methods

For obtaining journals most frequently used for publication scientometric, bibliometric, informetric, and webometric papers, 19 still active Price medallists were choosen, as peers: T Braun, B Cronin, L Egghe, P Ingwersen, W Glänzel, L Leydesdorff, K W McCain, Ben R Martin, H F Moed, F Narin, O Persson, R Rousseau, A Schubert, H Small, M Thelwall, AFJ vanRaan, P Vinkler, H White, and M Zitt. The persons listed may be regarded as scientists with outstanding contribution to the field of quantitative studies of science (For the history and selection of Price medallists, see: Erfanmanesh and Moghiseh 2019 and Zhou et al. 2019).

In the present paper Table 2 refers to the selected Price medallist listed above. These data were applied for obtaining most important information channels of the field. The data in Tables 5 and 6 refer to all scientists who were decorated with Price medall up to 2018.

Table 2 Number and share (in per cent) of all papers in the π-core and h-core within the total publications of Price medallists by journal (The starting year of the journal is given in parentheses)
Full size table

The field of activity in scientometrics, bibliometrics, informetrics, and webometrics (termed shortly as SBIW or scientometrics) may be defined as follows: “Scientometrics is a field of science dealing with the quantitative aspects of people, matters and phenomena in science, and their relationships, but which do not primarily belong within the scope of a particular scientific discipline” (Vinkler 2001; Vinkler 2010a, b). It is widely accepted that scientometrics is preferably concerned with the quantitative aspects of the generation, propagation, and utilization of scientific information (Braun et al. 1987).

In selecting journals on scientometrics, the Price medallists were regarded, as reviewers or peers. For obtaining the most relevant, recent publication channels, only Price medallists recently active were selected.

It is well known, the number of scientists and publications increases in the field dynamically (see e.g. Hood and Wilson 2001). Whereas the first journal (Scientometrics) devoted exclusively to scientometrics was launched by T. Braun already in 1978, one of the most important journals in the field (Journal of Informetrics) started only in 2007. Another important periodical, Annual Review of Information Science and Technology published its first volume in 2002 and Library Trends in 2005. The dynamic increase in the number of journal papers on scientometric topics can be demonstrated by the following data.

The number of papers published on the topics: scientometr* OR bibliometr* OR informetr* OR webometr* in time periods 1978–1987, 1988–1997, 1998–2007, and 2008–2017 were collected from Clarivate Analytics Web of Science All Databases, on 31. 05. 2019. The total number of papers is as follows: 260, 873, 3268, 12,858, respectively. Accordingly the ratios: 1.00, 3.36, 12.57, 49.45. The data clearly show the increase in journal publications in the field. This finding may indicate the selection of the still active scientometricians for obtaining core journals of the field.

The publications and citations of the Price medallists were obtained from Web of Science All Databases 1975–2017 on 15. 02. 2018. Table 1 summarizes the methods applied for calculating elite subsets.

The publications and citations in the journals (Tables 2, 3, 4, 5) were obtained from Web of Science Core Collection 1975–2017 on 01. 03. 2018.

Table 3 Total number of publications and number of h-core papers in the selected journals, share of SBIW papers in the h-core, and approximate number of SBIW papers in the whole journal
Full size table
Table 4 Number of papers and citation frequency limits of the elite (core) subsets in the combined set of scientometric, bibliometric, informetric and webometric papers (field) in the selected journals in Table 3
Full size table
Table 5 Number and distribution (in percentage share) of publications in the elite subsets in the combined set of publications from the selected journals
Full size table

To obtain the number of scientometric papers in the journals, the title and, if it was necessary, also the abstract of the papers in the elite sets of the studied journals was surveyed whether their content would correspond to the definition of the scientometric studies given above. Accordingly, the papers were selected into two different sets; SBIW: [scientometric, bibliometric, informetric, and webometric publications] and LIMP: [other publications]. The term “other” means non-scientometric topics, e.g. library science, research topics on information retrieval, data banks, research policy, social sciences, research management, economics, etc.). The publications in Scientometrics, Journal of Informetrics and Research Evaluation were not surveyed individually. All publications in those journals were attributed to the [scientometric, bibliometric, informetric, and webometric] field.

The number of and citations to the papers in the SBIW field (Tables 6, 7) were collected from Clarivate Analytics Web of Science All Databases 1975–2017 on 31. 07. 2018 applying the term: TS = (scientometr* OR bibliometr* OR informetr* OR webometr*).

Table 6 Number of papers in the elite subsets in scientometric (i.e. scientometric, bibliometric, informetric, and webometric) and non-scientometric journals in the publication set obtained by key words (scientometr* OR bibliometr* OR informetr* OR webometr*) (field) from WoS all databases
Full size table
Table 7 Number and (per cent, %) of papers in the elite subsets of the field (see Table 6) by the selected journals
Full size table

The statistical analyses were performed by STATISTICA data analysis software system version 13, TIBCO Software Inc. Palo Alto, USA.

Results and discussion

Elite sets of publications of Price medallists and core journals in scientometrics, bibliometrics, informetrics, and webometrics

The h-index is believed to represent the measure of scientific impact of a set of papers (attributed to individuals, teams, countries, and journals, etc.) both qualitatively and quantitatively (Iglesias and Pecharroman 2007). The index is equal to the highest rank number of the paper with citation frequency equal to (or higher than) the citation rank of the paper in the set analysed. Consequently, the h-index can be regarded as the size of the elite subset within the whole set studied. According to the π-method, the number of papers with highest impact in the studied set may be approximated by the square root of total papers (Table 1).

The publications and citations of 19 Price medallists in 1975–2017 were collected and the size of the corresponding h-set and π-set of each scientist was calculated according to the mentioned methods (Table 1). The papers of the individual scientists in these elite sets were classified according to the publishing journals (Table 2). The journals publishing only two or three elite papers are given below the table.

It is supposed that prominent scientists of a scientific field publish their most relevant results in the most relevant journals of the field, preferably. And, the journals having published the papers in the elite subsets of Price medallists, they may represent the relevant information base of the corresponding field appropriately. Therefore, the journals publishing majority of highly cited papers of eminent scientists may be regarded as the core journals of the field.

The total number of papers in the h-core of the Price medallists is 530 of which majority (494) is published in the scientometric journals listed in Table 2. The total number of π-core papers is significantly less: 194. The listed 10 journals contain 95.36 per cent and 93.21 per cent of all papers in π-core and h-core, respectively. From the π-core and h-core papers only 9 (4.64%) and 36 publications (6.79%), respectively appeared in journals not listed in Table 2. From the data in Table 2 it follows that in publishing Price medallists’ papers with potentially high impact, Scientometrics is preferred (39.69 and 45.47 per cent of all papers in the P(π) and P(h) subset, respectively). JASIS(T) figures as second with 29.38 and 26.42 per cent, respectively, whereas Journal of Informetrics with 4.64 and 5.09 per cent, respectively. It should be mentioned that the latter journal started only in 2007.

From the above data it may be concluded, the journals containing most relevant information in the field of scientometrics, bibliometrics, informetrics, and webometrics in the studied period, are the following: Scientometrics, JASIS(T), Information Processing Management, and Journal of Informetrics, at least according to the publication strategy of Price medallists. The mentioned journals contain 81.13 and 79.38 per cent of total papers in the corresponding h-core and π-core, respectively.

The list of the core journals suggested in the present study is similar to that given as the preferred set of journals on “bibliometrics, scientometrics, and informetrics” based on DIALOG’s rankings by Hood and Wilson (2001). The mentioned authors found in the field 4697 papers in 1960–2000. Among the most frequently used 10 journals suggested here (Table 2), in the list of the mentioned authors Scientometrics ranks first with 1197 records, JASIS as second with 319, Information Processing & Management as fourth with 128, Journal of Information Science as fifth with 127, and Journal of Documentation as sixth with 109 records. Two further journals are mentioned in both studies: Library and Information Science Research as 10th with 59 records and Library Trends as 16th with 42 records). The difference between the two lists may be caused by the different journal selection processes of the studies. The present study takes into account only SBIW papers defined as in the introduction, whereas Hood and Wilson (2001) extended their study towards library science and information science with topics far from scientometrics. Accordingly, their list covers also e.g. Nauchno-Tekhnicheskaya Informatsiya Series 1&2 as 3rd with 285 records, and Revista Espanola de Documentacion Cientifica as 7th with 95 records.

Moaghali et al. (2011) studied the literature of “scientometrics” in WoS in 1980–2009. They found 691 documents in this time period. The mentioned authors listed also the top ten most productive journals of the field. From those journals six are common with the present study: Scientometrics (320 records), Information Processing & Management (20 records), Journal of Information Science (20 records), JASIS (14 records), Journal of Documentation (9 records), and Reserch Evaluation (8 records).

More recently Martín-Martín et al. (2016) investigated authors, documents and journals in the field of [bibliometrics, scientometrics, informetrics, webometric, and altmetrics] applying Google Scholar Citations in 1969–2015. The top eight most influential journals in their paper (Table 4) are the following (the number of articles found is given in brackets): Scientometrics (284), JASIST (137), Research Policy (57), Journal of Informetrics (36), Journal of Documentation (25), Information Processing & Management (24), Journal of Information Science (20), and Research Evaluation (18). One more journal (Library Trends) from Table 2 of my study figures with 7 records in the list of the mentioned authors. In summary, out of the 10 journals in Table 2 of the present study, 9 journals are involved in the corresponding list of Martín-Martín et al. (2016). Only a single journal (Ann. Rev. of Inform T.) fails.

It should be noted that the set of journals classified under the name Library & Information Science by WoS does not contain Research Policy. Out of the Price medallists studied only a single person (Ben R. Martin) published in Research Policy papers belonging to the elite sets. Nevertheless, it is known that this journal publishes many highly cited papers dealing with patent statistics and cooperation between industry and university, and relations between GDP and scientific publications of countries, etc. Therefore a substantial part of the publications in Research Policy should also be taken into account in studying publications in the SBIW field.

Defining the part of SBIW papers in the individual journals

The number of publications on SBIW topics in the journals (Table 2) containing most frequently cited papers of Price medallists were determined. The set of journals was completed with a further journal (Libri). One of the papers in this journal e.g. was found in the combined elite set, P(h) of papers in the selected periodicals.

It is well-known; some of the listed journals publish papers not only on scientometrics but also on other topics. Therefore the h-core publications in the selected journals (Table 3) were classified wheather they belong to SBIW field [scientometric, bibliometric, informetric, and webometric publications] or LIMP field [library science, research topics on information retrieval, data banks, research policy, social sciences, research management, economics, etc. publications]. The immediate aim of this selection was to obtain the total number of papers in the SBIW field because calculating the size of the πv-set and π-set of the field requires the knowledge of the total number of publications. Three of the journals (Scientometrics, Journal of Informetrics, and Research Evaluation) however were accepted as publishing exclusively SBIW papers. The other journals were found to publish both SBIW and LIMP papers. The ratio of SBIW papers vs. LIMP papers in a whole journal was approximated by the ratio of the two types of papers in the h-core of the journal. E.g. JASIS(T) published 31.78 per cent and Journal of Information Science 13.79 per cent SBIW papers within the h-core in 1975–2017. Accordingly, only parts: 31.78% (1632) and 13.79% (291) of the total papers in that journals, respectively were attributed to the SBIW field in the mentioned period. The total number of SBIW papers in the combined set of the selected journals in 1976–2017 was calculated as 9994 (Table 3).

Characteristics of the elite subsets in the combined set consisting of SBIW papers in the selected journals

The total number of SBIW papers (9994) obtained from WoS, Core Collection in the selected 11 journals was regarded as a unified set. All the publications were ranked according to the decreasing number of citations obtained. Table 4 shows the number of journal papers in the individual elite subsets, as calculated by the methods in Table 1. The data in Table 4 reveal that the number of papers in the elite subsets increases: E/1 = 30 < E/2 = 54 < E/3 = 100 < E/4 = 117 < E/5 = 137 < E/6 = 225. (It is to be mentioned that the data for the individual journals show a similar trend, e.g. Scientometrics: 9, 22, 43, 53, 59, and 90, respectively.) Through selecting 1 per cent of total (9994), as papers belonging to the elite set, we obtain 99.94 (rounded as 100) papers. This value corresponds to E/3 (100), therefore it was not applied.

Table 4 shows also the number of citations (“citation limit”) obtained by the paper with the highest rank in the given core set. With the increase of the number of papers in the subsets, Mean Citation Rate (MCR) of the papers decreases. In E/1 obtained by the P(πv)-method e.g., the number of papers is only 30, and MCR = 471.87. The greatest subset obtained by the HCP-method (E/6, i.e. papers with citations equal to or higher than 100) contains 225 papers. The MCR value in this subset is 197.88. Each difference between the MCR values is significant at p ≤ 0.02, except for that between E/1–E/2; E/3–E/4; E/3–E/5 and E/4–E/5.

A clear trend could be observed in the increase of the publishing years from 1997.21 (E/1) up to 2000.81 (E/6). The differences are however not significant, except for E/2–E/6 at p <  0.05. The highest mean publishing year is shown by the papers in the greatest elite set (E/6: 2000.81). This would mean: greater elite subsets contain more recent papers, in average. Figure 1 demonstrates the dependence of the number of citations (C) to the individual papers from the publishing year (y) for the greatest elite set (E/6). The Spearman rank correlation coefficient (r = − 0.15, p < 0.05) shows a significant but week negative relationship. The equation of the function: C = − 2.36 (year) + 4932.71.

Fig. 1
figure 1

Relationship between the number of citations obtained in 1975–2017 and year of publication (1975–2017) of journal papers in the core periodicals of scientometrics

Full size image

Some of the papers with very high number of citations are the following:

  • Etzkowitz, H. & Leydesdorff, L.: The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university-industry-government relations. Research Policy, 29 (2) 109–123 (2000)

    C = 1650

  • Katz, J. S. & Martin, B. R.: What is research collaboration? Research Policy, 26 (1) 1–18, (1997)

    C = 921

  • Egghe, L.: Theory and practice of the g-index. Scientometrics, 69 (1) 131–152 (2006)

    C = 707

  • Price, D. J. D.: General theory of bibliometric and other cumulative advantage processes. Journal of the American Society for Information Science, 27 (5–6) 292–306 (1976)

    C = 698

  • White, H. D. & McCain, K. W.: Visualising a discipline: An author co-citation analysis of information science, 1972–1995. Journal of the American Society for Information Science, 49 (4) 327–355 (1998)

    C = 521

  • Narin, F., Hamilton, K. S. & Olivastro, D.: The increasing linkage between US technology and public science. Research Policy, 26 (3) 317–330 (1997)

    C = 510

Percentage share of papers in the elite subsets of the combined set of SBIW papers by journals

The total number of papers in the elite subsets (E/1–E/6) within the combined set of 9994 papers in the core journals of the field is as follows: 30, 54, 100, 117, 137, and 225 (Table 4). The percentage share of papers in the E/1–E/6 elite subsets are shown in Fig. 2 and Table 5 by journals. From Table 5 it concludes that Scientometrics has the greatest share (in average 40.35%). With 24.39% JASIS(T) ranks as second. Research Policy is on the third place with 16.38%, in average. Scientometrics shows the greatest share in all categories except for E/1 where JASIS(T) exceeds (its number of papers published in E/1 equals 10 (33.33%) vs. Scientometrics: 9 (30.00%)). The lowest mean number of papers in the elite subsets is shown by Research Evaluation and Libri (1.08% and 0.07%, respectively). Figure 2 shows that the share of the individual journals in the elite subsets may increase or decrease or may remain standard with increasing size of the set. Scientometrics (Sciento.) and Journal of Informetrics (JoI) show increasing shares, whereas the share of JASIS and Research Policy decreases with the increasing size of subsets.

Fig. 2
figure 2

Percentage share of journals in the elite sets (E/1–E/6, see Table 1) of the combined set of papers in the core journals of scientometrics

Full size image

The data in Table 5 reveal the outstanding role of Price medallists in publishing highly cited papers in the studied journals. In average, the share of the most influential papers written by Price medallists is 45.50 per cent. The share in the smallest subsets, (in E/1 and E/2 containing most cited papers) is highest: 50.00, 55.56 per cent, respectively. The high share of the publications by Price medallists in the elite subsets may validate the calculation methods applied for obtaining elite sets and, at the same time the selection method of journals as leading information sources in scientometrics.

The number of publications written by Price medallists is also studied in the individual journals. In the elite subsets of Scientometrics e.g. P(πv), P(π) and P(h), the number and share of publications by Price medallists within the total in 1978–2017: 13 (48.15%), 36 (50.00%), and 43 (45.26%), respectively. The same data for JASIS(T) in 1975–2017: 8 (36.36%), 17 (42.50%), and 30 (36.14%), for Journal of Informetrics in 2007–2017: 7 (36.84%), 10 (35.41%), and 19 (35.85%), and for Research Policy in 1975–2017: 2 (12.50%), 3 (14.29%), and 10 (12.20%), respectively.

For comparing the share of Price medallists among the authors in the elite subsets and in total publications, total number of scientific articles, reviews, short communications, letters, corrections, and notes were collected from 1978 (Vol. 1) to 2006 (Vol. 69), and from 2007 (Vol. 70) to 2018 (Vol. 117) in Scientometrics. In the first period a total of 2045 and in the second 3283 publications, respectively were found. From that 408 and 279 papers, respectively were signed by a single or more Prize medallists. Accordingly the ratio of Prize medallists’ papers: 19.95 and 8.50 per cent, respectively. The ratio for the total time-period: 12.89 per cent. In Journal of Informetrics the total number of papers: 888 in 2007–2018. The number of articles published by Price medallists: 159. Accordingly the ratio of papers written by Price medallists: 17.91 per cent.

The mentioned data, compared to that referring to the elite sets, clearly show the higher rate of papers signed by Price medallists in the elite subsets.

Percentage share of papers in the elite subsets of the SBIW field by journals

The journal papers on SBIW topics were collected also according to the following keywords: TS = (scientometr* OR bibliometr* OR informetr* OR webometr*) in WoS All Databases for the period 1975–2017. The total number of papers was found as 16,992. Accordingly, the number of papers in the elite subsets (see Table 1) is the following: P(πv) = 32, P(π) = 130, P(h) = 148, and P(HCP ≥ 100) = 282 (Table 6). The number of papers (16,992) in the SBIW field obtained by the mentioned keywords is naturally higher than that calculated as the sum of SBIW papers in the selected 11 periodicals (9994) (see Table 3). The difference in the number of total papers compared to the combined set of papers in the selected journals, explains the difference in the size of the elite subsets.

The publications in SBIW field determined by keywords were distributed into two parts:

  • Publications on SBIW topics published in scientometric journals (i.e. journals devoted to partly or exclusively to scientometrics, bibliometrics, informetrics, and webometrics), and

  • Publications in journals devoted to primarily life sciences, natural sciences, psychology, etc. (named as non-scientometric journals).

The data in Table 6 show that the majority of highly cited scientometric results (i.e. publications in the elite subsets) are published in non-scientometric journals (P(πv): 75.00%, P(π): 55.38%, P(h): 57.43%, P(HCP ≥ 100): 55.67%,). The percentage rate of highly acknowledged papers on SBIW topics in scientometric journals is only 25.00% in the elite set, P(πv) but, it is higher than 45% in the other elite sets, neither. It should be remembered that journal publications in P(πv) represent the most highly cited share of papers.

The values of mean citation rate (MCR) in Table 6 reveal that the scientometric papers in the elite subsets of the field published in non-scientometric journals are more frequently cited than that in journals devoted to exclusively or partly to scientometrics. The difference between the MCR values of E/3: P(π), E(5): P(h), and E/6: P(HCP ≥ 100) is highly significant at p ≤ 0.02, whereas that between E/3, E/5, and E/6 with E/1: P(πv) is not (p > 0.10). The latter result may be attributed to the low number of papers in the P(πv) set in scientometric journals (8). The citation rate of papers in the elite subset, E/1: P(πv) in the scientometric journals of the field is 427.50. This value does not differ significantly from that calculated for the combined set of papers in the leading journals in Table 4 (471.87). The trend in the citation rate of papers in the elite sets (E/1, E/3, E/5, E/6) in Table 6 corresponds to the trend shown in Table 4 for E/1–E/6.

The reasons for the higher citedness of scientometric papers in non-scientometric journals was not analysed in details. I guess however, in most cases those scientometric papers in non-scientometric journals may be highly cited which find interest not only by professional scientometricians but a wider public consisting of science politicians, researchers in natural and social sciences, or librarians and information scientists.

Table 6 also shows the share of papers in the elite subsets published by Price medallists. It is in average: 22.55%. The share of papers written by Price medallists is significantly higher (in average: 45.50%) in the elite publication subsets taking into account the papers in the 11 selected scientometric periodicals (Table 5). This finding is in accordance with the data in Table 2 which indicate, the decisive share of highly cited papers of Price medallists are published in the selected scientometric journals: 185 out of 194 and 494 out of 530 in P(π) and P(h) set, respectively.

It is known that 1 per cent of total publications is frequently regarded as the elite in a publication set. In the SBIW set consisting of 16,992 papers: 1% = 170 (rounded). The value is between the limits: P(h) = 148 and P(HCP ≥ 100) = 282. The mean citation rate (MCR) of the 170 papers: 275.22 (SD: 245.77). The number of papers authored by Price medallists: 44 (25.8%). Out of the 170 papers 74 papers (43.53%) were found in scientometric and 96 (56.47%) in non-scientometric journals. The mentioned data are in accordance with that given in Table 6.

Comparison with the data from Scopus data bank

The publications in SBIW field in 1975–2017 were collected also from Scopus data bank using the same keywords as for WoS (scientometr* OR bibliometr* OR informetr* OR webometr*).

The total number of papers: 18,206 (08 12 2018.). Accordingly, the size of the elite subsets: P(πv): 33, P(π): 135, P(h): 159, P(C ≥ 100): 348. The size of the subsets is somewhat larger than that for WoS (32, 130, 148, and 282, respectively).

The individual items (papers) in the elite subsets obtained from Scopus and WoS were compared. Overlap Measures (OM) were calculated as follows:

$$OM = 100 \, \left( {Pc\;{\text{in}}\;{\text{WoS}} + Pc\;{\text{in}}\;{\text{Scopus}}} \right)/\left( {Pt\;{\text{in}}\;{\text{WoS}} + Pt\;{\text{in}}\;{\text{Scopus}}} \right)$$

where Pc: number of common elements, and Pt: number of total elements.

The OM values for the elite subsets are the following: OM/P(πv)/= 93.85%, OM/P(π)/= 87.17%, OM/P(h)/= 82.39%. The data would indicate that the overlap of the publications in the elite subsets decreases with increasing number of publications in the corresponding set. Because of the high similarity, no detailed studies of the elite subsets obtained from Scopus were made.

Conclusions

This paper presents:

  • a new method for determining leading journals in scientometrics.

The study covers:

  • calculating the size of elite subsets by different methods,

  • comparing the mean citation rate of journal papers in the elite subsets,

  • determining the frequency of journals in the elite subsets, and

  • calculating the rate of journal papers published by Price medallists.

For obtaining leading journals in scientometrics, the publishing journals of the most cited publications of Price medallists were collected. It is assumed namely that the journals containing most cited publications of eminent scientists in a field may represent the most influential information channels of the field.

Elite subsets may be calculated from publication sets of individuals, teams, countries, journals, or even scientific fields by different statistics. The aim for obtaining elite subsets in different time periods may be:

  • revealing most relevant information channels, hot topics in science fields, most productive and highly cited scientists, laboratories, and countries in fields or on topics,

  • calculating different scientometric indicators from data of the elite subsets of different thematic units or organizations for evaluation purposes, and

  • calculating standards (e.g. world average citation rate in different fields, mean number of highly cited papers of university chairs in a given country, mean number of citation rate of teams working in a field within a country, etc.) referring to elite subsets for evaluating the publication activity of individuals, teams or countries, comparatively.

The data in the present paper show, majority of highly influencing scientometric results are published in non-scientometric journals (Table 6). Nevertheless, the high share of publications written by Price medallists in the elite subsets of the combined set of special journals in scientometrics (P(πv): 50.00%, P(π): 41.00%, and P(h): 42.34%) (Table 5) may indicate the eminent role of the journals devoted to fully or partly to scientometrics. From these periodicals any of the “big four” (Scientometrics, Journal of Informetrics, JASIS(T), and Research Policy) publish significantly more papers in the elite subsets than any of the other journals (Table 7). The share in the most influential papers is smaller for Journal of Information Science, Journal of Documentation and Annual Review of Information Science and Technology than that for journals belonging to the “big four”. Some multiscience journals (e.g. Nature, Science, Proceedings of the National Academy of the United States) or journals in life sciences (e.g. British Medical Journal, Journal of the American Medical Association) may be also important sources of scientometric information.

The results in the present paper may contribute to the verification of applying elite set indicators in scientometric assessments. The Price medallists namely are regarded as eminent scientists in the field. They may be assumed as peers in the corresponding field. The relatively high share of papers published by Price medallists in the elite subsets of journals on scientometrics, bibliometrics, informetrics, and webometrics may prove the relative excellence of the papers in these subsets. Consequently, the two factors: the high share of papers of the peers in the elite subsets and the selection methods for obtaining the elite sets, they may mutually validate each other.

It may be assumed that similar methods with detecting the publication strategy (PS) (Vinkler 1997) of highly influential scientists may be applied for obtaining core publication sources and elite set standards in different fields. An appropriate selection method would be the calculation of PS of highly decorated scientists or persons acting as editors or editorial board members of journals in a corresponding field (Vinkler 2017b).

The present study reveals, the size of elite subsets strongly depends on the selected method. The number of papers in the subsets increases in the rank: P(πv) < P(π) < P(h), in general. The mean citation rate of the publications in the subsets was found to decrease with increasing number of papers.

One of the main goals of evaluative scientometrics is to present methods and indicators for science managers and science policy makers applicable in the practice. Several papers in the literature (e.g. Bornmann et al. 2011; Wildgaard et al. 2014; Leydesdorff and Wagner 2014; Vinkler 2017a, b) indicate, the assessment of publications of individuals, teams or countries may be built on indices obtained from elite subsets. Selecting appropriate methods for obtaining elite subsets, it depends strongly on the size and topic of the publication set to be analysed. Earlier experiences (Vinkler 2010a, b, 2017a) would indicate that analysing publications would require preferably, at least 25–30 journal papers. Therefore the application of the πv-method is questionable if the size of the set to be analysed would be lower than the mentioned limit.

Both h and π-index strongly depend on the field. Therefore, for comparing publication activity in different fields through elite subsets, the application of field-independent methods are preferably recommended. Calculating the citation limit of highly citedness, e.g. the number of citations (Ci) equal to or higher than the corresponding field average (Ci ≥ MCR) or its square (Ci ≥ MCR2) may be recommended. Naturally, if the skewness of citedness is high, median values can be preferably used. As field-independent citation limit, the application of the most frequently cited 0.1 or 1, or 10 per cent of total papers may also be applied.

The similar high rate of papers in the elite publication subsets published by Price medallists (Tables 5, 6) indicates, for calculating scientometric indicators, any of the subsets presented here may be applied except for the P(πv)-method because of the relatively small size of the set. Accordingly, for selecting appropriate elite subset for evaluation, primarily the following factors should be taken into consideration: size of the set to be studied, thematic coverage with the set applied as standard, and aims of the assessment.