Advertisement

Using Google Scholar to track the scholarly output of research groups

  • Brent ThomaEmail author
  • Teresa M Chan
Open Access
Show and Tell

Abstract

Introduction

It is often necessary to demonstrate the impact of a research program over time both within and beyond institutions. However, it is difficult to accurately track the publications of research groups over time without significant effort. A simple, scalable, and economical way to track publications from research groups and their metrics would address this challenge.

Methods

Google Scholar automatically tracks the scholarly output and citation counts of individual researchers. We created Google Scholar profiles to track the scholarly productivity of five research groups: an institutional educational research program, a division of emergency medicine, a department of emergency medicine, a national educational scholarship working group, and an international organization dedicated to online education. We added the publications of each group member to their respective group Google Scholar profile and a junior faculty member monitored the citations that were suggested.

Results

Google Scholar tracked a diverse collection of five research groups over 6–36 months. In addition to having different organizational structures and purposes, the groups varied in size, consisting of 8–60 researchers, and prolificacy, with group citation counts between 1006–58,380 and group h‑indexes ranging from 19–101.

Discussion

We anticipate that as this innovation becomes better known it will increasingly be adopted by traditional and non-traditional research groups to easily track their productivity and impact. Additional initiatives will be needed to standardize reporting guidelines within and between institutions.

Keywords

Research group Google Scholar Research impact 

Introduction

Tracking the publications of a group of scientists (e.g. in an academic department, lab group, research collaborative, or institution) over time can be difficult [1]. Productive groups have numerous authors who publish frequently. Faculty, post-doctoral students, and graduate students move between institutions, muddying amalgamated metrics. Authors may not submit their latest publications on a regular basis resulting in missing metrics.

However, tracking these metrics is important to research groups who need to justify their use of resources, demonstrate the growth of their research programs over time, and promote the success of their research program [2]. As a result, several methods have been developed to track publication metrics. For example, some academic departments have their researchers submit their curricula vitae (CVs) annually. While this results in a comprehensive database, it is resource intensive to pull publications from CVs into a central registry. Digital solutions include institutional subscriptions to citation indices such as Scopus (Elsevier) and Web of Science (Clarivate Analytics) [3]. However, they cost money to access and do not make their results publicly available. Further, while they can track and amalgamate the publication metrics of individual faculty, they have more difficulty tracking research groups that are more ambiguous. ResearchGate provides a free service that can be used by research or project groups; however, its citation tracking is incomplete (it is limited to papers that have been uploaded to ResearchGate) [4].

Google Scholar has been used successfully by individual researchers to track their scholarly output and citations and is thought to be as good as many other search engines as a source of bibliometric data [5]. Notably, once papers have been added to a profile, it is able to identify new publications by the same researchers. Although Google Scholar has been criticized for being over-inclusive, it is becoming an accepted academic standard [5]. Here, we aim to determine whether it is feasible for a variety of research groups to use Google Scholar profiles to track their scholarly output.

Methods

We created Google Scholar profiles for five types of research groups: the members of an educational research program (the McMaster program for Education Research, Innovation and Theory), the faculty of an academic division (the Division of Emergency Medicine at McMaster University), the faculty of an academic department (the Department of Emergency Medicine at the University of Saskatchewan), the members of a national working group (the Education Working Group (EWG) of the Canadian Association of Emergency Physicians (CAEP)), and the editors of an international online education website (Academic Life in Emergency Medicine (ALiEM), https://aliem.com/). These five groups were intentionally selected to range broadly in terms of size (from small to large) and ambiguity (from a defined department to a relatively poorly defined working group). Institutional Review Board/Research Ethics Board approval was not sought as this work met the criteria for exemption under performance review. Further, only publicly available data were utilized.

A labelled outline of a group Google Scholar profile is provided in Fig. 1 while a screencast demonstrating the creation of a group profile is available as a Video on the journal website. In short, Google email (Gmail) addresses were created for each account using an institutional email address. The authors developed a list of members for each research group and searched for their research on Google Scholar. Each member’s research was found on Google Scholar and added to the group profile. To prevent the addition of inappropriate papers and metrics to the accounts, the Google Scholar settings were set to require approval from an account administrator before adding new papers to the publication list. Criteria for the appropriateness of an article were determined by the function of the group (e.g. all medical articles were included for the division/department, but only medical education articles were included for the education scholars group).
Fig. 1

Labelled Google Scholar profile of the University of Saskatchewan Department of Emergency Medicine as of 24 July 2018

The outcome of interest was the feasibility of setting up group Google Scholar profiles for various research groups. The quantitative characteristics of each research group were collected to demonstrate their diversity. Qualitative descriptions were used to describe the feasibility and challenges encountered in the set-up and maintenance of the research groups.

Results

We were able to set up and begin populating new Google Scholar accounts for each research group (Table 1) within minutes. When a member of the research group had a pre-existing, up-to-date Google Scholar profile, adding their publications to the group profile page was as simple as identifying it. When a member of the research group did not have a pre-existing Google Scholar profile, it was equally quick to find a publication list, but it had to be vetted with more care. After adding roughly 10 papers by a given author, the Google Scholar algorithms were able to accurately suggest additional updates over time. The groups varied in size (8–60 researchers) and publication history (citations ranged from 1006 to 58,380; h‑index ranged from 19–101).
Table 1

Group Google Scholar Profile data as of 5 August 2018

Type of group

Tracking since

Institution or group

Number of scholars

Total citations

Group h‑index

Academic division

September 2015

McMaster University Division of Emergency Medicine

33

12,590

 50

Academic department

August 2015

University of Saskatchewan Department of Emergency Medicine

11

  1411

 23

Research centre

November 2017

McMaster Program for Education Research, Innovation, and Theory

 8

58,380

101

Online education organization

October 2017

Academic Life in Emergency Medicine

15

  1006

 19

National working group

January 2018

Canadian Association of Emergency Physicians Education Scholars Group

60

13,035

 53

Due to its size and scope, the creation of the profile for the CAEP Education Scholars Group [6] provided the greatest challenge. It included 60 researchers from multiple institutions and many of them had never published with each other before. However, even in this case it was possible to create and populate the Google Profile page in less than an hour. Given the large volume of articles and the focus of the group on medical education scholarship, it took another 2 hours to review the articles suggested for the members to determine whether they were appropriate for inclusion.

After the groups were created, the scholarly output of the research groups was tracked for 6–30 months. Each account was managed by a single junior faculty member who reviewed the auto-updates for a few minutes every 1–2 months. Human oversight was required to ensure that the suggested additions were accurate, particularly if a group member had a common name. It was helpful for the junior faculty member reviewing the suggested additions to have knowledge of the individuals within the group and their research programs as this allowed them to include and exclude articles quickly.

Discussion

While the creation of group Google Scholar profiles has been described in guides online as early as 2013 [7], this is the first report that we are aware of in the published literature that demonstrates the potential and versatility of this innovation. Given the ease of set-up and tracking we anticipate that, if the ability to create group profiles were more widely known, traditional and non-traditional academic groups would increasingly create profiles to quantify their team’s scholarly productivity. These profiles may be particularly useful to small or poorly resourced research groups or institutions, medical education centres which draw faculty from numerous departments, and research entities with faculty disseminated across institutions.

There are multiple potential benefits to this beyond tracking the growth of group publications and citation metrics [2]. First, the creation of a group Google Scholar page can serve to advertise the success of a group in a publicly accessible way. Second, it provides a relatively comparable way to contrast the research productivity of various groups. Third, other scholars interested in the work of a group will be able to find it without searching between authors and journals over years. Fourth, new research published by a group could be easily tracked by signing up for publication alerts from its Google Scholar profile. Finally, we hypothesize that tracking the collective research output of a group (as opposed to an individual) is likely to foster collaboration and the pursuit of collective goals. For example, the Department of Emergency Medicine at the University of Saskatchewan uses its Google Scholar profile to track the productivity goals in its strategic plan, direct trainees to faculty conducting research in various areas, and compare its progress to related groups such as the Division of Emergency Medicine at McMaster University.

As outlined in Table 1, not all groups are the same and interpretations of the collective scholarly output should be made only while considering the context (e.g. number of researchers, field of research, mission of the group) of a given group. At the very minimum, the number of scholars within a group should be taken into consideration. For instance, it may be worth pursuing some sort of comparative metric that allows for some element of adjustment for the number of scholars within a given group. To compare groups, new metrics will be required to adjust for various elements that can skew the data. For instance, a single eminent scholar with many citations may mask the lower performance of new faculty or an entire group. Finally, Google Scholar is but one method by which one might quantify bibliometric contributions and productivity. There are many types of scholarship that simply will not be well captured via bibliometric measures—those leaders looking to encourage curriculum design for teachers or quality improvement in their hospitals may find these metrics limited in their ability to capture these newer forms of academic health sciences scholarship. A judicious examination of one’s own group that aligns outcome measures with desired productivity would be optimal for fostering the success of all [8].

Conclusions

While research metrics have been tracked for decades, Google Scholar profiles can provide a wide variety of research groups with an easy, simple, free, and scalable solution to the problem of tracking their collective scholarly output. With broader adoption of these techniques it will become increasingly possible to quantify and compare the productivity of a variety of types of scholarly organizations.

Notes

Conflict of interest

B. Thoma and T.M. Chan declare that they have no competing interests.

Supplementary material

Video 1. Video guide to creating a group Google Scholar profile

References

  1. 1.
    Toutkoushian RK, Porter SR, Danielson C, Hollis PR. Using publications counts to measure an institution’s research productivity. Res High Educ. 2003;44:121–48.CrossRefGoogle Scholar
  2. 2.
    Greenhalgh T, Raftery J, Hanney S, Glover M. Research impact: a narrative review. Bmc Med. 2016;14(1):78.CrossRefGoogle Scholar
  3. 3.
    Bakkalbasi N, Bauer K, Glover J, Wang L. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomed Digit Libr. 2006;3:1–8.CrossRefGoogle Scholar
  4. 4.
    Citations. ResearchGate. https://explore.researchgate.net/display/support/Citations. Accessed 22 Jan 2019.
  5. 5.
    Harzing AW, Alakangas S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics. 2016;106:787–804.CrossRefGoogle Scholar
  6. 6.
    Woods R, Chan T, Thoma B, Sherbino J. Education scholarship in Canadian emergency medicine: the past, present, and future. Can J Emerg Med. 2018;20:164–8.Google Scholar
  7. 7.
    Rowlands I. Creating resarch group profiles in Google scholar. Univ Leicester Libr. 2013. https://www2.le.ac.uk/library/downloads/research-services/UoL_Google_Scholar_Group_Profiles.pdf. Accessed 24 July 2018.Google Scholar
  8. 8.
    Chan T, Kuehl D. On lampposts, sneetches, and stars: a call to go beyond bibliometrics for determining academic value. Acad Emerg Med. 2019. https://onlinelibrary.wiley.com/doi/abs/10.1111/acem.13707

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Emergency MedicineUniversity of SaskatchewanSaskatoonCanada
  2. 2.Division of Emergency Medicine, Department of MedicineMcMaster UniversityHamiltonCanada
  3. 3.McMaster program for Educational Research, Innovation, and TheoryMcMaster UniversityHamiltonCanada

Personalised recommendations