Identifying emerging scholars: seeing through the crystal ball of scholarship selection committees

  • Vincent ChandlerEmail author


To better understand the added-value of the academic evaluation process, this paper studies the relationship between scores given by 105 evaluators to 1900 doctoral candidates who received a scholarship and their outcomes 10 years after the competition. I first find that a one point increase in total score is associated with a 1.4 percentage point (2.1% of a s.e.) increase in the probability of completing a Ph.D. in 5 years, with a 1.0 percentage point (2.1% of a s.e.) increase in the probability of completing a Ph.D. in 10 years, and with a 1.4 percentage point increase (3% of a s.e.) in the probability of becoming a tenure-track professor 10 years after the competition. I then use the individual evaluator-candidate scores to provide evidence that male evaluators give higher scores than do female evaluators to students who complete their doctoral program in 5 years. Since there is no difference between scores given by male and female evaluators to candidates who become tenure-track professors, male evaluators seem more focused on shorter time to degree than are female evaluators.


Gender Prediction Communication Selection Graduate scholarships Selection Higher education Research 

JEL Classification

H52 I23 J16 


  1. Bagues, M., Sylos-Labini, M., & Zinovyeva, N. (2017). Does the gender composition of scientific committees matter? American Economic Review, 107(4), 1207–38.CrossRefGoogle Scholar
  2. Bornmann, L., Mutz, R., & Daniel, H. D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS ONE, 5(12), e14331.CrossRefGoogle Scholar
  3. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology, 45(1), 197–245.CrossRefGoogle Scholar
  4. CAUT. (2014). CAUT almanach of post-secondary education in Canada.Google Scholar
  5. Chandler, V. (2018). Short and long-term impacts of an increase in graduate funding. Economics of Education Review, 62, 104–112.CrossRefGoogle Scholar
  6. Dittrich, M., & Leipold, K. (2014). Gender differences in time preferences. Economics Letters, 122(3), 413–415.CrossRefGoogle Scholar
  7. Edge, J. (2015). Inside and outside the academy: Valuing and preparing Ph.Ds. for careers. In Conference board of Canada.Google Scholar
  8. Fang, F. C., Bowen, A., & Casadevall, A. (2016). NIH peer review percentile scores are poorly predictive of grant productivity. Elife, 5, e13323.CrossRefGoogle Scholar
  9. Jacob, B., & Lefgren, L. (2011). The impact of NIH research grant funding on scientific productivity. Journal of Public Economics, 45(9–10), 1168–1177.CrossRefGoogle Scholar
  10. King, D. (2008). Doctoral Graduates in Canada: Findings from the survey of earned doctorates, 2004/2005. Culture, Tourism and the Centre for Education Statistics Research papers. Statistics Canada Catalogue no. 81-595-M No. 065.Google Scholar
  11. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.CrossRefGoogle Scholar
  12. Li, D., & Agha, L. (2015). Big names or big ideas: Do peer review panels select the best science proposals. Science, 348(6233), 434–438.CrossRefGoogle Scholar
  13. Matsa, D., & Miller, A. (2013). A female style in corporate leadership? Evidence from Quotas, American Economic Journal: Applied Economics, 5(3), 136–169.Google Scholar
  14. Moneta-Koehler, L., Brown, A. M., Petrie, K. A., Evans, B. J., & Chalkley, R. (2017). The limitations of the GRE in predicting success in biomedical graduate school. PLoS ONE, 12(1), e0166742.CrossRefGoogle Scholar
  15. SSHRC. (2016). Apply for funding: Doctoral awards. Accessed 1 Feb 2019.
  16. Sternberg, R. J., & Williams, W. M. (1997). Does the Graduate Record Examination predict meaningful success in the graduate training of psychology? A case study. American Psychologist, 52(6), 630.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2019

Authors and Affiliations

  1. 1.UQOGatineauCanada

Personalised recommendations