Journal of Labor Research

, Volume 39, Issue 3, pp 235–258 | Cite as

Using Samples-of-Opportunity to Assess Gender Bias in Principal Evaluations of Teachers: A Cautionary Tale

  • David BjerkEmail author
  • Serkan Ozbeklik


This paper uses two “samples-of-opportunity” datasets to examine whether principal evaluations of teachers differ systematically across genders after controlling for arguably gender unbiased measures of teacher productivity---namely value-added student test scores calculated relative to other teachers in the same grade/school (where teachers are randomly allocated to classrooms within the same grade/school). While the two datasets appear to be quite similar in nature, both were samples-of-opportunity in that they were not representative of any particular population. Our findings differ substantially across datasets. This exercise reveals how results in the education and discrimination literature may be sensitive to the sample used.


Discrimination Teacher quality Principal ratings External validity 

JEL Classification

J71 J16 I21 


Compliance with Ethical Standards

The authors declare that they have no conflict of interest.


  1. Albanesi S, Olivetti C (2009) Home production, market production and the gender wage gap: incentives and expectations. Rev Econ Dyn 12(1):80–107CrossRefGoogle Scholar
  2. Altonji J, Blank R (1999) Race and gender in the labor market. Handbook of Labor Economics 3(part C):3143–3259CrossRefGoogle Scholar
  3. Banerjee A, Duflo E (2009) The experimental approach to development economics. Annual Review of Economics 1:151–178CrossRefGoogle Scholar
  4. Bertrand M, Mullainathan S (2004) Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am Econ Rev 94(4):991–1013CrossRefGoogle Scholar
  5. Bill and Melinda Gates Foundation (2013) Measures of effective teaching, ICPSR34771-v2. Inter-University Consortium for Political and Social Research, Ann ArborGoogle Scholar
  6. Bjerk D (2008) Glass ceilings or sticky floors? Statistical discrimination in a dynamic model of promotion and hiring. Econ J 118:961–982CrossRefGoogle Scholar
  7. Bjerk D, Han S (2007) Assortative marriage and the effects of government homecare subsidy programs on gender wage and participation inequality. J Public Econ 91(5–6):1135–1150CrossRefGoogle Scholar
  8. Blau FD, Ferber MA, Winkler AE (2010) The economics of women, men, and work, 6th edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
  9. Blau FD, Kahn L (2000) Gender differences in pay. J Econ Perspect 14:75–99CrossRefGoogle Scholar
  10. Cameron AC, Gelbach JB, Miller DL (2008) Bootstrap-based improvements for inference with clustered errors. Rev Econ Stat 90(3):414–427CrossRefGoogle Scholar
  11. Carrell S, Sacerdote B, West J (2013) From natural variation to optimal policy? The importance of endogenous peer group formation. Econometrica 81(3):855–882CrossRefGoogle Scholar
  12. Coate S, Loury G (1993) Will affirmative action eliminate negative stereotypes? Am Econ Rev 83(5):1220–1240Google Scholar
  13. Cross H, Kenney G, Mell J, Zimmerman W (1990) Employer hiring practices: differential treatment of Hispanic and Anglo job seekers. Urban Institute report 90–4. The Urban Institute, Washington DCGoogle Scholar
  14. Doleac J, Stein L (2013) The visible hand: race and online market outcomes. Econ J 123:F469–F492CrossRefGoogle Scholar
  15. Francois P (1998) Gender discrimination without gender difference: theory and policy responses. J Public Econ 68:1–32CrossRefGoogle Scholar
  16. Ginther D, Hayes K (2003) Gender differences in salary and promotion for Faculty in the Humanities 1977-95. J Hum Resour 38:34–73CrossRefGoogle Scholar
  17. Goldberger A (1991) A course in econometrics. Harvard University Press, CambrigdeGoogle Scholar
  18. Goldin C, Rouse C (2000) Orchestrating impartiality: the impact of `Blind' auditions on female musicians. Am Econ Rev 90:715–741CrossRefGoogle Scholar
  19. Hanna R, Linden L (2012) Discrimination in grading. Am Econ J Econ Pol 4(4):145–168Google Scholar
  20. Heckman JJ (1998) Detecting discrimination. J Econ Perspect 12(2):101–116CrossRefGoogle Scholar
  21. Institute of Educational Sciences (2009) Teachers trained through Differet routes of certification (TTTDR). Private use data fileGoogle Scholar
  22. Jacob B, Lefgren L (2008) Can principals identify effective teachers? Evidence on subjective performance evaluation in education. J Labor Econ 26(1):101–136CrossRefGoogle Scholar
  23. Lang K (1986) A language theory of discrimination. Q J Econ 101:363–382CrossRefGoogle Scholar
  24. Lang K, Manove M, Dickens WT (2005) Racial discrimination in markets with announced wages. Am Econ Rev 95:1327–1340CrossRefGoogle Scholar
  25. Moro A, Norman P (2004) A general equilibrium model of statistical discrimination. J Econ Theory 114:1–30CrossRefGoogle Scholar
  26. Neumark D (1996) Sex discrimination in hiring in the restaurant industry: an audit study. Q J Econ 106(3):915–942CrossRefGoogle Scholar
  27. Pager D (2003) The mark of a criminal record. Am J Sociol 108(5):937–975CrossRefGoogle Scholar
  28. Pager D, Western B, Bonikowski B (2009) Discrimination in a Low-Wage Labor Market: A Field Experiment. Am Sociol Rev 74:777–799CrossRefGoogle Scholar
  29. Sacerdote B (2001) Peer effects with random assignment: results for Dartmouth roommates. Q J Econ 116(2):681–704CrossRefGoogle Scholar
  30. Spurr SJ (1990) Sex discrimination in the legal profession: a study of promotion. Ind Labor Relat Rev 43(4):406–417CrossRefGoogle Scholar
  31. Turner MA, Fix M, Struyk R (1991) Opportunities denied, opportunities diminished: racial discrimination in hiring. Urban Institute report 91–9. The Urban Institute, Washington DCGoogle Scholar
  32. Wu C-FJ (1986) Jackknife, bootstrap and other resampling methods in regression analysis. Ann Stat 14:1261–1350CrossRefGoogle Scholar
  33. Yinger J (1986) Measuring racial discrimination with fair housing audits: caught in the act. Am Econ Rev 76(5):881–893Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The Robert Day School of Economics and FinanceClaremont McKenna CollegeClaremontUSA

Personalised recommendations