Skip to main content
SpringerLink
Log in

Using Samples-of-Opportunity to Assess Gender Bias in Principal Evaluations of Teachers: A Cautionary Tale

  • Published:
Journal of Labor Research Aims and scope Submit manuscript

Abstract

This paper uses two “samples-of-opportunity” datasets to examine whether principal evaluations of teachers differ systematically across genders after controlling for arguably gender unbiased measures of teacher productivity---namely value-added student test scores calculated relative to other teachers in the same grade/school (where teachers are randomly allocated to classrooms within the same grade/school). While the two datasets appear to be quite similar in nature, both were samples-of-opportunity in that they were not representative of any particular population. Our findings differ substantially across datasets. This exercise reveals how results in the education and discrimination literature may be sensitive to the sample used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. http://www.dol.gov/wb/stats/leadoccupations.htm

  2. The Urban Institute also did a number of these audit studies in the early 1990s regarding racial discrimination in hiring. For example, see Turner et al. (1991) and Cross et al. (1990). Yinger (1986) uses a similar matched pairs audit type approach to look at racial discrimination by real estate agents in the housing market.

  3. Another related study is that of Doleac and Stein (2013), who look at racial discrimination among buyers by posting online ads selling Ipods and randomly assigning a picture of either a white hand or a black hand holding the Ipod. They then examined how responses and offers to the ad differed by the race of the hand holding the Ipod in the picture.

  4. Indeed, in Spurr’s (1990) analysis school rank also had no relationship with whether or not a lawyer was promoted to partner.

  5. Note this does not necessarily mean Xi is a gender unbiased signal of productivity as defined in the previous subsection. While the assumption that ui is mean zero for both genders assures E[θi|Xi,male] = E[θi|Xi,female], without further restricting the distribution of ui, we cannot rule out E[θji|Xi,male] = E[θji |Xi,female] for j > 1.

  6. Note, the reason we denote the signal observed by the employer as Xi rather than Si as used earlier, is because we want to emphasize that the productivity skill an employer observes might be different than the productivity signal the researcher observes, which is what we denoted as Si above.

  7. A handful of “pairs” were made up of more than 2 teachers, but each had at least one AC and one TC teacher.

References

  • Albanesi S, Olivetti C (2009) Home production, market production and the gender wage gap: incentives and expectations. Rev Econ Dyn 12(1):80–107

    Article  Google Scholar 

  • Altonji J, Blank R (1999) Race and gender in the labor market. Handbook of Labor Economics 3(part C):3143–3259

    Article  Google Scholar 

  • Banerjee A, Duflo E (2009) The experimental approach to development economics. Annual Review of Economics 1:151–178

    Article  Google Scholar 

  • Bertrand M, Mullainathan S (2004) Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am Econ Rev 94(4):991–1013

    Article  Google Scholar 

  • Bill and Melinda Gates Foundation (2013) Measures of effective teaching, ICPSR34771-v2. Inter-University Consortium for Political and Social Research, Ann Arbor

    Google Scholar 

  • Bjerk D (2008) Glass ceilings or sticky floors? Statistical discrimination in a dynamic model of promotion and hiring. Econ J 118:961–982

    Article  Google Scholar 

  • Bjerk D, Han S (2007) Assortative marriage and the effects of government homecare subsidy programs on gender wage and participation inequality. J Public Econ 91(5–6):1135–1150

    Article  Google Scholar 

  • Blau FD, Ferber MA, Winkler AE (2010) The economics of women, men, and work, 6th edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Blau FD, Kahn L (2000) Gender differences in pay. J Econ Perspect 14:75–99

    Article  Google Scholar 

  • Cameron AC, Gelbach JB, Miller DL (2008) Bootstrap-based improvements for inference with clustered errors. Rev Econ Stat 90(3):414–427

    Article  Google Scholar 

  • Carrell S, Sacerdote B, West J (2013) From natural variation to optimal policy? The importance of endogenous peer group formation. Econometrica 81(3):855–882

    Article  Google Scholar 

  • Coate S, Loury G (1993) Will affirmative action eliminate negative stereotypes? Am Econ Rev 83(5):1220–1240

    Google Scholar 

  • Cross H, Kenney G, Mell J, Zimmerman W (1990) Employer hiring practices: differential treatment of Hispanic and Anglo job seekers. Urban Institute report 90–4. The Urban Institute, Washington DC

    Google Scholar 

  • Doleac J, Stein L (2013) The visible hand: race and online market outcomes. Econ J 123:F469–F492

    Article  Google Scholar 

  • Francois P (1998) Gender discrimination without gender difference: theory and policy responses. J Public Econ 68:1–32

    Article  Google Scholar 

  • Ginther D, Hayes K (2003) Gender differences in salary and promotion for Faculty in the Humanities 1977-95. J Hum Resour 38:34–73

    Article  Google Scholar 

  • Goldberger A (1991) A course in econometrics. Harvard University Press, Cambrigde

  • Goldin C, Rouse C (2000) Orchestrating impartiality: the impact of `Blind' auditions on female musicians. Am Econ Rev 90:715–741

    Article  Google Scholar 

  • Hanna R, Linden L (2012) Discrimination in grading. Am Econ J Econ Pol 4(4):145–168

  • Heckman JJ (1998) Detecting discrimination. J Econ Perspect 12(2):101–116

    Article  Google Scholar 

  • Institute of Educational Sciences (2009) Teachers trained through Differet routes of certification (TTTDR). Private use data file

  • Jacob B, Lefgren L (2008) Can principals identify effective teachers? Evidence on subjective performance evaluation in education. J Labor Econ 26(1):101–136

    Article  Google Scholar 

  • Lang K (1986) A language theory of discrimination. Q J Econ 101:363–382

    Article  Google Scholar 

  • Lang K, Manove M, Dickens WT (2005) Racial discrimination in markets with announced wages. Am Econ Rev 95:1327–1340

    Article  Google Scholar 

  • Moro A, Norman P (2004) A general equilibrium model of statistical discrimination. J Econ Theory 114:1–30

    Article  Google Scholar 

  • Neumark D (1996) Sex discrimination in hiring in the restaurant industry: an audit study. Q J Econ 106(3):915–942

    Article  Google Scholar 

  • Pager D (2003) The mark of a criminal record. Am J Sociol 108(5):937–975

    Article  Google Scholar 

  • Pager D, Western B, Bonikowski B (2009) Discrimination in a Low-Wage Labor Market: A Field Experiment. Am Sociol Rev 74:777–799

    Article  Google Scholar 

  • Sacerdote B (2001) Peer effects with random assignment: results for Dartmouth roommates. Q J Econ 116(2):681–704

    Article  Google Scholar 

  • Spurr SJ (1990) Sex discrimination in the legal profession: a study of promotion. Ind Labor Relat Rev 43(4):406–417

    Article  Google Scholar 

  • Turner MA, Fix M, Struyk R (1991) Opportunities denied, opportunities diminished: racial discrimination in hiring. Urban Institute report 91–9. The Urban Institute, Washington DC

    Google Scholar 

  • Wu C-FJ (1986) Jackknife, bootstrap and other resampling methods in regression analysis. Ann Stat 14:1261–1350

    Article  Google Scholar 

  • Yinger J (1986) Measuring racial discrimination with fair housing audits: caught in the act. Am Econ Rev 76(5):881–893

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Robert Day School of Economics and Finance, Claremont McKenna College, Claremont, USA

    David Bjerk & Serkan Ozbeklik

Authors
  1. David Bjerk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Serkan Ozbeklik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Bjerk.

Ethics declarations

The authors declare that they have no conflict of interest.

Additional information

Both authors have directly participated in the planning and execution of this research study, and have read and approved the final version submitted. The contents of this manuscript have not been copyrighted or published previously and are not now under consideration for publication elsewhere. Furthermore, this manuscript will not be copyrighted, submitted, or published elsewhere, while acceptance by the Journal is under consideration. The publisher will not be held legally responsible should there be any claims for compensation.

Appendix

Appendix

Principal Evaluation Questions IES

  • Reading/Language Arts Questions (all on a 1–5 scale)

    • Discerns learnings needs of students in reading/language arts?

    • Uses advance planning in reading/language arts?

    • Leads instructional activities during reading/language arts?

    • Modifies instructional activities during reading/language arts?

  • Math Questions (all on a 1–5 scale)

    • Discerns learnings needs of students in math?

    • Uses advance planning in math?

    • Leads instructional activities during math?

    • Modifies instructional activities during math?

  • Classroom Management Questions (all on a 1–5 scale)

    • Establishes and enforces classroom rules and procedures?

    • Manages classroom time to keep students on task?

    • Encouraged desired behavior through praise, support, etc.?

    • Engages students in learning?

    • Utilize parents and school resources?

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bjerk, D., Ozbeklik, S. Using Samples-of-Opportunity to Assess Gender Bias in Principal Evaluations of Teachers: A Cautionary Tale. J Labor Res 39, 235–258 (2018). https://doi.org/10.1007/s12122-018-9271-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12122-018-9271-1

Keywords

JEL Classification

Search

Navigation