A distributional analysis of upper secondary school performance

Abstract

In many countries, the performance of young people in upper secondary education helps determine whether or not they participate in higher education. One of the weaknesses in much of the literature in this area to date has been a focus on how potential determinants, such as socio-economic status, impact the conditional mean of secondary school performance. To address this, we instead examine the relationship between the distribution of upper secondary school performance and a range of individual and school-level characteristics using unconditional quantile regression methods and data from Ireland. We find that determinants such as parental occupation group, maternal unemployment, extra private tuition and working part-time have differential effects for low- and high-ability students and that important insights are lost by focussing on the conditional mean. The implication is that while certain factors can impact on whether or not a student is likely to proceed to higher education, other factors may affect where students go and what they study.

This is a preview of subscription content, access via your institution.

Fig. 1

Source: Analysis of SLS data for 2007

Fig. 2

Source: Analysis of SLS data for 2007

Fig. 3

Source: Analysis of SLS data for 2007

Fig. 4

Source: Analysis of SLS data for 2007

Notes

  1. 1.

    This process is described in more detail in Sect. 3.

  2. 2.

    Heckman and Carneiro (2002) and Dearden et al. (2004) also argue, for the USA and UK, respectively, that credit constraints have a quite small impact on progression to third level education.

  3. 3.

    Since school type is to some extent a choice variable, controlling for endogeneity may be important. An analysis of performance in PISA tests for Ireland found that once selection was controlled for, the apparent benefit of fee paying schools disappeared (Pfefferman and Landesman 2011).

  4. 4.

    Another study within the economics of education field to use the unconditional quantile model is Andrews et al. (2016), which examines the relationship between the distribution of earnings and college quality.

  5. 5.

    See www.cao.ie for more details on the nature of the system.

  6. 6.

    This represents the latest dataset available that contains comprehensive information on CAO points and other relevant variables for young people in Ireland. Unfortunately, the survey was discontinued after 2007. According to data from the CAO, the numbers of students taking the Leaving Certificate examination has remained relatively constant over the period 2005 (54,069 candidates) to 2018 (54,440 candidates).

  7. 7.

    See Byrne et al. (2008) for more details on this dataset.

  8. 8.

    All models in the paper are estimated using sample weights. The specific weights used are designed to compensate for biases in the distribution of characteristics in the completed survey sample compared to the population of interest, which might occur due to sampling error, from the nature of the sampling frame used, differential response rates, or attrition. For more detail on the SLS sampling process, see Byrne et al. (2008).

  9. 9.

    No systematic differences between those with and without grade information was observed. Furthermore, while no national database exists to validate the representativeness of our sample on the basis of all of the variables we examine, the estimation sample was compared to a range of external data on the basis of selected variables including gender (State Examinations Commission 2019), DEIS school status (Department of Education 2019), and fee-paying school status (Department of Education 2019). Overall our data was found to be broadly representative on this basis.

  10. 10.

    A standardized variable (sometimes called a z-score or a standard score) is a variable that has been rescaled to have a mean of zero and a standard deviation of one. This is calculated by subtracting the mean of the variable, and then dividing by its standard deviation, for each individual. We present models with a standardized dependent variable and thus the estimated coefficients should be interpreted as effects measured in terms of standard deviations of CAO points. This may be more useful for readers less familiar with CAO points and the Irish system.

  11. 11.

    The quantile regressions of Koenker and Bassett (1978) model conditional quantiles but the interpretation of these is less straightforward than the approach here since individuals can be, for example, at a high unconditional quantile but a low conditional one, or vice versa.

  12. 12.

    We have excluded mother’s disability status due to multicollinearity.

  13. 13.

    See Pfeifer and Corneliẞen (2010) for a recent application. Bradley et al. (2013) analyse a single school in Ireland and find the same pattern as we do.

  14. 14.

    This shows that selection on observables is sufficient to explain the apparent premium to fee paying schools. Pfefferman & Landesman (2011) compare fee paying status of schools using Irish PISA data. They find that allowing for selection on unobservables is sufficient to drive the estimated benefit of fee-paying schools to zero (or less).

  15. 15.

    While there is overlap in the confidence intervals of some coefficients across the four quantiles examined, we focus our discussion in this section on effects that are statistically different across quantiles.

  16. 16.

    As an extension to this analysis, we also used decomposition methods to further examine the gender gap in attainment. In particular, we applied the conventional Blinder-Oaxaca decomposition to the mean (i.e. using the OLS models) as well as across the distribution. Overall, this decomposition analysis did not reveal any particularly interesting or informative results and therefore we do not present them here. They are however available from the authors on request.

References

  1. Agyire-Tettey F, Ackah CG, Asuman D (2017) An unconditional quantile regression based decomposition of spatial welfare inequalities in Ghana. J Dev Stud. https://doi.org/10.1080/00220388.2017.1296571

    Article  Google Scholar 

  2. Andrews R, Jing L, Lovenheim MF (2016) Quantile treatment effects of college quality on earnings. J Hum Resour 51:200–238

    Google Scholar 

  3. Behrendt A, Eisenach J, Johnson WR (1986) Selectivity bias and the determinants of SAT scores. Econ Educ Rev 5:363–371

    Google Scholar 

  4. Bitler MP, Gelbach JP, Hoynes HW (2006) What mean impacts miss: distributional effects of welfare reform experiments. Am Econ Rev 96:988–1012

    Google Scholar 

  5. Blundell R, Dearden L, Goodman A, Reed H (2000) The returns to higher education in Britain: evidence from a British cohort. Econ J 110:82–99

    Google Scholar 

  6. Borah B, Basu A (2013) Highlighting difference between conditional and unconditional quantile regression approaches through an application to assess medication adherence. Health Econ 22:1052–1070

    Google Scholar 

  7. Bradley S, Taylor J (1998) The effect of school size on exam performance in secondary schools. Oxford Bull Econ Stat 60:291–324

    Google Scholar 

  8. Bradley J, Keane F, Crawford S (2013) School sport and academic achievement. J Sch Health 83:8–13

    Google Scholar 

  9. Byrne D, McCoy S, Watson D (2008) School Leavers’ Survey Report 2007. Economic and Social Research Institute and Department of Education and Science, Dublin

    Google Scholar 

  10. Card D, Payne A (2002) School finance reform, the distribution of school spending, and the distribution of student test scores. J Public Econ 83:49–82

    Google Scholar 

  11. Chowdry H, Crawford C, Dearden L, Goodman A, Vignoles A (2013) Widening participation in higher education: analysis using linked administrative data. Ser A (Stat Soc) 176:431–457

    Google Scholar 

  12. CSO (2019) Retention rates of pupils in second level schools by sex, type of school and year. Available at https://www.cso.ie/px/pxeirestat/Statire/SelectVarVal/Define.asp?maintable=EDA14&PLanguage=0. Accessed 20 April 2019

  13. Cullinan J, Flannery D, Walsh S, Mc Coy S (2013) Distance effects, social class and the decision to participate in higher education in Ireland. Econ Soc Rev 44:19–51

    Google Scholar 

  14. Dearden L, Ferri J, Meghir C (2002) The effect of school quality on educational attainment and wages. Rev Econ Stat 84:1–20

    Google Scholar 

  15. Dearden L, McGranahan L, Sianesi B (2004) The role of credit constraints in educational choices: evidence from the NCDS and BCS70. CEEDP 48: Centre for the Economics of Education, London School of Economics

  16. Denny K (2010) What did abolishing university fees in Ireland do? UCD Geary Institute Discussion paper series 2010/26

  17. Denny K (2014) The effect of abolishing university tuition costs: evidence from Ireland. Labour Economics 26:26–33

    Google Scholar 

  18. Denny K, Flannery D (2017) The economics of higher education participation. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  19. Denny K, Oppedisano V (2013) The surprising effect of larger class sizes. Labour Econ 23:57–65

    Google Scholar 

  20. Depalo D, Giordano R, Papapetrou E (2015) Public–private wage differentials in Euro-area countries: evidence from quantile decomposition analysis. Empir Econ 49:985–1015

    Google Scholar 

  21. Department of Education and Science (2005) Delivering equality of opportunity in schools: an action plan for educational inclusion. Department of Education and Science, Dublin

    Google Scholar 

  22. Department of Education and Skills (2017) Data on individual schools: post primary schools list. https://www.education.ie/en/Publications/Statistics/Data-on-Individual-Schools/

  23. Department of Education and Skills (2019) DEIS School List. Available at https://www.education.ie/en/Schools-Colleges/Services/DEIS-Delivering-Equality-of-Opportunity-in-Schools-/. Accessed 17 April 2019

  24. Dickson M, Harmon C (2011) Economic returns to education: what we know, what we don’t know, and where we are going—some brief pointers. Econ Educ Rev 30:1118–1122

    Google Scholar 

  25. Dolton P, Sandi M (2017) Returning to returns: revisiting the British education evidence. Labour Economics 48:87–104

    Google Scholar 

  26. Eide E, Showalter MH (1998) The effect of school quality on student performance: a quantile regression approach. Econ Lett 58:345–350

    Google Scholar 

  27. Eyles A, Hupkau C, Machin S (2016) School reforms and pupil performance. Labour Econ 41:9–19

    Google Scholar 

  28. Firpo S, Fortin NM, Lemieux T (2009) Unconditional quantile regressions. Econometrica 77:953–973

    Google Scholar 

  29. Flannery D, Cullinan J (2014) Where they go, what they do and why it matters: the importance of geographic accessibility and social class for decisions relating to higher education institution type, degree level and field of study. Appl Econ 46:2952–2965

    Google Scholar 

  30. Flannery D, Cullinan J (2017) Economics and higher education policy. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  31. Flannery D, O’Donoghue C (2009) The determinants of higher education participation in Ireland: a micro analysis. Econ Soc Rev 40:73–107

    Google Scholar 

  32. Flannery D, O’Donoghue C (2017) The returns to third level education. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  33. Foreman-Peck J, Foreman-Peck L (2006) Should schools be smaller? The size performance relationship for Welsh schools. Econ Educ Rev 25:157–171

    Google Scholar 

  34. Fortin N, Lemieux T, Firpo S (2011) Decomposition methods in economics. In: Card D, Ashenfelter O (eds) Handbook of labor economics, vol 4. North-Holland, Amsterdam

    Google Scholar 

  35. Frölich M, Melly B (2013) Unconditional quantile treatment effects under endogeneity. J Bus Econ Stat 31:346–357

    Google Scholar 

  36. Fryer R (2010) Racial inequality in the 21st century: the declining significance of discrimination. Handb Labor Econ 4:855–971

    Google Scholar 

  37. Fuchs T, Weissman L (2007) What accounts for international differences in student performance? A re-examination using PISA data. Empir Econ 32:433–464

    Google Scholar 

  38. Galego A, Pereira J (2014) Decomposition of regional wage differences along the wage distribution in Portugal: the importance of covariates. Environ Plan A 46:2514–2532

    Google Scholar 

  39. Gorry D (2016) Heterogenous effects of sports participation on education and labor market outcomes. Educ Econ 24:622–638

    Google Scholar 

  40. Graham A, Husted T (1993) Understanding state variation in SAT scores. Econ Educ 12:197–202

    Google Scholar 

  41. Gronqvist E, Vlachos J (2016) One size fits all? The effects of teachers’ cognitive and social abilities on student achievement. Labour Econ 42:138–150

    Google Scholar 

  42. Guimarães J, Sampaio B (2013) Family background and students’ achievement on a university entrance exam in Brazil. Educ Econ 21:38–59

    Google Scholar 

  43. Haile GA, Nguyen AN (2008) Determinants of academic attainment in the United States: a quantile regression analysis of test scores. Educ Econ 16:29–57

    Google Scholar 

  44. HEA (2018) Progress review of national access plan and priorities to 2021. Higher Education Authority, Dublin

    Google Scholar 

  45. Heckman JJ, Carneiro P (2002) The evidence on credit constraints in post-secondary schooling. Econ J 112:705–734

    Google Scholar 

  46. Hinnerich BJ, Vlachos J (2017) The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations. Labour Econ 47:1–14

    Google Scholar 

  47. Humlum MK, Smith N (2015) Long-term effects of school size on students’ outcomes. Econ Educ Rev 45:28–43

    Google Scholar 

  48. Jeffers G (2002) Transition year programme and educational disadvantage. Irish Educ Stud 21:47–64

    Google Scholar 

  49. Jepsen C (2015) Class size: does it matter for student achievement? IZA World of Labor 2015:190. https://doi.org/10.15185/izawol.190

    Article  Google Scholar 

  50. Kelly E, O’Connell PJ, Smyth E (2010) The economic returns to field of study and competencies among higher education graduates in Ireland. Econ Educ Rev 29:650–657

    Google Scholar 

  51. Koenker R, Bassett G (1978) Regression quantiles. Econometrica 46:33–50

    Google Scholar 

  52. Krassel K, Heinesen E (2014) Class-size effects in secondary school. Educ Econ 22:412–426

    Google Scholar 

  53. Leithwood K, Jantzi D (2009) A review of empirical evidence about school size effects: a policy perspective. Rev Educ Res 79:464–490

    Google Scholar 

  54. Lounkaew K (2013) Explaining urban–rural differences in educational achievement in Thailand: evidence from PISA literacy data. Econ Educ Rev 37:213–225

    Google Scholar 

  55. Lunn P, Kelly E, Fitzpatrick N (2013) Keeping them in the game: taking up and dropping out of sport and exercise in Ireland. ESRI Research Series 33. ESRI, Dublin

  56. McCoy S, Smyth E (2011) Higher education expansion and differentiation in the Republic of Ireland. High Educ 61:243–260

    Google Scholar 

  57. McGuinness S, Bergin A, Kelly E, McCoy S, Smyth E, Watson D, Whelan A (2018) Evaluation of PLC Programme Provision. Dublin: ESRI Research Series No. 61

  58. O’Connell PJ, Clancy D, McCoy S (2006) Who went to college in 2004? A national survey of new entrants to higher education. Higher Education Authority, Dublin

    Google Scholar 

  59. Oreopoulos P, Salvanes KG (2011) Priceless: the non-pecuniary benefits of schooling. J Econ Inq 25:159–184

    Google Scholar 

  60. Pfefferman D, Landesman V (2011) Are private schools better than public schools? Appraisal for Ireland by methods for observational studies. Ann Appl Stat 5:1726–1751

    Google Scholar 

  61. Pfeifer C, Corneliẞen T (2010) The impact of participation in sports on educational attainment—new evidence from Germany. Econ Educ Rev 29:94–103

    Google Scholar 

  62. Porter S (2015) Quantile regression: analyzing changes in distributions instead of means. In: Paulsen MB (ed) Higher education: handbook of theory and research, vol 30. Springer, Berlin, pp 335–381

  63. Reeves R, Halikias D (2017) Race gaps in SAT scores highlight inequality and hinder social mobility. Brooking Papers on Social Mobility. Brooking Institute, Washington, DC

    Google Scholar 

  64. Rothstein J (2005) College performance predictions and SAT scores. J Econom 121:297

    Google Scholar 

  65. Schindler-Rangvid (2007) School composition effects in Denmark: quantile regression evidence from PISA 2000. Empir Econ 32:359–388

    Google Scholar 

  66. Smyth E (1999) Do schools differ?. Oak Tree Press/ESRI, Dublin

    Google Scholar 

  67. Smyth E (2009) Buying your way into college? Private tuition and the transition to higher education in Ireland. Oxf Rev Educ 35:1–22

    Google Scholar 

  68. Smyth E, Hannon C (2007) School processes and the transition to higher education. Oxf Rev Educ 33:175–194

    Google Scholar 

  69. Smyth E, McCoy S (2013) Learning from the evidence: Using educational research to inform policy. In: Lunn P, Ruanne F (eds) Using evidence to inform policy. Gill & Macmillan, Dublin

    Google Scholar 

  70. State Examinations Commission (2019) Leaving certificate sits by county and gender. https://www.examinations.ie/statistics/. Accesssed 17 April 2019

  71. Sullivan A, Heath A (2003) Intakes and examination results at state and private schools. In: Walford G (ed) British private schools: research on policy and practice. Woburn Press, London

    Google Scholar 

  72. Sullivan A, Parsons S, Wiggins R, Heath A, Green F (2014) Social origins, school type and higher education destinations. Oxf Rev Educ 40:739–763

    Google Scholar 

  73. Sylwester K (2002) Can education expenditures reduce income inequality? Econ Educ Rev 21:43–52

    Google Scholar 

  74. Weissman L (2016) The importance of school systems: evidence from international differences in student achievement. J Econ Perspect 30:3–32

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Darragh Flannery.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Fig. 4 and Tables 4, 5, 6 and 7.

Table 4 Linear regression models of upper secondary performance—raw points.
Table 5 Unconditional quantile regression models of upper secondary performance—raw points.
Table 6 Unconditional quantile regression models of upper secondary performance by gender.
Table 7 Conditional quantile regression models of upper secondary performance.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cullinan, J., Denny, K. & Flannery, D. A distributional analysis of upper secondary school performance. Empir Econ 60, 1085–1113 (2021). https://doi.org/10.1007/s00181-019-01756-8

Download citation

Keywords

  • Secondary school performance
  • Distribution
  • Unconditional quantile regression
  • Ireland

JEL Classification

  • I20
  • I21
  • J00
  • J01