Skip to main content

Advertisement

SpringerLink
Log in

A distributional analysis of upper secondary school performance

  • Published:
Empirical Economics Aims and scope Submit manuscript

Abstract

In many countries, the performance of young people in upper secondary education helps determine whether or not they participate in higher education. One of the weaknesses in much of the literature in this area to date has been a focus on how potential determinants, such as socio-economic status, impact the conditional mean of secondary school performance. To address this, we instead examine the relationship between the distribution of upper secondary school performance and a range of individual and school-level characteristics using unconditional quantile regression methods and data from Ireland. We find that determinants such as parental occupation group, maternal unemployment, extra private tuition and working part-time have differential effects for low- and high-ability students and that important insights are lost by focussing on the conditional mean. The implication is that while certain factors can impact on whether or not a student is likely to proceed to higher education, other factors may affect where students go and what they study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Source: Analysis of SLS data for 2007

Fig. 2

Source: Analysis of SLS data for 2007

Fig. 3

Source: Analysis of SLS data for 2007

Fig. 4

Source: Analysis of SLS data for 2007

Similar content being viewed by others

Notes

  1. This process is described in more detail in Sect. 3.

  2. Heckman and Carneiro (2002) and Dearden et al. (2004) also argue, for the USA and UK, respectively, that credit constraints have a quite small impact on progression to third level education.

  3. Since school type is to some extent a choice variable, controlling for endogeneity may be important. An analysis of performance in PISA tests for Ireland found that once selection was controlled for, the apparent benefit of fee paying schools disappeared (Pfefferman and Landesman 2011).

  4. Another study within the economics of education field to use the unconditional quantile model is Andrews et al. (2016), which examines the relationship between the distribution of earnings and college quality.

  5. See www.cao.ie for more details on the nature of the system.

  6. This represents the latest dataset available that contains comprehensive information on CAO points and other relevant variables for young people in Ireland. Unfortunately, the survey was discontinued after 2007. According to data from the CAO, the numbers of students taking the Leaving Certificate examination has remained relatively constant over the period 2005 (54,069 candidates) to 2018 (54,440 candidates).

  7. See Byrne et al. (2008) for more details on this dataset.

  8. All models in the paper are estimated using sample weights. The specific weights used are designed to compensate for biases in the distribution of characteristics in the completed survey sample compared to the population of interest, which might occur due to sampling error, from the nature of the sampling frame used, differential response rates, or attrition. For more detail on the SLS sampling process, see Byrne et al. (2008).

  9. No systematic differences between those with and without grade information was observed. Furthermore, while no national database exists to validate the representativeness of our sample on the basis of all of the variables we examine, the estimation sample was compared to a range of external data on the basis of selected variables including gender (State Examinations Commission 2019), DEIS school status (Department of Education 2019), and fee-paying school status (Department of Education 2019). Overall our data was found to be broadly representative on this basis.

  10. A standardized variable (sometimes called a z-score or a standard score) is a variable that has been rescaled to have a mean of zero and a standard deviation of one. This is calculated by subtracting the mean of the variable, and then dividing by its standard deviation, for each individual. We present models with a standardized dependent variable and thus the estimated coefficients should be interpreted as effects measured in terms of standard deviations of CAO points. This may be more useful for readers less familiar with CAO points and the Irish system.

  11. The quantile regressions of Koenker and Bassett (1978) model conditional quantiles but the interpretation of these is less straightforward than the approach here since individuals can be, for example, at a high unconditional quantile but a low conditional one, or vice versa.

  12. We have excluded mother’s disability status due to multicollinearity.

  13. See Pfeifer and Corneliẞen (2010) for a recent application. Bradley et al. (2013) analyse a single school in Ireland and find the same pattern as we do.

  14. This shows that selection on observables is sufficient to explain the apparent premium to fee paying schools. Pfefferman & Landesman (2011) compare fee paying status of schools using Irish PISA data. They find that allowing for selection on unobservables is sufficient to drive the estimated benefit of fee-paying schools to zero (or less).

  15. While there is overlap in the confidence intervals of some coefficients across the four quantiles examined, we focus our discussion in this section on effects that are statistically different across quantiles.

  16. As an extension to this analysis, we also used decomposition methods to further examine the gender gap in attainment. In particular, we applied the conventional Blinder-Oaxaca decomposition to the mean (i.e. using the OLS models) as well as across the distribution. Overall, this decomposition analysis did not reveal any particularly interesting or informative results and therefore we do not present them here. They are however available from the authors on request.

References

  • Agyire-Tettey F, Ackah CG, Asuman D (2017) An unconditional quantile regression based decomposition of spatial welfare inequalities in Ghana. J Dev Stud. https://doi.org/10.1080/00220388.2017.1296571

    Article  Google Scholar 

  • Andrews R, Jing L, Lovenheim MF (2016) Quantile treatment effects of college quality on earnings. J Hum Resour 51:200–238

    Google Scholar 

  • Behrendt A, Eisenach J, Johnson WR (1986) Selectivity bias and the determinants of SAT scores. Econ Educ Rev 5:363–371

    Google Scholar 

  • Bitler MP, Gelbach JP, Hoynes HW (2006) What mean impacts miss: distributional effects of welfare reform experiments. Am Econ Rev 96:988–1012

    Google Scholar 

  • Blundell R, Dearden L, Goodman A, Reed H (2000) The returns to higher education in Britain: evidence from a British cohort. Econ J 110:82–99

    Google Scholar 

  • Borah B, Basu A (2013) Highlighting difference between conditional and unconditional quantile regression approaches through an application to assess medication adherence. Health Econ 22:1052–1070

    Google Scholar 

  • Bradley S, Taylor J (1998) The effect of school size on exam performance in secondary schools. Oxford Bull Econ Stat 60:291–324

    Google Scholar 

  • Bradley J, Keane F, Crawford S (2013) School sport and academic achievement. J Sch Health 83:8–13

    Google Scholar 

  • Byrne D, McCoy S, Watson D (2008) School Leavers’ Survey Report 2007. Economic and Social Research Institute and Department of Education and Science, Dublin

    Google Scholar 

  • Card D, Payne A (2002) School finance reform, the distribution of school spending, and the distribution of student test scores. J Public Econ 83:49–82

    Google Scholar 

  • Chowdry H, Crawford C, Dearden L, Goodman A, Vignoles A (2013) Widening participation in higher education: analysis using linked administrative data. Ser A (Stat Soc) 176:431–457

    Google Scholar 

  • CSO (2019) Retention rates of pupils in second level schools by sex, type of school and year. Available at https://www.cso.ie/px/pxeirestat/Statire/SelectVarVal/Define.asp?maintable=EDA14&PLanguage=0. Accessed 20 April 2019

  • Cullinan J, Flannery D, Walsh S, Mc Coy S (2013) Distance effects, social class and the decision to participate in higher education in Ireland. Econ Soc Rev 44:19–51

    Google Scholar 

  • Dearden L, Ferri J, Meghir C (2002) The effect of school quality on educational attainment and wages. Rev Econ Stat 84:1–20

    Google Scholar 

  • Dearden L, McGranahan L, Sianesi B (2004) The role of credit constraints in educational choices: evidence from the NCDS and BCS70. CEEDP 48: Centre for the Economics of Education, London School of Economics

  • Denny K (2010) What did abolishing university fees in Ireland do? UCD Geary Institute Discussion paper series 2010/26

  • Denny K (2014) The effect of abolishing university tuition costs: evidence from Ireland. Labour Economics 26:26–33

    Google Scholar 

  • Denny K, Flannery D (2017) The economics of higher education participation. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  • Denny K, Oppedisano V (2013) The surprising effect of larger class sizes. Labour Econ 23:57–65

    Google Scholar 

  • Depalo D, Giordano R, Papapetrou E (2015) Public–private wage differentials in Euro-area countries: evidence from quantile decomposition analysis. Empir Econ 49:985–1015

    Google Scholar 

  • Department of Education and Science (2005) Delivering equality of opportunity in schools: an action plan for educational inclusion. Department of Education and Science, Dublin

    Google Scholar 

  • Department of Education and Skills (2017) Data on individual schools: post primary schools list. https://www.education.ie/en/Publications/Statistics/Data-on-Individual-Schools/

  • Department of Education and Skills (2019) DEIS School List. Available at https://www.education.ie/en/Schools-Colleges/Services/DEIS-Delivering-Equality-of-Opportunity-in-Schools-/. Accessed 17 April 2019

  • Dickson M, Harmon C (2011) Economic returns to education: what we know, what we don’t know, and where we are going—some brief pointers. Econ Educ Rev 30:1118–1122

    Google Scholar 

  • Dolton P, Sandi M (2017) Returning to returns: revisiting the British education evidence. Labour Economics 48:87–104

    Google Scholar 

  • Eide E, Showalter MH (1998) The effect of school quality on student performance: a quantile regression approach. Econ Lett 58:345–350

    Google Scholar 

  • Eyles A, Hupkau C, Machin S (2016) School reforms and pupil performance. Labour Econ 41:9–19

    Google Scholar 

  • Firpo S, Fortin NM, Lemieux T (2009) Unconditional quantile regressions. Econometrica 77:953–973

    Google Scholar 

  • Flannery D, Cullinan J (2014) Where they go, what they do and why it matters: the importance of geographic accessibility and social class for decisions relating to higher education institution type, degree level and field of study. Appl Econ 46:2952–2965

    Google Scholar 

  • Flannery D, Cullinan J (2017) Economics and higher education policy. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  • Flannery D, O’Donoghue C (2009) The determinants of higher education participation in Ireland: a micro analysis. Econ Soc Rev 40:73–107

    Google Scholar 

  • Flannery D, O’Donoghue C (2017) The returns to third level education. In: Cullinan J, Flannery D (eds) Economic insights on higher education policy in Ireland. Palgrave Macmillan, Cham

    Google Scholar 

  • Foreman-Peck J, Foreman-Peck L (2006) Should schools be smaller? The size performance relationship for Welsh schools. Econ Educ Rev 25:157–171

    Google Scholar 

  • Fortin N, Lemieux T, Firpo S (2011) Decomposition methods in economics. In: Card D, Ashenfelter O (eds) Handbook of labor economics, vol 4. North-Holland, Amsterdam

    Google Scholar 

  • Frölich M, Melly B (2013) Unconditional quantile treatment effects under endogeneity. J Bus Econ Stat 31:346–357

    Google Scholar 

  • Fryer R (2010) Racial inequality in the 21st century: the declining significance of discrimination. Handb Labor Econ 4:855–971

    Google Scholar 

  • Fuchs T, Weissman L (2007) What accounts for international differences in student performance? A re-examination using PISA data. Empir Econ 32:433–464

    Google Scholar 

  • Galego A, Pereira J (2014) Decomposition of regional wage differences along the wage distribution in Portugal: the importance of covariates. Environ Plan A 46:2514–2532

    Google Scholar 

  • Gorry D (2016) Heterogenous effects of sports participation on education and labor market outcomes. Educ Econ 24:622–638

    Google Scholar 

  • Graham A, Husted T (1993) Understanding state variation in SAT scores. Econ Educ 12:197–202

    Google Scholar 

  • Gronqvist E, Vlachos J (2016) One size fits all? The effects of teachers’ cognitive and social abilities on student achievement. Labour Econ 42:138–150

    Google Scholar 

  • Guimarães J, Sampaio B (2013) Family background and students’ achievement on a university entrance exam in Brazil. Educ Econ 21:38–59

    Google Scholar 

  • Haile GA, Nguyen AN (2008) Determinants of academic attainment in the United States: a quantile regression analysis of test scores. Educ Econ 16:29–57

    Google Scholar 

  • HEA (2018) Progress review of national access plan and priorities to 2021. Higher Education Authority, Dublin

    Google Scholar 

  • Heckman JJ, Carneiro P (2002) The evidence on credit constraints in post-secondary schooling. Econ J 112:705–734

    Google Scholar 

  • Hinnerich BJ, Vlachos J (2017) The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations. Labour Econ 47:1–14

    Google Scholar 

  • Humlum MK, Smith N (2015) Long-term effects of school size on students’ outcomes. Econ Educ Rev 45:28–43

    Google Scholar 

  • Jeffers G (2002) Transition year programme and educational disadvantage. Irish Educ Stud 21:47–64

    Google Scholar 

  • Jepsen C (2015) Class size: does it matter for student achievement? IZA World of Labor 2015:190. https://doi.org/10.15185/izawol.190

    Article  Google Scholar 

  • Kelly E, O’Connell PJ, Smyth E (2010) The economic returns to field of study and competencies among higher education graduates in Ireland. Econ Educ Rev 29:650–657

    Google Scholar 

  • Koenker R, Bassett G (1978) Regression quantiles. Econometrica 46:33–50

    Google Scholar 

  • Krassel K, Heinesen E (2014) Class-size effects in secondary school. Educ Econ 22:412–426

    Google Scholar 

  • Leithwood K, Jantzi D (2009) A review of empirical evidence about school size effects: a policy perspective. Rev Educ Res 79:464–490

    Google Scholar 

  • Lounkaew K (2013) Explaining urban–rural differences in educational achievement in Thailand: evidence from PISA literacy data. Econ Educ Rev 37:213–225

    Google Scholar 

  • Lunn P, Kelly E, Fitzpatrick N (2013) Keeping them in the game: taking up and dropping out of sport and exercise in Ireland. ESRI Research Series 33. ESRI, Dublin

  • McCoy S, Smyth E (2011) Higher education expansion and differentiation in the Republic of Ireland. High Educ 61:243–260

    Google Scholar 

  • McGuinness S, Bergin A, Kelly E, McCoy S, Smyth E, Watson D, Whelan A (2018) Evaluation of PLC Programme Provision. Dublin: ESRI Research Series No. 61

  • O’Connell PJ, Clancy D, McCoy S (2006) Who went to college in 2004? A national survey of new entrants to higher education. Higher Education Authority, Dublin

    Google Scholar 

  • Oreopoulos P, Salvanes KG (2011) Priceless: the non-pecuniary benefits of schooling. J Econ Inq 25:159–184

    Google Scholar 

  • Pfefferman D, Landesman V (2011) Are private schools better than public schools? Appraisal for Ireland by methods for observational studies. Ann Appl Stat 5:1726–1751

    Google Scholar 

  • Pfeifer C, Corneliẞen T (2010) The impact of participation in sports on educational attainment—new evidence from Germany. Econ Educ Rev 29:94–103

    Google Scholar 

  • Porter S (2015) Quantile regression: analyzing changes in distributions instead of means. In: Paulsen MB (ed) Higher education: handbook of theory and research, vol 30. Springer, Berlin, pp 335–381

  • Reeves R, Halikias D (2017) Race gaps in SAT scores highlight inequality and hinder social mobility. Brooking Papers on Social Mobility. Brooking Institute, Washington, DC

    Google Scholar 

  • Rothstein J (2005) College performance predictions and SAT scores. J Econom 121:297

    Google Scholar 

  • Schindler-Rangvid (2007) School composition effects in Denmark: quantile regression evidence from PISA 2000. Empir Econ 32:359–388

    Google Scholar 

  • Smyth E (1999) Do schools differ?. Oak Tree Press/ESRI, Dublin

    Google Scholar 

  • Smyth E (2009) Buying your way into college? Private tuition and the transition to higher education in Ireland. Oxf Rev Educ 35:1–22

    Google Scholar 

  • Smyth E, Hannon C (2007) School processes and the transition to higher education. Oxf Rev Educ 33:175–194

    Google Scholar 

  • Smyth E, McCoy S (2013) Learning from the evidence: Using educational research to inform policy. In: Lunn P, Ruanne F (eds) Using evidence to inform policy. Gill & Macmillan, Dublin

    Google Scholar 

  • State Examinations Commission (2019) Leaving certificate sits by county and gender. https://www.examinations.ie/statistics/. Accesssed 17 April 2019

  • Sullivan A, Heath A (2003) Intakes and examination results at state and private schools. In: Walford G (ed) British private schools: research on policy and practice. Woburn Press, London

    Google Scholar 

  • Sullivan A, Parsons S, Wiggins R, Heath A, Green F (2014) Social origins, school type and higher education destinations. Oxf Rev Educ 40:739–763

    Google Scholar 

  • Sylwester K (2002) Can education expenditures reduce income inequality? Econ Educ Rev 21:43–52

    Google Scholar 

  • Weissman L (2016) The importance of school systems: evidence from international differences in student achievement. J Econ Perspect 30:3–32

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. JE Cairnes School of Business and Economics, National University of Ireland, Galway, Ireland

    John Cullinan

  2. School of Economics, Geary Institute for Public Policy, University College Dublin, Dublin, Ireland

    Kevin Denny

  3. Department of Economics, Kemmy Business School, University of Limerick, Limerick, Ireland

    Darragh Flannery

Authors
  1. John Cullinan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Denny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Darragh Flannery
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darragh Flannery.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Fig. 4 and Tables 4, 5, 6 and 7.

Table 4 Linear regression models of upper secondary performance—raw points.
Full size table
Table 5 Unconditional quantile regression models of upper secondary performance—raw points.
Full size table
Table 6 Unconditional quantile regression models of upper secondary performance by gender.
Full size table
Table 7 Conditional quantile regression models of upper secondary performance.
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cullinan, J., Denny, K. & Flannery, D. A distributional analysis of upper secondary school performance. Empir Econ 60, 1085–1113 (2021). https://doi.org/10.1007/s00181-019-01756-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00181-019-01756-8

Keywords

JEL Classification

Search

Navigation