Advertisement

Statistical Methods & Applications

, Volume 21, Issue 2, pp 193–209 | Cite as

Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator

  • Isabella SulisEmail author
  • Mariano Porcu
Article

Abstract

Taking into account the students’ evaluation of the quality of degree programs this paper presents a proposal for building up an adjusted performance indicator based on Latent Class Regression Analysis. The method enables us (i) to summarize in a single indicator statement multiple facets evaluated by students through a survey questionnaire and (ii) to control the variability in the evaluations that is mainly attributable to the characteristics (often referred as the Potential Confounding Factors) of the evaluators (students) rather than to real differences in the performances of the degree programs under evaluation. A simulation study is implemented in order to test the method and assess its potential when the composition of the degree programs as regards to students’ characteristics is sensibly different between one another. Results suggest that when the evaluations are strongly affected by the students’ covariates, the assessment based on the value of an unadjusted indicator can lead to bias and unreliable conclusions about the differences in performance. An application to real data is also provided.

Keywords

University evaluation Adjusted indicators Latent class regression analysis Potential confounding factors 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agresti A (2002) Categorical data analysis. Wiley-Interscience, HobokenzbMATHCrossRefGoogle Scholar
  2. Bartholomew DJ (1998) Scaling unobservable constructs in social science. Appl Stat 47: 1–13zbMATHGoogle Scholar
  3. Bartholomew DJ, Steele F, Moustaki I, Galbraith JI (2002) The analysis and interpretation of multivariate analysis for social scientists. Chapman & Hall, Boca RatonzbMATHGoogle Scholar
  4. Bayol MP, de la Foye A, Tellier C, Tenenhaus M (2000) Use of PLS path modelling to estimate the European consumer satisfaction index (ECSI) model. Stat Appl 12(3): 361–375Google Scholar
  5. Bird SM, Cox D, Goldstein H, Holt T, Smith PC (2005) Performance indicators: good, bad and ugly. J R Stat Soc A 168(1): 1–27MathSciNetzbMATHCrossRefGoogle Scholar
  6. Capursi V, Porcu M (2001) La didattica universitaria valutata dagli studenti: un indicatore basato su misure di distanza fra distribuzioni di giudizi. In: Atti Convegno Intermedio della Società Italiana di Statistica ‘Processi e Metodi Statistici di Valutazione’, Roma 4–6 giugno 2001. Società Italiana di StatisticaGoogle Scholar
  7. Cox DR, Fitzpatrick R, Fletcher AE, Gore SM, Spiegelhalter DJ, Jones DR (1992) Quality of life assessment: can we keep it simple?. J R Stat Soc A 155: 353–393CrossRefGoogle Scholar
  8. Draper D, Gittoes M (2004) Statistical analysis of performance indicators in UK higher education. J R Stat Soc A 167(3): 449–474MathSciNetCrossRefGoogle Scholar
  9. Goldstein H, Spiegelhalter DJ (1996) League tables and their limitations: statistical issues in comparisons of istitutional performance. J R Stat Soc A 159: 385–443CrossRefGoogle Scholar
  10. Grilli L, Rampichini C (2007) Multilevel factor models for ordinal variables. Struct Equ Model A Multidiscip J 14(1): 1–25MathSciNetGoogle Scholar
  11. Hagenaars JA, McCutcheon AL (2002) Applied latent class analysis. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  12. Lanza ST, Collins LM, Lemmon DR, Schafer JL (2007) PROC LCA: a SAS procedure for latent class analysis. Struct Equ Model A Multidiscip J 14(4): 671–694MathSciNetCrossRefGoogle Scholar
  13. Leckie G, Goldstein H (2009) The limitation of using school league tables to inform school choice. J R Stat Soc A 172(4): 835–851MathSciNetCrossRefGoogle Scholar
  14. Leti G (1983) Statistica Descrittiva. Il Mulino, BolognaGoogle Scholar
  15. Linzer DA, Lewis J (2010) poLCA: polytomous variable latent class analysis. R package version 1.2. http://userwww.service.emory.edu/~dlinzer/poLCA/
  16. Moustaki I, Knott D (2000) Generalized latent trait models. Psychometrika 65: 391–411MathSciNetCrossRefGoogle Scholar
  17. Quatto P (2011) Descriptive analysis of student ratings. Book of short abstracts IES 2011. (http://ies2011.ds.unifi.it/BookOfAbstractsIES2011.pdf)
  18. Rampichini C, Grilli L, Petrucci A (2004) Analysis of university course evaluations: from descriptive measures to multilevel models. Stat Methods Appl 13(3): 357–371MathSciNetzbMATHGoogle Scholar
  19. Skrondal A, Rabe-Hesketh S (2004) Generalized latent variables modeling. Chapman & Hall, Boca RatonCrossRefGoogle Scholar
  20. Sulis I, Porcu M (2011) Assessing the quality of the management of degree programs by latent class analysis. In: Attanasio M, Capursi V (eds) Statistical methods for the evaluation of university systems, series contributions to statistics. Springer, Berlin, pp 161–172CrossRefGoogle Scholar
  21. Vermunt JK (2003) Multilevel latent class models. Sociol Methodol 33: 213–234CrossRefGoogle Scholar
  22. Vermunt JK (2008) Latent class and finite mixture models for multilevel data sets. Stat Methods Med Res 17: 33–51MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Dipartimento Scienze Sociali e delle IstituzioniUniversità di CagliariCagliariItaly

Personalised recommendations