Social Indicators Research

, Volume 147, Issue 2, pp 361–381 | Cite as

A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching

  • Marco Guerra
  • Francesca BassiEmail author
  • José G. Dias
Original Research


This paper describes a multi-indicator latent growth mixture model built on the data collected by a large Italian university to track students’ satisfaction over time. The analysis of the data involves two steps: first, a pre-processing of data selects the items to be part of the synthetic indicator that measures students’ satisfaction; the second step then retrieves heterogeneity that allows the identification of a clustering structure with a group of university courses (outliers) which underperform in terms of students’ satisfaction over time. Regression components of the model identify courses in need of further improvement and that are prone to receiving low classifications from students. Results show that it is possible to identify a large group of didactic activities with a high satisfaction level that stays constant over time; there is also a small group of problematic didactic activities with low satisfaction that decreases over the period under analysis.


Higher education Quality of didactics Latent growth mixture models Outlier detection Synthetic indicator Data science 



The authors would like to thank the editor and three anonymous reviewers for their constructive comments, which helped us to improve the manuscript. This work was funded by the Portuguese Foundation for Science and Technology (Grant UID/GES/00315/2013 and UID/GES/00315/2019) and by Grant BIRD162088/16 financed by the University of Padua for the project entitled “Advances in Multilevel and Longitudinal Modelling”.


  1. Akaike, H. (1974). A new look at statistical-model identification. IEEE Transactions on Automatic Control, AC,19(6), 716–723.Google Scholar
  2. Alves, B. C., & Dias, J. G. (2015). Survival mixture models in behavioral scoring. Expert Systems with Applications,42(2), 3902–3910.Google Scholar
  3. Bacci, S., Bartolucci, F., Grilli, L., & Rampichini, C. (2017). Evaluation of student performance through a multidimensional finite mixture IRT model. Multivariate Behavioral Research,52(6), 732–746.Google Scholar
  4. Baker, R. S. (2014). Educational data mining: An advance for intelligent systems in education. IEEE Intelligent Systems,29(3), 78–82.Google Scholar
  5. Bartholomew, D. J., & Knott, M. (1999). Latent variable models and factor analysis. London: Arnold.Google Scholar
  6. Bartolucci, F., Bacci, S., & Gnaldi, M. (2015). Statistical analysis of questionnaires: A unified approach based on R and stata. London: Chapman and Hall/CRC.Google Scholar
  7. Bartolucci, F., Farcomeni, A., & Pennoni, F. (2012). Latent Markov models for longitudinal data. Boca Raton: Chapman & Hall/CRC.Google Scholar
  8. Bassi, F. (2010). Experiential goods and customer satisfaction: An application to movies. Quality Technology & Quantitative Management,7(1), 51–67.Google Scholar
  9. Bassi, F. (2016). Dynamic segmentation with growth mixture models. Advances in Data Analysis and Classification,10(2), 263–279.Google Scholar
  10. Bassi, F., Clerici, R., & Aquario, D. (2017). Students’ evaluation of teaching at a large Italian university: measurement scale validation. Electronic Journal of Applied Statistical Analysis,10(1), 93–117.Google Scholar
  11. Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970). A maximization technique occurring in statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics,41(1), 164–171.Google Scholar
  12. Biemer, P. P. (2011). Latent class analysis of survey error. Hoboken: Wiley.Google Scholar
  13. Bishop, J., Geiser, C., & Cole, D. A. (2015). Modeling latent growth with multiple indicators: A comparison of three approaches. Psychological Methods,20(1), 43–62.Google Scholar
  14. Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken: Wiley.Google Scholar
  15. Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Retrieved from Science Open.Google Scholar
  16. Braga, M., Paccagnella, M., & Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review,41(C), 71–88.Google Scholar
  17. Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research,16(1), 64–73.Google Scholar
  18. Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 311–359). New York: Plenum Press.Google Scholar
  19. Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Hoboken: Wiley.Google Scholar
  20. Connel, A. M., & Frey, A. A. (2006). Growth mixture modelling in developmental psychology: overview and demonstration of heterogeneity in developmental trajectories of adolescent antisocial behaviour. Infant and Child Development,15(6), 609–621.Google Scholar
  21. Dalla Zuanna, G., Bassi F., Clerici, R., Paccagnella, O., Paggiaro, A., Aquario D., Mazzuco C., Martinoia, S., Stocco, C., & Pierobon, S. (2015). Tools for teaching assessment at Padua University: role, development and validation. PRODID Project (Teacher professional development and academic educational innovation) (Report of Research Unit n.3). Padua: Department of Statistical Sciences, University of Padua.Google Scholar
  22. Dayton, C. M., & Macready, G. B. (1988). Concomitant-variable latent class models. Journal of the American Statistical Association,83(401), 173–178.Google Scholar
  23. de Angelis, L., & Dias, J. G. (2014). Mining categorical sequences from data using a hybrid clustering method. European Journal of Operational Research,234(3), 720–730.Google Scholar
  24. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B,39, 1–38.Google Scholar
  25. Dias, J. G. (2006). Latent class analysis and model selection. In M. Spiliopoulou, R. Kruse, C. Borgelt, A. Nürnberger, & W. Gaul (Eds.), From data and information analysis to knowledge engineering (pp. 95–102). Berlin: Springer.Google Scholar
  26. Dias, J. G. (2007). Model selection criteria for model-based clustering of categorical time series data: A Monte Carlo study. In R. Decker & H. J. Lenz (Eds.), Advances in data analysis (p. 23). Berlin: Springer.Google Scholar
  27. Dias, J. G., & Ramos, S. B. (2014). The aftermath of the subprime crisis—a clustering analysis of world banking sector. Review of Quantitative Finance and Accounting,42(2), 293–308.Google Scholar
  28. Dias, J. G., & Vermunt, J. K. (2007). Latent class modeling of website users’ search patterns: implications for online market segmentation. Journal of Retailing and Consumer Services,14(6), 359–368.Google Scholar
  29. Dias, J. G., Vermunt, J. K., & Ramos, S. B. (2015). Clustering financial time series: New insights from an extended hidden Markov model. European Journal of Operational Research,243(3), 852–864.Google Scholar
  30. Esling, P., & Agon, C. (2012). Time series data analysis. ACM Computing Survey,45(19), 12.Google Scholar
  31. European University Association. (2016). Quality culture in European Universities: A bottom up approach. Brussels: EUA.Google Scholar
  32. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods,4(3), 272–299.Google Scholar
  33. Formann, A. K. (1992). Linear logistic latent class analysis for polytomous data. Journal of the American Statistical Association,87(418), 476–486.Google Scholar
  34. Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. New York: Springer.Google Scholar
  35. Goodman, L. A. (1974). Exploratory latent factor analysis using both identifiable and unidentifiable models. Biometrika,61(2), 215–231.Google Scholar
  36. Grilli, L., & Varriale, R. (2014). Specifying measurement error correlations in latent growth curve models with multiple indicators. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences,10(4), 117–125.Google Scholar
  37. Grimm, K. J., & Widaman, K. F. (2010). Residual structures in latent growth curve modelling. Structural Equation Modeling,17(3), 424–442.Google Scholar
  38. Hagenaars, J. A., & McCutcheon, A. L. (2002). Applied latent class analysis. Dordrecht: Kluwer Academic Publishers.Google Scholar
  39. Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education,18(1), 9–34.Google Scholar
  40. Hornstein, H. A. (2017). Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education,4(1), 1–8.Google Scholar
  41. La Rocca, M., Parrella, L., Primerano, I., Sulis, I., & Vitale, M. P. (2017). An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models. Quality & Quantity,51(2), 675–691.Google Scholar
  42. Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.Google Scholar
  43. Masserini, L., Liberati, C., & Mariani, P. (2017). Quality service in banking: a longitudinal approach. Quality & Quantity,51(2), 509–523.Google Scholar
  44. McArdle, J. J., & Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Development,58, 110–133.Google Scholar
  45. McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.Google Scholar
  46. Meggiolaro, S., Giraldo, A., & Clerici, R. (2017). A multilevel competing risks model for analysis of university students’ careers in Italy. Studies in Higher Education,42(7), 1259–1274.Google Scholar
  47. Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika,55(1), 107–122.Google Scholar
  48. Murias, P., de Miguel, J. C., & Rodríguez, D. (2008). A composite indicator for university quality assessment: The case of Spanish higher education system. Social Indicators Research,89(1), 129–146.Google Scholar
  49. Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Newbury Park: Sage Publications.Google Scholar
  50. Muthén, B., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics,55(2), 463–469.Google Scholar
  51. Nagin, D. S., & Land, K. C. (1993). Age, criminal, careers, and population heterogeneity—Specification and estimation of a nonparametric, mixed Poisson model. Criminology,31(3), 327–362.Google Scholar
  52. Nixon, E., Scullion, R., & Hearn, R. (2018). Her majesty the student: Marketised higher education and the narcissistic (dis)satisfactions of the student-consumer. Studies in Higher Education,43(6), 927–943.Google Scholar
  53. Pennoni, F., & Romeo, I. (2017). Latent Markov and growth mixture models for ordinal individual responses with covariates: a comparison. Statistical Analysis and Data Mining: The ASA Data Science Journal,10(1), 29–39.Google Scholar
  54. Ramaswamy, V., Desarbo, W. S., Reibstein, D. J., & Robinson, W. T. (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science,12(1), 103–124.Google Scholar
  55. Rampichini, C., Grilli, L., & Petrucci, A. (2004). Analysis of university course evaluations: from descriptive measures to multilevel models. Statistical Methods and Applications,13, 357–373.Google Scholar
  56. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics,6, 461–464.Google Scholar
  57. Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: the state of the art. Review of Educational Research,83(4), 598–642.Google Scholar
  58. Stroebe, W. (2016). Why good teaching evaluations may reward bad teaching: on grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science,1(86), 800–816.Google Scholar
  59. Svinicki, M., & McKeachie, W. J. (2013). McKeachie’s teaching tips: Strategies, research, and theory for College and University teachers (13th ed.). Belmont: Wadsworth.Google Scholar
  60. Theall, M., & Franklin, J. (Eds.). (2007). Student ratings of instruction: Issues for improving practice: New directions for teaching and learning. San Francisco: Jossey-Bass.Google Scholar
  61. Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation,54, 22–42.Google Scholar
  62. van de Pol, F., & Langeheine, R. (1990). Mixed Markov latent class models. Sociological Methodology,20, 213–247.Google Scholar
  63. van de Pol, F., & Mannan, H. (2002). Questions of a novice in latent Markov modelling. Methods of Psychological Research Online,7(2), 1–18.Google Scholar
  64. van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer.Google Scholar
  65. Vermunt, J. K., Langeheine, R., & Bockenholt, U. (1999). Discrete-time discrete-state latent Markov models with time-constant and time-varying covariates. Journal of Educational and Behavioral Statistics,24(2), 179–207.Google Scholar
  66. Wedel, M., & Kamakura, W. A. (2000). Market segmentation: Conceptual and methodological foundations. Dordrecht: Kluwer Academic Publishers.Google Scholar
  67. Weng, L.-J. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement,64(6), 956–972.Google Scholar
  68. Wiggins, L. M. (1955). Mathematical models for the interpretation of attitude and behavior change. Columbia: Columbia University.Google Scholar
  69. Wu, H., & Leung, S.-O. (2017). Can Likert scales be treated as interval scales? A simulation study, Journal of Social Service Research,43(4), 527–532.Google Scholar
  70. Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education,12, 55–76.Google Scholar
  71. Zucchini, W., MacDonald, I. L., & Langrock, R. (2016). Hidden Markov models for time series: An introduction using R. Boca Raton: Chapman and Hall/CRC.Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Statistical SciencesUniversity of PaduaPaduaItaly
  2. 2.Business Research Unit (BRU-IUL)Instituto Universitário de Lisboa (ISCTE-IUL)LisbonPortugal

Personalised recommendations