Skip to main content
Log in

A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching

Social Indicators Research Aims and scope Submit manuscript

Abstract

This paper describes a multi-indicator latent growth mixture model built on the data collected by a large Italian university to track students’ satisfaction over time. The analysis of the data involves two steps: first, a pre-processing of data selects the items to be part of the synthetic indicator that measures students’ satisfaction; the second step then retrieves heterogeneity that allows the identification of a clustering structure with a group of university courses (outliers) which underperform in terms of students’ satisfaction over time. Regression components of the model identify courses in need of further improvement and that are prone to receiving low classifications from students. Results show that it is possible to identify a large group of didactic activities with a high satisfaction level that stays constant over time; there is also a small group of problematic didactic activities with low satisfaction that decreases over the period under analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. We use course and didactic activity as synonymous. To be more precise, didactic activity is understood as the course-teacher pair.

  2. Since we have longitudinal data for only three consecutive waves, only a linear latent trajectory may be specified.

  3. Exploratory factor analysis on the data referring to the three academic years gave similar results; therefore, we decided to report only for the most recent academic year 2014–2015. Extracting factors with principal-components factoring gave the same solution; on the other hand, extracting the factors with methods such as iterative maximum likelihood did not provide good results as a Heywood case was encountered (Fabrigar et al. 1999). KMO index is equal to 0.956.

  4. Three-component mixture models always perform worse than the two-component mixture model (based on BIC).

  5. The number of credits has been excluded from the potential covariates since it is highly correlated with the number of hours.

  6. The classification is based on the estimates given by Eq. (7) and assigned to the latent class with maximum probability (Bayes’ optimal rule). The relative entropy index, which varies between 0 (min) and 1 (max), is 0.908. Thus, we have a good level of separation of the mixture components (Ramaswamy et al. 1993).

References

  • Akaike, H. (1974). A new look at statistical-model identification. IEEE Transactions on Automatic Control, AC, 19(6), 716–723.

    Article  Google Scholar 

  • Alves, B. C., & Dias, J. G. (2015). Survival mixture models in behavioral scoring. Expert Systems with Applications, 42(2), 3902–3910.

    Article  Google Scholar 

  • Bacci, S., Bartolucci, F., Grilli, L., & Rampichini, C. (2017). Evaluation of student performance through a multidimensional finite mixture IRT model. Multivariate Behavioral Research, 52(6), 732–746.

    Article  Google Scholar 

  • Baker, R. S. (2014). Educational data mining: An advance for intelligent systems in education. IEEE Intelligent Systems, 29(3), 78–82.

    Article  Google Scholar 

  • Bartholomew, D. J., & Knott, M. (1999). Latent variable models and factor analysis. London: Arnold.

    Google Scholar 

  • Bartolucci, F., Bacci, S., & Gnaldi, M. (2015). Statistical analysis of questionnaires: A unified approach based on R and stata. London: Chapman and Hall/CRC.

    Book  Google Scholar 

  • Bartolucci, F., Farcomeni, A., & Pennoni, F. (2012). Latent Markov models for longitudinal data. Boca Raton: Chapman & Hall/CRC.

    Book  Google Scholar 

  • Bassi, F. (2010). Experiential goods and customer satisfaction: An application to movies. Quality Technology & Quantitative Management, 7(1), 51–67.

    Article  Google Scholar 

  • Bassi, F. (2016). Dynamic segmentation with growth mixture models. Advances in Data Analysis and Classification, 10(2), 263–279.

    Article  Google Scholar 

  • Bassi, F., Clerici, R., & Aquario, D. (2017). Students’ evaluation of teaching at a large Italian university: measurement scale validation. Electronic Journal of Applied Statistical Analysis, 10(1), 93–117.

    Google Scholar 

  • Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970). A maximization technique occurring in statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41(1), 164–171.

    Article  Google Scholar 

  • Biemer, P. P. (2011). Latent class analysis of survey error. Hoboken: Wiley.

    Google Scholar 

  • Bishop, J., Geiser, C., & Cole, D. A. (2015). Modeling latent growth with multiple indicators: A comparison of three approaches. Psychological Methods, 20(1), 43–62.

    Article  Google Scholar 

  • Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken: Wiley.

    Google Scholar 

  • Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Retrieved from Science Open.

  • Braga, M., Paccagnella, M., & Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review, 41(C), 71–88.

    Article  Google Scholar 

  • Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73.

    Article  Google Scholar 

  • Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 311–359). New York: Plenum Press.

    Chapter  Google Scholar 

  • Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Hoboken: Wiley.

    Google Scholar 

  • Connel, A. M., & Frey, A. A. (2006). Growth mixture modelling in developmental psychology: overview and demonstration of heterogeneity in developmental trajectories of adolescent antisocial behaviour. Infant and Child Development, 15(6), 609–621.

    Article  Google Scholar 

  • Dalla Zuanna, G., Bassi F., Clerici, R., Paccagnella, O., Paggiaro, A., Aquario D., Mazzuco C., Martinoia, S., Stocco, C., & Pierobon, S. (2015). Tools for teaching assessment at Padua University: role, development and validation. PRODID Project (Teacher professional development and academic educational innovation) (Report of Research Unit n.3). Padua: Department of Statistical Sciences, University of Padua.

  • Dayton, C. M., & Macready, G. B. (1988). Concomitant-variable latent class models. Journal of the American Statistical Association, 83(401), 173–178.

    Article  Google Scholar 

  • de Angelis, L., & Dias, J. G. (2014). Mining categorical sequences from data using a hybrid clustering method. European Journal of Operational Research, 234(3), 720–730.

    Article  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B, 39, 1–38.

    Google Scholar 

  • Dias, J. G. (2006). Latent class analysis and model selection. In M. Spiliopoulou, R. Kruse, C. Borgelt, A. Nürnberger, & W. Gaul (Eds.), From data and information analysis to knowledge engineering (pp. 95–102). Berlin: Springer.

    Chapter  Google Scholar 

  • Dias, J. G. (2007). Model selection criteria for model-based clustering of categorical time series data: A Monte Carlo study. In R. Decker & H. J. Lenz (Eds.), Advances in data analysis (p. 23). Berlin: Springer.

    Chapter  Google Scholar 

  • Dias, J. G., & Ramos, S. B. (2014). The aftermath of the subprime crisis—a clustering analysis of world banking sector. Review of Quantitative Finance and Accounting, 42(2), 293–308.

    Article  Google Scholar 

  • Dias, J. G., & Vermunt, J. K. (2007). Latent class modeling of website users’ search patterns: implications for online market segmentation. Journal of Retailing and Consumer Services, 14(6), 359–368.

    Article  Google Scholar 

  • Dias, J. G., Vermunt, J. K., & Ramos, S. B. (2015). Clustering financial time series: New insights from an extended hidden Markov model. European Journal of Operational Research, 243(3), 852–864.

    Article  Google Scholar 

  • Esling, P., & Agon, C. (2012). Time series data analysis. ACM Computing Survey, 45(19), 12.

    Google Scholar 

  • European University Association. (2016). Quality culture in European Universities: A bottom up approach. Brussels: EUA.

    Google Scholar 

  • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.

    Article  Google Scholar 

  • Formann, A. K. (1992). Linear logistic latent class analysis for polytomous data. Journal of the American Statistical Association, 87(418), 476–486.

    Article  Google Scholar 

  • Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. New York: Springer.

    Google Scholar 

  • Goodman, L. A. (1974). Exploratory latent factor analysis using both identifiable and unidentifiable models. Biometrika, 61(2), 215–231.

    Article  Google Scholar 

  • Grilli, L., & Varriale, R. (2014). Specifying measurement error correlations in latent growth curve models with multiple indicators. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 10(4), 117–125.

    Article  Google Scholar 

  • Grimm, K. J., & Widaman, K. F. (2010). Residual structures in latent growth curve modelling. Structural Equation Modeling, 17(3), 424–442.

    Article  Google Scholar 

  • Hagenaars, J. A., & McCutcheon, A. L. (2002). Applied latent class analysis. Dordrecht: Kluwer Academic Publishers.

    Book  Google Scholar 

  • Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18(1), 9–34.

    Article  Google Scholar 

  • Hornstein, H. A. (2017). Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education, 4(1), 1–8.

    Article  Google Scholar 

  • La Rocca, M., Parrella, L., Primerano, I., Sulis, I., & Vitale, M. P. (2017). An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models. Quality & Quantity, 51(2), 675–691.

    Article  Google Scholar 

  • Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.

    Google Scholar 

  • Masserini, L., Liberati, C., & Mariani, P. (2017). Quality service in banking: a longitudinal approach. Quality & Quantity, 51(2), 509–523.

    Article  Google Scholar 

  • McArdle, J. J., & Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Development, 58, 110–133.

    Article  Google Scholar 

  • McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.

    Book  Google Scholar 

  • Meggiolaro, S., Giraldo, A., & Clerici, R. (2017). A multilevel competing risks model for analysis of university students’ careers in Italy. Studies in Higher Education, 42(7), 1259–1274.

    Article  Google Scholar 

  • Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55(1), 107–122.

    Article  Google Scholar 

  • Murias, P., de Miguel, J. C., & Rodríguez, D. (2008). A composite indicator for university quality assessment: The case of Spanish higher education system. Social Indicators Research, 89(1), 129–146.

    Article  Google Scholar 

  • Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Newbury Park: Sage Publications.

    Google Scholar 

  • Muthén, B., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics, 55(2), 463–469.

    Article  Google Scholar 

  • Nagin, D. S., & Land, K. C. (1993). Age, criminal, careers, and population heterogeneity—Specification and estimation of a nonparametric, mixed Poisson model. Criminology, 31(3), 327–362.

    Article  Google Scholar 

  • Nixon, E., Scullion, R., & Hearn, R. (2018). Her majesty the student: Marketised higher education and the narcissistic (dis)satisfactions of the student-consumer. Studies in Higher Education, 43(6), 927–943.

    Article  Google Scholar 

  • Pennoni, F., & Romeo, I. (2017). Latent Markov and growth mixture models for ordinal individual responses with covariates: a comparison. Statistical Analysis and Data Mining: The ASA Data Science Journal, 10(1), 29–39.

    Article  Google Scholar 

  • Ramaswamy, V., Desarbo, W. S., Reibstein, D. J., & Robinson, W. T. (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12(1), 103–124.

    Article  Google Scholar 

  • Rampichini, C., Grilli, L., & Petrucci, A. (2004). Analysis of university course evaluations: from descriptive measures to multilevel models. Statistical Methods and Applications, 13, 357–373.

    Article  Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.

    Article  Google Scholar 

  • Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: the state of the art. Review of Educational Research, 83(4), 598–642.

    Article  Google Scholar 

  • Stroebe, W. (2016). Why good teaching evaluations may reward bad teaching: on grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 1(86), 800–816.

    Article  Google Scholar 

  • Svinicki, M., & McKeachie, W. J. (2013). McKeachie’s teaching tips: Strategies, research, and theory for College and University teachers (13th ed.). Belmont: Wadsworth.

    Google Scholar 

  • Theall, M., & Franklin, J. (Eds.). (2007). Student ratings of instruction: Issues for improving practice: New directions for teaching and learning. San Francisco: Jossey-Bass.

    Google Scholar 

  • Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42.

    Article  Google Scholar 

  • van de Pol, F., & Langeheine, R. (1990). Mixed Markov latent class models. Sociological Methodology, 20, 213–247.

    Article  Google Scholar 

  • van de Pol, F., & Mannan, H. (2002). Questions of a novice in latent Markov modelling. Methods of Psychological Research Online, 7(2), 1–18.

    Google Scholar 

  • van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer.

    Book  Google Scholar 

  • Vermunt, J. K., Langeheine, R., & Bockenholt, U. (1999). Discrete-time discrete-state latent Markov models with time-constant and time-varying covariates. Journal of Educational and Behavioral Statistics, 24(2), 179–207.

    Article  Google Scholar 

  • Wedel, M., & Kamakura, W. A. (2000). Market segmentation: Conceptual and methodological foundations. Dordrecht: Kluwer Academic Publishers.

    Book  Google Scholar 

  • Weng, L.-J. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64(6), 956–972.

    Article  Google Scholar 

  • Wiggins, L. M. (1955). Mathematical models for the interpretation of attitude and behavior change. Columbia: Columbia University.

    Google Scholar 

  • Wu, H., & Leung, S.-O. (2017). Can Likert scales be treated as interval scales? A simulation study, Journal of Social Service Research, 43(4), 527–532.

    Google Scholar 

  • Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education, 12, 55–76.

    Article  Google Scholar 

  • Zucchini, W., MacDonald, I. L., & Langrock, R. (2016). Hidden Markov models for time series: An introduction using R. Boca Raton: Chapman and Hall/CRC.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editor and three anonymous reviewers for their constructive comments, which helped us to improve the manuscript. This work was funded by the Portuguese Foundation for Science and Technology (Grant UID/GES/00315/2013 and UID/GES/00315/2019) and by Grant BIRD162088/16 financed by the University of Padua for the project entitled “Advances in Multilevel and Longitudinal Modelling”.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Francesca Bassi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guerra, M., Bassi, F. & Dias, J.G. A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching. Soc Indic Res 147, 361–381 (2020). https://doi.org/10.1007/s11205-019-02169-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-019-02169-x

Keywords

Navigation