Abstract
This paper describes a multi-indicator latent growth mixture model built on the data collected by a large Italian university to track students’ satisfaction over time. The analysis of the data involves two steps: first, a pre-processing of data selects the items to be part of the synthetic indicator that measures students’ satisfaction; the second step then retrieves heterogeneity that allows the identification of a clustering structure with a group of university courses (outliers) which underperform in terms of students’ satisfaction over time. Regression components of the model identify courses in need of further improvement and that are prone to receiving low classifications from students. Results show that it is possible to identify a large group of didactic activities with a high satisfaction level that stays constant over time; there is also a small group of problematic didactic activities with low satisfaction that decreases over the period under analysis.
Similar content being viewed by others
Notes
We use course and didactic activity as synonymous. To be more precise, didactic activity is understood as the course-teacher pair.
Since we have longitudinal data for only three consecutive waves, only a linear latent trajectory may be specified.
Exploratory factor analysis on the data referring to the three academic years gave similar results; therefore, we decided to report only for the most recent academic year 2014–2015. Extracting factors with principal-components factoring gave the same solution; on the other hand, extracting the factors with methods such as iterative maximum likelihood did not provide good results as a Heywood case was encountered (Fabrigar et al. 1999). KMO index is equal to 0.956.
Three-component mixture models always perform worse than the two-component mixture model (based on BIC).
The number of credits has been excluded from the potential covariates since it is highly correlated with the number of hours.
The classification is based on the estimates given by Eq. (7) and assigned to the latent class with maximum probability (Bayes’ optimal rule). The relative entropy index, which varies between 0 (min) and 1 (max), is 0.908. Thus, we have a good level of separation of the mixture components (Ramaswamy et al. 1993).
References
Akaike, H. (1974). A new look at statistical-model identification. IEEE Transactions on Automatic Control, AC, 19(6), 716–723.
Alves, B. C., & Dias, J. G. (2015). Survival mixture models in behavioral scoring. Expert Systems with Applications, 42(2), 3902–3910.
Bacci, S., Bartolucci, F., Grilli, L., & Rampichini, C. (2017). Evaluation of student performance through a multidimensional finite mixture IRT model. Multivariate Behavioral Research, 52(6), 732–746.
Baker, R. S. (2014). Educational data mining: An advance for intelligent systems in education. IEEE Intelligent Systems, 29(3), 78–82.
Bartholomew, D. J., & Knott, M. (1999). Latent variable models and factor analysis. London: Arnold.
Bartolucci, F., Bacci, S., & Gnaldi, M. (2015). Statistical analysis of questionnaires: A unified approach based on R and stata. London: Chapman and Hall/CRC.
Bartolucci, F., Farcomeni, A., & Pennoni, F. (2012). Latent Markov models for longitudinal data. Boca Raton: Chapman & Hall/CRC.
Bassi, F. (2010). Experiential goods and customer satisfaction: An application to movies. Quality Technology & Quantitative Management, 7(1), 51–67.
Bassi, F. (2016). Dynamic segmentation with growth mixture models. Advances in Data Analysis and Classification, 10(2), 263–279.
Bassi, F., Clerici, R., & Aquario, D. (2017). Students’ evaluation of teaching at a large Italian university: measurement scale validation. Electronic Journal of Applied Statistical Analysis, 10(1), 93–117.
Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970). A maximization technique occurring in statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41(1), 164–171.
Biemer, P. P. (2011). Latent class analysis of survey error. Hoboken: Wiley.
Bishop, J., Geiser, C., & Cole, D. A. (2015). Modeling latent growth with multiple indicators: A comparison of three approaches. Psychological Methods, 20(1), 43–62.
Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken: Wiley.
Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Retrieved from Science Open.
Braga, M., Paccagnella, M., & Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review, 41(C), 71–88.
Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73.
Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 311–359). New York: Plenum Press.
Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Hoboken: Wiley.
Connel, A. M., & Frey, A. A. (2006). Growth mixture modelling in developmental psychology: overview and demonstration of heterogeneity in developmental trajectories of adolescent antisocial behaviour. Infant and Child Development, 15(6), 609–621.
Dalla Zuanna, G., Bassi F., Clerici, R., Paccagnella, O., Paggiaro, A., Aquario D., Mazzuco C., Martinoia, S., Stocco, C., & Pierobon, S. (2015). Tools for teaching assessment at Padua University: role, development and validation. PRODID Project (Teacher professional development and academic educational innovation) (Report of Research Unit n.3). Padua: Department of Statistical Sciences, University of Padua.
Dayton, C. M., & Macready, G. B. (1988). Concomitant-variable latent class models. Journal of the American Statistical Association, 83(401), 173–178.
de Angelis, L., & Dias, J. G. (2014). Mining categorical sequences from data using a hybrid clustering method. European Journal of Operational Research, 234(3), 720–730.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B, 39, 1–38.
Dias, J. G. (2006). Latent class analysis and model selection. In M. Spiliopoulou, R. Kruse, C. Borgelt, A. Nürnberger, & W. Gaul (Eds.), From data and information analysis to knowledge engineering (pp. 95–102). Berlin: Springer.
Dias, J. G. (2007). Model selection criteria for model-based clustering of categorical time series data: A Monte Carlo study. In R. Decker & H. J. Lenz (Eds.), Advances in data analysis (p. 23). Berlin: Springer.
Dias, J. G., & Ramos, S. B. (2014). The aftermath of the subprime crisis—a clustering analysis of world banking sector. Review of Quantitative Finance and Accounting, 42(2), 293–308.
Dias, J. G., & Vermunt, J. K. (2007). Latent class modeling of website users’ search patterns: implications for online market segmentation. Journal of Retailing and Consumer Services, 14(6), 359–368.
Dias, J. G., Vermunt, J. K., & Ramos, S. B. (2015). Clustering financial time series: New insights from an extended hidden Markov model. European Journal of Operational Research, 243(3), 852–864.
Esling, P., & Agon, C. (2012). Time series data analysis. ACM Computing Survey, 45(19), 12.
European University Association. (2016). Quality culture in European Universities: A bottom up approach. Brussels: EUA.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299.
Formann, A. K. (1992). Linear logistic latent class analysis for polytomous data. Journal of the American Statistical Association, 87(418), 476–486.
Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. New York: Springer.
Goodman, L. A. (1974). Exploratory latent factor analysis using both identifiable and unidentifiable models. Biometrika, 61(2), 215–231.
Grilli, L., & Varriale, R. (2014). Specifying measurement error correlations in latent growth curve models with multiple indicators. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 10(4), 117–125.
Grimm, K. J., & Widaman, K. F. (2010). Residual structures in latent growth curve modelling. Structural Equation Modeling, 17(3), 424–442.
Hagenaars, J. A., & McCutcheon, A. L. (2002). Applied latent class analysis. Dordrecht: Kluwer Academic Publishers.
Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18(1), 9–34.
Hornstein, H. A. (2017). Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education, 4(1), 1–8.
La Rocca, M., Parrella, L., Primerano, I., Sulis, I., & Vitale, M. P. (2017). An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models. Quality & Quantity, 51(2), 675–691.
Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.
Masserini, L., Liberati, C., & Mariani, P. (2017). Quality service in banking: a longitudinal approach. Quality & Quantity, 51(2), 509–523.
McArdle, J. J., & Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Development, 58, 110–133.
McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.
Meggiolaro, S., Giraldo, A., & Clerici, R. (2017). A multilevel competing risks model for analysis of university students’ careers in Italy. Studies in Higher Education, 42(7), 1259–1274.
Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55(1), 107–122.
Murias, P., de Miguel, J. C., & Rodríguez, D. (2008). A composite indicator for university quality assessment: The case of Spanish higher education system. Social Indicators Research, 89(1), 129–146.
Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Newbury Park: Sage Publications.
Muthén, B., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics, 55(2), 463–469.
Nagin, D. S., & Land, K. C. (1993). Age, criminal, careers, and population heterogeneity—Specification and estimation of a nonparametric, mixed Poisson model. Criminology, 31(3), 327–362.
Nixon, E., Scullion, R., & Hearn, R. (2018). Her majesty the student: Marketised higher education and the narcissistic (dis)satisfactions of the student-consumer. Studies in Higher Education, 43(6), 927–943.
Pennoni, F., & Romeo, I. (2017). Latent Markov and growth mixture models for ordinal individual responses with covariates: a comparison. Statistical Analysis and Data Mining: The ASA Data Science Journal, 10(1), 29–39.
Ramaswamy, V., Desarbo, W. S., Reibstein, D. J., & Robinson, W. T. (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12(1), 103–124.
Rampichini, C., Grilli, L., & Petrucci, A. (2004). Analysis of university course evaluations: from descriptive measures to multilevel models. Statistical Methods and Applications, 13, 357–373.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: the state of the art. Review of Educational Research, 83(4), 598–642.
Stroebe, W. (2016). Why good teaching evaluations may reward bad teaching: on grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 1(86), 800–816.
Svinicki, M., & McKeachie, W. J. (2013). McKeachie’s teaching tips: Strategies, research, and theory for College and University teachers (13th ed.). Belmont: Wadsworth.
Theall, M., & Franklin, J. (Eds.). (2007). Student ratings of instruction: Issues for improving practice: New directions for teaching and learning. San Francisco: Jossey-Bass.
Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42.
van de Pol, F., & Langeheine, R. (1990). Mixed Markov latent class models. Sociological Methodology, 20, 213–247.
van de Pol, F., & Mannan, H. (2002). Questions of a novice in latent Markov modelling. Methods of Psychological Research Online, 7(2), 1–18.
van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer.
Vermunt, J. K., Langeheine, R., & Bockenholt, U. (1999). Discrete-time discrete-state latent Markov models with time-constant and time-varying covariates. Journal of Educational and Behavioral Statistics, 24(2), 179–207.
Wedel, M., & Kamakura, W. A. (2000). Market segmentation: Conceptual and methodological foundations. Dordrecht: Kluwer Academic Publishers.
Weng, L.-J. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educational and Psychological Measurement, 64(6), 956–972.
Wiggins, L. M. (1955). Mathematical models for the interpretation of attitude and behavior change. Columbia: Columbia University.
Wu, H., & Leung, S.-O. (2017). Can Likert scales be treated as interval scales? A simulation study, Journal of Social Service Research, 43(4), 527–532.
Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education, 12, 55–76.
Zucchini, W., MacDonald, I. L., & Langrock, R. (2016). Hidden Markov models for time series: An introduction using R. Boca Raton: Chapman and Hall/CRC.
Acknowledgements
The authors would like to thank the editor and three anonymous reviewers for their constructive comments, which helped us to improve the manuscript. This work was funded by the Portuguese Foundation for Science and Technology (Grant UID/GES/00315/2013 and UID/GES/00315/2019) and by Grant BIRD162088/16 financed by the University of Padua for the project entitled “Advances in Multilevel and Longitudinal Modelling”.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Guerra, M., Bassi, F. & Dias, J.G. A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching. Soc Indic Res 147, 361–381 (2020). https://doi.org/10.1007/s11205-019-02169-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-019-02169-x