Advertisement

Social Indicators Research

, Volume 140, Issue 3, pp 907–927 | Cite as

Examining Measurement Isomorphism of Multilevel Constructs: The Case of Political Trust

  • Anna Ruelens
  • Bart Meuleman
  • Ides Nicaise
Article

Abstract

There is a growing recognition of the importance of testing for measurement equivalence when comparing latent constructs—such as political trust—across countries and over time. Indeed, equivalence of measurements across countries and time points is a precondition for making meaningful and valid comparisons of means, scores, and relationships between constructs. Until recently, most efforts in this area of comparative research have focused on establishing measurement equivalence across groups, such as countries or cultural regions. In contrast, scarce attention has been paid to examining the measurement equivalence of constructs across levels of data. Rather, most empirical studies that use cross-national survey data assume that the factorial structure of a given construct is isomorphic, in other words, similar in measurement and meaning at the individual and country level. This assumption is not always justified, as the dimensions found at the individual level of data do not always generalize to the country level. In such cases, the results and substantive conclusions based on the assumption of measurement isomorphism may be misleading. In this article, we emphasize the importance of examining measurement isomorphism when working with cross-national survey data and describe a testing procedure using multilevel confirmatory factor analysis. We illustrate the procedure by examining cross-level equivalence of the political trust scale included in the European Social Survey (2008–2009). The results of the analysis indicate that the structure of political trust factor can be considered configurally isomorphic across individual and country levels. In addition, the measurement scale across the two levels is found to be partially isomorphic. These findings could be regarded as an encouraging result for the applied researchers who use the aggregated individual scores of political trust. At the same time, we demonstrate that measurement isomorphism cannot be simply assumed and hence should be examined prior to analysis to ensure valid and meaningful results.

Keywords

Political trust European Social Survey Measurement isomorphism Measurement equivalence Multilevel confirmatory factor analysis 

Notes

Funding

This research was part of the Re-InVEST project (Rebuilding an Inclusive, Value-based Europe of Solidarity and Trust through Social Investments), funded by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No 649447.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

11205_2017_1799_MOESM1_ESM.pdf (281 kb)
Supplementary material 1 (PDF 281 kb)

References

  1. Allum, N., Read, S., & Sturgis, P. (2011). Evaluating change in social and political trust in Europe. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 35–53). New York, London: Routledge.Google Scholar
  2. André, S. (2014). Does trust mean the same for migrants and natives? Testing measurement models of political trust with multi-group confirmatory factor analysis. Social Indicators Research, 115(3), 963–982.CrossRefGoogle Scholar
  3. Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303–316.CrossRefGoogle Scholar
  4. Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258.CrossRefGoogle Scholar
  5. Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. London: Routledge.Google Scholar
  6. Byrne, B. M., & van de Vijver, F. J. R. (2014). Factorial structure of the Family Values Scale from a multilevel-multicultural perspective. International Journal of Testing, 14(2), 168–192.CrossRefGoogle Scholar
  7. Cheung, M. W. L., & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling, 12(4), 598–619.CrossRefGoogle Scholar
  8. Cheung, M. W. L., Leung, K., & Au, K. (2006). Evaluating multilevel models in cross-cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology, 37(5), 522–541.CrossRefGoogle Scholar
  9. Cronbach, L. J. (with Deken, J.E., & Webb, N.). (1976). Research on classrooms and schools: Formulation of questions, design and analysis. Occasional Paper of the Stanford Evaluation Consortium, Stanford University, Stanford, CA.Google Scholar
  10. Davidov, E., Dülmer, H., Schlüter, E., Schmidt, P., & Meuleman, B. (2012). Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. Journal of Cross-Cultural Psychology, 43(4), 558–575.CrossRefGoogle Scholar
  11. Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual Review of Sociology, 40, 55–75.CrossRefGoogle Scholar
  12. D’haenens, E., Van Damme, J., & Onghena, P. (2010). Multilevel exploratory factor analysis: Illustrating its surplus value in educational effectiveness research. School Effectiveness and School Improvement, 21(2), 209–235.CrossRefGoogle Scholar
  13. Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. The Leadership Quarterly, 16(1), 149–167.CrossRefGoogle Scholar
  14. Fischer, R. (2012). Value isomorphism in the European Social Survey: Exploration of meaning shifts in values across levels. Journal of Cross-Cultural Psychology, 43(6), 883–898.CrossRefGoogle Scholar
  15. Fontaine, J. (2008). Traditional and multilevel approaches in cross-cultural research: An integration of methodological frameworks. In F. J. R. Van de Vijver, D. A. Van Hemert, & Y. H. Poortinga (Eds.), Multilevel analysis of individuals and cultures (pp. 65–92). Hove: Psychology Press.Google Scholar
  16. Fontaine, J., & Fischer, R. (2011). Data analytic approaches for investigating isomorphism between the individual-level and the cultural-level internal structure. In D. Matsumoto & F. J. R. van de Vijver (Eds.), Cross-cultural research methods in psychology (pp. 273–298). New York: Cambridge University Press.Google Scholar
  17. Goldstein, H., & McDonald, R. P. (1988). A general model for the analysis of multilevel data. Psychometrika, 53(4), 455–467.CrossRefGoogle Scholar
  18. Härnqvist, K. (1978). Primary mental abilities at collective and individual levels. Journal of Educational Psychology, 70(5), 706.CrossRefGoogle Scholar
  19. Hoofs, H., van de Schoot, R., Jansen, N. W., & Kant, I. (2017). Evaluating model fit in Bayesian confirmatory factor analysis with large samples: Simulation study introducing the BRMSEA. Educational and Psychological Measurement.  https://doi.org/10.1177/0013164417709314.CrossRefGoogle Scholar
  20. Hooghe, M. (2011). Why there is basically only one form of political trust. The British Journal of Politics & International Relations, 13(2), 269–275.CrossRefGoogle Scholar
  21. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144.CrossRefGoogle Scholar
  22. Hox, J. J., Moerbeek, M., & van de Schoot, R. (2010). Multilevel analysis: Techniques and applications. London: Routledge.CrossRefGoogle Scholar
  23. Hox, J., van de Schoot, R., & Matthijsse, S. (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6(2), 87–93.Google Scholar
  24. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.CrossRefGoogle Scholar
  25. Jak, S. (2014). Testing strong factorial invariance using three-level structural equation modeling. Frontiers in Psychology, 5, 745.  https://doi.org/10.3389/fpsyg.2014.00745.CrossRefGoogle Scholar
  26. Jak, S. (2017). Testing and explaining differences in common and residual factors across many countries. Journal of Cross-Cultural Psychology, 48(1), 75–92.CrossRefGoogle Scholar
  27. Jak, S., Oort, F. J., & Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling: A Multidisciplinary Journal, 20(2), 265–282.CrossRefGoogle Scholar
  28. Jak, S., Oort, F. J., & Dolan, C. V. (2014). Measurement bias in multilevel data. Structural Equation Modeling: A Multidisciplinary Journal, 21(1), 31–39.CrossRefGoogle Scholar
  29. Kaplan, D., & Depaoli, S. (2012). Bayesian structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modelling (pp. 650–673). New York, NY: Guilford Press.Google Scholar
  30. Kaplan, D., Kim, J. S., & Kim, S. Y. (2009). Multilevel latent variable modeling: Current research and recent developments. In R. E. Millsap & A. Maydeu-Olivares (Eds.), The SAGE handbook of quantitative methods in psychology (pp. 592–612). Thousand Oaks, CA: SAGE.CrossRefGoogle Scholar
  31. Marien, S. (2011). Measuring political trust across time and space. In M. Hooghe & S. Zmerli (Eds.), Political trust: Why context matters (pp. 13–46). Colchester: ECPR Press.Google Scholar
  32. Marien, S. (2017). The measurement equivalence of political trust. In T. W. G. Van der Meer & S. Zmerli (Eds.), Handbook of political trust. Cheltenham: Edward Elgar Publishing.Google Scholar
  33. Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10(3), 259.CrossRefGoogle Scholar
  34. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543.CrossRefGoogle Scholar
  35. Meuleman, B., & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? Survey Research Methods, 3(1), 45–58.Google Scholar
  36. Meuleman, B., Davidov, E., & Billiet, J. (2009). Changing attitudes toward immigration in Europe, 2002–2007: A dynamic group conflict theory approach. Social Science Research, 38(2), 352–365.CrossRefGoogle Scholar
  37. Muthén, B. O. (1990). Mean and covariance structure analysis of hierarchical data. Department of Statistics, UCLA Statistics Series, No 62. Los Angeles: University of California.Google Scholar
  38. Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28(4), 338–354.CrossRefGoogle Scholar
  39. Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22(3), 376–398.CrossRefGoogle Scholar
  40. Muthén, B. O., & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups. Unpublished technical report. Accessed June 14, 2017, from http://www.statmodel.com.
  41. Muthén, B. O., & Muthén, L. K. (2012). Mplus 7 base program. Los Angeles, CA: Muthén & Muthén.Google Scholar
  42. Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316.CrossRefGoogle Scholar
  43. Muthén, L. K., & Muthén, B. O. (2004). Mplus user’s guide (3rd ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  44. Oskarsson, S. (2010). Generalized trust and political support: A cross-national investigation. Acta Politica, 45(4), 423–443.CrossRefGoogle Scholar
  45. Poznyak, D., Meuleman, B., Abts, K., & Bishop, G. F. (2014). Trust in American government: Longitudinal measurement equivalence in the ANES, 1964–2008. Social Indicators Research, 118(2), 741–758.CrossRefGoogle Scholar
  46. Puntscher, S., Hauser, C., Walde, J., & Tappeiner, G. (2016). Measuring social capital with aggregated indicators: A case of ecological fallacy? Social Indicators Research, 125(2), 431–449.CrossRefGoogle Scholar
  47. Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69(2), 167–190.CrossRefGoogle Scholar
  48. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.Google Scholar
  49. Rothstein, B., & Stolle, D. (2008). The state and social capital: An institutional theory of generalized trust. Comparative Politics, 40(4), 441–459.CrossRefGoogle Scholar
  50. Schneider, I. (2016). Can we trust measures of political trust? Assessing measurement equivalence in diverse regime types. Social Indicators Research, 133(3), 963–984.CrossRefGoogle Scholar
  51. Schwartz, S. H. (1992). Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25, 1–65.CrossRefGoogle Scholar
  52. Schwartz, S. H. (1994). Beyond individualism/collectivism: New cultural dimensions of values. In U. Kim, H. C. Triandis, C. Kagitcibasi, S. C. Choi, & G. Yoon (Eds.), Individualism and collectivism: Theory, method and applications (pp. 85–119). Thousand Oaks: SAGE.Google Scholar
  53. Schwartz, S. H. (2006). A theory of cultural value orientations: Explication and applications. Comparative Sociology, 5(2), 137–182.CrossRefGoogle Scholar
  54. Schweig, J. (2014). Cross-level measurement invariance in school and classroom environment surveys: Implications for policy and practice. Educational Evaluation and Policy Analysis, 36(3), 259–280.CrossRefGoogle Scholar
  55. Snijders, T., & Bosker, R. (1999). Multilevel modeling: An introduction to basic and advanced multilevel modeling. London: SAGE.Google Scholar
  56. Stankov, L., & Saucier, G. (2015). Social axioms in 33 countries: Good replicability at the individual but less so at the country level. Journal of Cross-Cultural Psychology, 46(2), 296–315.CrossRefGoogle Scholar
  57. Steenkamp, J. B. E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90.CrossRefGoogle Scholar
  58. Stegmueller, D. (2013). How many countries for multilevel modeling? A comparison of frequentist and Bayesian approaches. American Journal of Political Science, 57(3), 748–761.CrossRefGoogle Scholar
  59. Tay, L., Woo, S. E., & Vermunt, J. K. (2014). A conceptual and methodological framework for psychometric isomorphism: Validation of multilevel construct measures. Organizational Research Methods, 17(1), 77–106.CrossRefGoogle Scholar
  60. Turper, S., & Aarts, K. (2017). Political trust and sophistication: Taking measurement seriously. Social Indicators Research, 130(1), 415–434.CrossRefGoogle Scholar
  61. van de Schoot, R., & Depaoli, S. (2014). Bayesian analyses: Where to start and what to report. European Health Psychologist, 16(2), 75–84.Google Scholar
  62. Van de Vijver, F. J. R. (1998). Towards a theory of bias and equivalence. Zuma Nachrichten Spezial, Cross-cultural survey equivalence, 3, 41–65.Google Scholar
  63. Van de Vijver, F. J. R., & Poortinga, Y. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33(2), 141–156.CrossRefGoogle Scholar
  64. Van de Vijver, F. J. R., Van Hemert, D. A., & Poortinga, Y. H. (Eds.). (2008). Individuals and cultures in multilevel analysis. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  65. Van de Vijver, F. J. R., & Watkins, D. (2006). Assessing similarity of meaning at the individual and country level. European Journal of Psychological Assessment, 22(2), 69–77.CrossRefGoogle Scholar
  66. Van der Brug, W., & Van Praag, P. (2007). Erosion of political trust in the Netherlands: Structural or temporarily? A research note. Acta Politica, 42(4), 443–458.CrossRefGoogle Scholar
  67. Van der Veld, W. M., & Saris, W. E. (2011). Causes of generalized social trust. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 207–247). New York, London: Routledge.Google Scholar
  68. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.CrossRefGoogle Scholar
  69. Zmerli, S., & Newton, K. (2008). Social trust and attitudes toward democracy. Public Opinion Quarterly, 72(4), 706–724.CrossRefGoogle Scholar
  70. Zyphur, M. J., Kaplan, S. A., & Christian, M. S. (2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics: Theory, Research, and Practice, 12(2), 127.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2017

Authors and Affiliations

  1. 1.Research Institute for Work and SocietyUniversity of LeuvenLeuvenBelgium
  2. 2.Centre for Sociological ResearchUniversity of LeuvenLeuvenBelgium

Personalised recommendations