Abstract
For over 50 years (1958–2012) the American National Election Studies (ANES) survey has been measuring citizens’ evaluations of the trustworthiness of the “government in Washington”—an indicator that has been widely used to monitor the dynamics of political trust in the US over time. However, a critical assumption in using attitudinal constructs for longitudinal research is that the meaning-and-interpretation of such items should be comparable across groups of respondents at any one point in time and across samples over time. Using multigroup confirmatory factor analysis for ordered-categorical data, we test the measurement equivalence assumption with data collected by the ANES from 1964 to 2008. The results confirm that the ANES’ political trust scale has the same basic factorial structure over time. But for two key items, several threshold parameters were found to be different across time points, indicating that the meaning-and-interpretation of these questions, and especially the question about whether the government in Washington wastes money that people pay in taxes, varies significantly over time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Although ANES has been measuring political trust since 1958, the presumably comparable four-item scale of trust has been only available since 1964.
Examples of alternative approaches are Item Response Theory (IRT: Raju et al. 2002; Stegmueller 2011) or Latent Class Analysis (Kankaraš and Moors 2010). However, the MGCFA approach has received most scholarly attention in the field of measurement equivalence (see e.g. Steenkamp and Baumgartner 1998; Billiet 2003; Vandenberg and Lance 2000; Millsap and Meredith 2007). As a consequence, the implications of certain research practices (e.g. modeling strategies, the use of fit criteria) have been studied most extensively.
Yet, a simulation by Lubke and Muthén (2004) shows that treating categorical data as continuous leads to inaccurate estimation.
In principle, a categorical variable with c + 1 categories has c + 2 thresholds. However, the first and last threshold equal -∞ respectively +∞ by definition. Therefore, only c threshold parameters need to be estimated.
Thresholds for a an item with 4 categories, and a (0.25, 0.25, 0.25, 0.25) distribution equal: \(\varPhi^{ - 1} (0.25)\, = \, - 0.67;\,\varPhi^{ - 1} (0.50)\, = \,0;\,\varPhi^{ - 1} (0.75)\, = \, 0.67\).
Besides fixing these intercepts to zero, additional model restrictions are needed to make MGCFA models for categorical data identified. In Mplus, two model parameterizations—namely theta and delta—are possible. For invariance testing, the so-called theta parameterization is preferable (Millsap and Yun-Tein 2004: 489). In this approach, residual variances of the latent response variables are allowed to be parameters in the models, and the scale factors (defining the metric of the continuous latent response variables) are not. The residual variances in the first group are fixed at unity, and residual variances in the other groups are estimated freely (Muthén and Muthén 1998–2012). Latent means are fixed to 0 in the reference group, and estimated freely in other groups. Furthermore, cross-group equality constraints are imposed on thresholds and factor loadings, depending on the required level of measurement equivalence. For a more detailed discussion of the complex issue of model identification in MGCFA for categorical data (see Millsap and Yun-Tein 2004; Temme 2006; Muthén and Asparouhov 2002).
The cumulative dataset to which we refer includes all survey waves in the 1948–2008 period. However, the presumably comparable, four-item measure of political trust has only been available since 1964, except for 1966 and 1982 when political trust was measured with three items; 1986 when it was measured by a single “Do What is Right” item; and 2006 when there were no ANES trust items.
The number of thresholds in the item depends on the number of response categories. Since the thresholds capture transition from one response category to another, in a variable with c + 1 response categories, the number of free thresholds equals c (Davidov et al. 2011). In our case, the Do What is Right, Crooked and Waste items each have two thresholds, and the binary Interest item has one threshold.
The model fit statistics are not presented but can be made available by the authors upon request.
See the online appendix for the Mplus syntax used to test the measurement equivalence of the political trust construct.
The χ2 ‘goodness of fit’-test is overly sensitive when sample sizes are as large as the one employed here (Brown 2006: 81). Therefore, we will abstain from making conclusions based on the χ2- and p-value statistics (although they are reported). Instead, alternative measures of global fit are used (for further information, see e.g. Schumacher and Lomax 2004; Mulaik 2009; Taylor 2008).
The partially equivalent model entails that respondents with the same score on the latent trust scale at different points of time would have same scores on the Do What Is Right and Interest items, but may have different scores on the Waste and Crooked items.
Although the same concerns apply to the Crooked indicator, it exhibits substantially less invariance between samples.
It should be noted that the principle of partial equivalence has been established in the literature for continuous data. Further research is warranted in order to confirm that the same principles apply to ordered-categorical scales.
In order to construct the additive index of trust, the responses are first transformed from 0 for distrusting responses, 50 for middle position (except for the dummy-coded Interest item), and 100 for trusting responses, summed and divided by the number of valid responses. In case the volunteered option “none of the time” of the Do What Is Right item is included in the analysis, the item is recoded as follows: None of the Time = 0, Some of the Time = 33, Most of the Time = 67, Just About Always = 100 (www.electionstudies.org/nesguide/toptable/tab5a_5.htm).
References
Ariely, G., & Davidov, E. (2010). Can we rate public support for democracy in a comparable way? Cross-national equivalence of democratic attitudes in the World value survey. Social Indicators Research, 104(2), 271–286.
Asparouhov, T., & Muthén, B. O. (2010). Weighted least squares estimation with missing data. Technical report. http://www.statmodel.com/download/GstrucMissingRevision.pdf.
Bennett, S. E. (2001). Were the halcyon days really golden? An analysis of Americans’ attitudes about the political system, 1945–1965. In J. R. Hibbing, & E. Theiss-Morse (Eds.), What is it About Government that Americans Dislike (pp. 47–58). Cambridge: Cambridge University Press.
Bennett, S. E., Rhine, S. L., Flickinger, R. S., & Bennett, L. L. M. (1999). “Video Malaise” Revisited: Public Trust in the Media and Government. The International Journal of Press/Politics , 4(4), 8–23.
Bergsma, W., Croon, M., & Hagenaars, J. A. (2009). Marginal models: For dependent, clustered, and longitudinal categorical data. Berlin: Springer.
Billiet, J. (2003). Cross-cultural equivalence with structural equation modeling. In J. Harkness, F. Van de Vijver, & P. Mohler (Eds.), Cross-cultural survey methods (pp. 247–264). New York, NY: Wiley.
Bishop, G. F. (1990). Issue involvement and response effects in public opinion surveys. Public Opinion Quarterly, 54(2), 209–218.
Bishop, G. F. (2005). The illusion of public opinion: Fact and artifact in american public opinion polls. Lanham, MD: Rowman & Littlefield.
Bishop, G.F., & Mockabee, S.T. (2011). Comparability of measurement in public opinion polls. Survey Practice , 4(5).
Bishop, G. F., Oldendick, R. W., & Tuchfarber, A. J. (1982). Political information processing: Question order and context effects. Political Behavior, 4(2), 177–200.
Blasius, J., & Thiessen, V. (2001). Methodological artifacts in measures of political efficacy and trust: A multiple correspondence analysis. Political Analysis , 9(1), 1–20.
Blendon, R. J., Benson, J. M., Morin, R., Altman, D. E., Brodie, M., Brossard, M., & James, M. (1997). Changing Attitudes in America. In J. S. Nye, P. Zelikow, & D. C. King (Eds.), Why people don’t trust government (pp. 205–216). Cambridge, MA: Harvard University Press.
Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin , 110(2), 305–314.
Brown, T. A. (2010). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
Byrne, B. M., Shavelson, R. J., & Muthén, B. O. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466.
Chanley, V. A. (2002). Trust in Government in the aftermath of 9/11: Determinants and consequences. Political Psychology, 23(3), 469–483.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance evaluating goodness-. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255.
Citrin, J. (1974). Comment: The political relevance of trust in government. The American Political Science Review , 68(3), 973–988.
Citrin, J., & Green, D. P. (1986). Presidential leadership and the resurgence of trust in government. British Journal of Political Science, 16(4), 431–453.
Citrin, J., & Muste, C. (1999). Trust in Government. In J. R. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of political attitudes (pp. 465–532). San Diego, CA: Academic Press.
Cook, T. E., & Gronke, P. (2005). The skeptical American: Revisiting the meanings of trust in government and confidence in institutions. The Journal of Politics , 67(3), 784–803.
Davidov, E., Datler, G., Schmidt, P., & Schwartz, S. H. (2011). Testing the invariance of values in the Benelux countries with the European Social Survey: Accounting for ordinality. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 149–168). London: Routledge.
Davidov, E., Schmidt, P., & Schwartz, S. H. (2008). Bringing values back in: The adequacy of the European social survey to measure values in 20 Countries. The Public Opinion Quarterly, 72(3), 420–445.
De Beuckelaer, A., & Swinnen, G. (2011). Biased latent variable mean comparison due to measurement noninvariance. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-Cultural Analysis: Methods and Applications. New York: Routledge.
DiStefano, C. (2002). The impact of categorization with confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 9(3), 327–346.
Donnelan, M. B., & Conger, R. D. (2009). Designing and implementing Longitudinal studies. In R. F. K. Richard, W. Robins, & R. C. Fraley (Eds.), Handbook of research methods in personality psychology (p. 719). New York, NY: Guilford Press.
Easton, D. (1965). A systems analysis of political life. New York, NY: Wiley.
Eid, M., Langeheine, R., & Diener, E. (2003). Comparing typological structures across cultures by multigroup latent class analysis: A primer. Journal of Cross-Cultural Psychology , 34(2), 195–210.
Gamson, W. A. (1968). Power and discontent. Homewood, IL: Dorsey Press.
Groves, R. M. (1989). Survey errors and survey costs. New York, NY: Wiley.
Hetherington, M. J. (2005). Why trust matters: Declining political trust and the demise of American Liberalism. Princeton, NJ, USA: Princeton University Press.
Hetherington, M. J., & Rudolph, T. J. (2008). Priming, performance, and the dynamics of political trust. The Journal of Politics, 70(2), 498–512.
Hibbing, J. R., & Theiss-Morse, E. (1995). Congress as public enemy: Public attitudes toward American political institutions (p. 186). Cambridge: Cambridge University Press.
Hibbing, J. R., & Theiss-Morse, E. (Eds.). (2001). What is it about government that Americans dislike? (p. 279). Cambridge: Cambridge University Press.
Hibbing, J. R., & Theiss-Morse, E. (2002). Stealth democracy: Americans’ beliefs about how government should work (p. 284). Cambridge: Cambridge University Press.
Horn, J. L., & McArdle, J. L. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144.
Johnson, T. P. (1998). Approaches to equivalence in cross-cultural and cross-national survey-research. ZUMA Nachrichten Spezial, Cross-cultural survey equivalence, 3, 1–40.
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409–426.
Jöreskog, K. G. (1990). New developments in LISREL: analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity , 24(4), 387–404.
Kankaraš, M., & Moors, G. (2010). Researching measurement equivalence in cross-cultural studies. Psihologija, 43(2), 121–136.
Keele, L. (2005). The authorities really do matter: Party control and trust in government. The Journal of Politics, 67(3), 873–886.
Kent, R. (2001). Data construction and data analysis for survey research. Basingstoke: Palgrave Macmillan.
Levi, M., & Stoker, L. (2000). Political trust and trustworthiness. Annual Review of Political Science , 3, 475–507.
Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scales complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.
Meredith, W. (1993). Measurment invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543.
Miller, A. H. (1974). Political issues and trust in government: 1964–1970. The American Political Science Review , 68(3), 951–972.
Millsap, R. E., & Meredith, W. (2007). Factorial Invariance: Historical perspectives and new problems. In R. C. M. Cudeck (Ed.), Factor analysis at 100: Historical developments and future directions (pp. 131–152). Mahwah, NJ, USA: Lawrence Erlbaum.
Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479–515.
Mulaik, S. A. (2009). Linear causal modeling with structural equations. Boca Raton, FL: CRC Press.
Muthén, B. O., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. Mplus Web Note No. 4. http://www.statmodel.com/mplus/examples/webnote.html.
Muthén, B. O., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189.
Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén.
Nye, J. S., Zelikow, P., & King, D. C. (Eds.) (1997). Why people don’t trust government. (p. 337). Cambridge: Harvard University Press.
Parker, S. L. (1986). The dynamics of changing system support in the United States: 1964–1980. PhD. Dissertation. Florida State University.
Poortinga, Y. H. (1989). Equivalence of cross-cultural data: An overview of basic issues. International Journal of Psychology, 24, 737–756.
Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 517–529.
Reeskens, T., & Hooghe, M. (2007). Cross-cultural measurement equivalence of generalized trust. Evidence from the European social survey (2002 and 2004). Social Indicators Research, 85(3), 515–532.
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 113(3), 552–566.
Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation and analysis of questionnaires for survey research. New York, NY: Wiley.
Schumacher, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling. Mahwah, NJ: Lawrence Erlbaum Associates.
Schuman, H., & Presser, S. (1996). Questions and answers in attitude surveys: Experiments on question form, wording, and context. Thousand Oaks, CA: Sage Publications.
Schwarz, N., & Sudman, S. (1992). Context effects in social and psychological research. New York: Springer.
Shevlin, M., Miles, J., & Bunting, B. (1997). Summated rating scales. A Monte Carlo investigation of the effects of reliability and collinearity in regression models. Personality and Individual Differences , 23(4), 665–676.
Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. The Journal of Consumer Research, 25(1), 78–90.
Stegmueller, D. (2011). Apples and oranges? The problem of equivalence in comparative research. Political Analysis, 19, 471–487.
Steinmetz, H. (2012). Analyzing observed composite differences across groups. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(1), 1–12.
Stokes, D. E. (1962). Popular evaluations of Government: An empirical assessment. In H. Cleveland & H. D. Laswell (Eds.), Ethics and business: Scientific academic religious political and military (pp. 61–72). New York: Harper & Brothers.
Taylor, A. B. (2008). Two new methods of studying the performance of SEM fit indexes. PhD Diss. Arizona State University.
Temme, D. (2006). Assessing measurement invariance of ordinal indicators in cross-national research. In S. Diehl, R. Terlutter, & P. Weinberg (Eds.), International advertising and communication—New insights and empirical findings (pp. 455–472). Gabler.
The American National Election Studies Time-Series Cumulative Data File (1948–2008). Stanford University and the University of Michigan [producers and distributors]. http://www.electionstudies.org.
Turner, C. F. (1984). Why do surveys disagree? Some preliminary hypotheses and some disagreeable examples. In C. F. Turner & E. Martin (Eds.), Surveying subjective phenomena (pp. 159–214). New York: Russell Sage Foundation.
Ulbig, S. G. (2002). Subnational contextual influences on political trust. PhD Dissertation. Rice University.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Poznyak, D., Meuleman, B., Abts, K. et al. Trust in American Government: Longitudinal Measurement Equivalence in the ANES, 1964–2008. Soc Indic Res 118, 741–758 (2014). https://doi.org/10.1007/s11205-013-0441-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-013-0441-5