Multi-group Invariance Testing: An Illustrative Comparison of PLS Permutation and Covariance-Based SEM Invariance Analysis

  • Wynne W. ChinEmail author
  • Annette M. Mills
  • Douglas J. Steel
  • Andrew Schwarz
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 173)


This paper provides a didactic example of how to conduct multi-group invariance testing distribution-free multi-group permutation procedure used in conjunction with Partial Least Squares (PLS).To address the likelihood that methods such as covariance-based SEM (CBSEM) with chi-square difference testing can enable group effects that mask noninvariance at lower levels of analysis problem, a variant of CBSEM invariance testing that focuses the evaluation on one parameter at a time (i.e. single parameter invariance testing) is proposed. Using a theoretical model from the field of Information Systems, with three exogenous constructs (routinization, infusion, and faithfulness of appropriation) predicting the endogenous construct of deep usage, the results show both techniques yield similar outcomes for the measurement and structural paths. The results enable greater confidence in the permutation-based procedure with PLS. The pros and cons of both techniques are also discussed.


Multi-group Invariance Testing Permutation Analysis PLS Covariance Based SEM 


  1. Bagozzi, R.P., Foxall, G.R.: Construct validity and generalizability of the Kirton adaption-innovation inventory. Eur. J. Personal. 9, 185–206 (1995)CrossRefGoogle Scholar
  2. Bentler, P.M.: EQS: Structural Equations Program Manual. BMDP Statistical Software, Los Angeles (1992)Google Scholar
  3. Bhattacherjee, A.: Understanding information systems continuance: an expectation-confirmation model. MIS Q. 25, 351–370 (2001)CrossRefGoogle Scholar
  4. Bollen, K.A.: Structural Equations with Latent Variables. Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics Section. John Wiley, New York (1989)CrossRefzbMATHGoogle Scholar
  5. Breckler, S.J.: Applications of covariance structure modeling in psychology: cause for concern? Psychol. Bull. 107, 260–273 (1990)CrossRefGoogle Scholar
  6. Byrne, B.M.: The Maslach Burnout inventory: testing for factorial validity and invariance across elementary, intermediate and secondary teachers. J. Occup. Organ. Psychol. 66, 197–212 (1993)CrossRefGoogle Scholar
  7. Byrne, B.M.: Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming. Lawrence Erlbaum Associates, Mahwah (2010)Google Scholar
  8. Cheung, G.W., Rensvold, R.B.: Evaluating Goodness-of-fit Indexes for Testing Measurement Invariance. Lawrence Erlbaum Associates, Hillsdale, NJ (2002)Google Scholar
  9. Chin, W.W.: A permutation procedure for multi-group comparison of PLS models. Invited presentation. In: Vilares, M., Tenenhaus, M., Coelho, P., Esposito Vinzi, V., Morineau, A. (eds.) PLS and Related Methods, PLS’03 International Symposium – “Focus on Customers”, Lisbon, pp. 33–43 (2003)Google Scholar
  10. Chin, W.W.: How to write up and report PLS analyses. In: Esposito Vinzi, V., Chin, W.W., Hensler, J., Wang, H. (eds.) Handbook of Partial Least Squares. Springer Handbooks of Computational Statistics, pp. 655–690. Springer, Berlin/Heidelberg (2010)CrossRefGoogle Scholar
  11. Chin, W.W., Dibbern, J.: An introduction to a permutation based procedure for multi-group PLS analysis: results of tests of differences on simulated data and a cross cultural analysis of the sourcing of information system services between Germany and the USA. In: Esposito Vinzi, V., Chin, W.W., Hensler, J., Wang, H. (eds.) Handbook of Partial Least Squares. Springer Handbooks of Computational Statistics, pp. 171–193. Springer, Berlin/Heidelberg (2010)CrossRefGoogle Scholar
  12. Chin, W.W., Marcolin, B.L.: The future of diffusion research. Data Base Adv. Inf. Syst. 32, 8–12(2001)CrossRefGoogle Scholar
  13. Chin, W.W., Gopal, A., Salisbury, W.D.: Advancing the theory of adaptive structuration: the development of a scale to measure faithfulness of appropriation. Inf. Syst. Res. 8, 342–367 (1997)CrossRefGoogle Scholar
  14. Deng, X.D., Doll, W.J., Al-Gahtani, S.S., Larsen, T.J., Pearson, J.M., Raghunathan, T.S.: A cross-cultural analysis of the end-user computing satisfaction instrument: a multi-group invariance analysis. Inf. Manag. 45, 211–220 (2008)CrossRefGoogle Scholar
  15. Doll, W.J., Deng, X.D., Raghunathan, T.S., Torkzadeh, G., Xia, W.D.: The meaning and measurement of user satisfaction: a multigroup invariance analysis of the end-user computing satisfaction instrument. J. Manag. Inf. Syst. 21, 227–262 (2004)Google Scholar
  16. Edgington, E.S.: Randomization Tests. Marcel Dekker, New York (1987)zbMATHGoogle Scholar
  17. Good, P.: Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses, 2nd edn. Springer, New York (2000)CrossRefzbMATHGoogle Scholar
  18. Hsieh, J.J., Rai, A., Xu, S.X.: Extracting business value from IT: a sensemaking perspective of post-adoptive use. Manag. Sci. 57, 2018–2039 (2011)CrossRefGoogle Scholar
  19. Keil, M., Tan, B.C.Y., Wei, K.-K., Saarinen, T., Tuunainen, V., Wassenaar, A.: A cross-cultural study on escalation of commitment behavior in software projects. MIS Q. 24, 299–325 (2000)CrossRefGoogle Scholar
  20. Lai, V.S., Li, H.: Technology acceptance model for internet banking: an invariance analysis. Inf. Manag. 42, 373–386 (2005)CrossRefGoogle Scholar
  21. Malhotra, M.K., Sharma, S.: Measurement equivalence using generalizability theory: an examination of manufacturing flexibility dimensions. Decis. Sci. 39, 643–669 (2008)CrossRefGoogle Scholar
  22. Mardia, K.V.: Measures of multivariate skewness and kurtosis with applications. Biometrika 57, 519–530 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  23. Noreen, E.W.: Computer Intensive Methods for Testing Hypotheses: An Introduction. John Wiley, New York (1989)Google Scholar
  24. Saeed, K.A., Abdinnour-Helm, S.: Examining the effects of information system characteristics and perceived usefulness on post adoption usage of information systems. Inf. Manag. 45, 376–386 (2008)CrossRefGoogle Scholar
  25. Saga, V.L., Zmud, R.W.: The nature and determinants of IT acceptance, routinization and infusion. In: Levine, L. (ed.) Diffusion, Transfer and Implementation of Information Technology, pp. 67–86. Elsevier Science B.V./North-Holland, Amsterdam (1994)Google Scholar
  26. Sarstedt, M., Henseler, J., Ringle, C.M.: Multigroup analysis in partial least squares (PLS) path modeling: alternative methods and empirical results. In: Sarstedt, M., Schwaiger, M., Taylor, C.R. (eds.) Measurement and Research Methods in International Marketing. Advances in International Marketing, pp. 195–218. Emerald Group Publishing, Bingley (2011)CrossRefGoogle Scholar
  27. Schwarz, A.H.: Defining information technology acceptance: a human-centered, management-oriented perspective, University of Houston, Houston (2003)Google Scholar
  28. Steenkamp, J.E.M., Baumgartner, H.: Assessing measurement invariance in cross-national consumer research. J. Consum. Res. 25, 78–90 (1998)CrossRefGoogle Scholar
  29. Sundaram, S., Schwarz, A., Jones, E., Chin, W.W.: Technology use on the front line: how information technology enhances individual performance. J. Acad. Mark. Sci. 35, 101–112 (2007)CrossRefGoogle Scholar
  30. Teo, T., Lee, C.B., Chai, C.S., Wong, S.L.: Assessing the intention to use technology among pre-service teachers in Singapore and Malaysia: a multigroup invariance analysis of the technology acceptance model (TAM). Comput. Educ. 53, 1000–1009 (2009)CrossRefGoogle Scholar
  31. Vandenberg, R.J., Lance, C.E.: A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3, 4–70 (2000)CrossRefGoogle Scholar
  32. Wang, W., Hsieh, P.: Beyond routine: symbolic adoption, extended use, and emergent use of complex information systems in the mandatory organizational context. In: Proceedings: International Conference of Information System, Milwaukee (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Wynne W. Chin
    • 1
    Email author
  • Annette M. Mills
    • 2
  • Douglas J. Steel
    • 3
  • Andrew Schwarz
    • 4
  1. 1.Department of Decision and Information SystemsC. T. Bauer College of Business, University of HoustonHoustonUSA
  2. 2.Department of Accounting and Information SystemsCollege of Business and Economics, University of CanterburyIlam ChristchurchNew Zealand
  3. 3.Department of Management Information SystemsSchool of Business, University of Houston-Clear LakeHoustonUSA
  4. 4.Louisiana State UniversityBaton RougeUSA

Personalised recommendations