Advertisement

Controlling Acquiescence Bias with Multidimensional IRT Modeling

  • Ricardo PrimiEmail author
  • Nelson Hauck-Filho
  • Felipe Valentini
  • Daniel Santos
  • Carl F. Falk
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 265)

Abstract

Acquiescence is a commonly observed response style that may distort respondent scores. One approach to control for acquiescence involves creating a balanced scale and computing sum scores. Other model-based approaches may explicitly include an acquiescence factor as part of a factor analysis or multidimensional item response model. Under certain assumptions, both approaches may result in acquiescence-controlled scores for each respondent. However, the validity of the resulting scores is one issue that is sometimes ignored. In this paper, we present an application of these approaches under both balanced and unbalanced scales, and we report changes in criterion validity and respondent scores.

Keywords

Acquiescence bias Item response modeling 

References

  1. Allen, M. S., Walter, E. E., & McDermott, M. S. (2017). Personality and sedentary behavior: A systematic review and meta-analysis. Health Psychology, 36(3), 255–263.  https://doi.org/10.1037/hea0000429.CrossRefGoogle Scholar
  2. Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling, 7(4), 608–628.  https://doi.org/10.1207/S15328007SEM0704_5.CrossRefGoogle Scholar
  3. Cai, L. (2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75, 581–612.  https://doi.org/10.1007/s11336-010-9178-0.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Chalmers, R. P. (2012). MIRT: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. Retrieved from http://www.jstatsoft.org/v48/i06/.
  5. De Ayala, R. J. (2009). The theory and practice of item response theory. New York: Guilford Publications.Google Scholar
  6. Embretson, S. E., & Reise, S. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  7. Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482.  https://doi.org/10.1086/504455.CrossRefGoogle Scholar
  8. Huang, I.-C., Lee, J. L., Ketheeswaran, P., Jones, C. M., Revicki, D. A., & Wu, A. W. (2017). Does personality affect health-related quality of life? A systematic review. PLOS ONE, 12(3), e0173806.  https://doi.org/10.1371/journal.pone.0173806.CrossRefGoogle Scholar
  9. John, O., Naumann, L. P., & Soto, C. J. (2008). Paradigm shift to the integrative Big-Five trait taxonomy: History, measurement, and conceptual issues. In O. John, R. W. Robins, & L. A. Pervin (Eds.), Handbook of personality: Theory and research (pp. 114–158). New York: Guilford Press.Google Scholar
  10. Kam, C. C. S., & Meyer, J. P. (2015). How careless responding and acquiescence response bias can influence construct dimensionality. Organizational Research Methods, 18(3), 512–541.  https://doi.org/10.1177/1094428115571894.CrossRefGoogle Scholar
  11. Knowles, E. S., & Nathan, K. T. (1997). Acquiescent responding in self-reports: cognitive style or social concern? Journal of Research in Personality, 31(2), 293–301.  https://doi.org/10.1006/jrpe.1997.2180.CrossRefGoogle Scholar
  12. Kyllonen, P. C., Lipnevich, A. A., Burrus, J., & Roberts, R. D. (2014). Personality, motivation, and college readiness: A prospectus for assessment and development. ETS Research Report Series, 2014(1), 1–48.  https://doi.org/10.1002/ets2.12004.CrossRefGoogle Scholar
  13. Lipnevich, A. A., Preckel, F., & Roberts, R. D. (2016). Psychosocial skills and school systems in the 21th century. New York: Springer.Google Scholar
  14. Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344–362.  https://doi.org/10.1037/1082-989X.11.4.344.CrossRefGoogle Scholar
  15. Maydeu-Olivares, A., & Steenkamp, J. E. M. (2018). An integrated procedure to control for common method variance in survey data using random intercept factor analysis models. https://www.academia.edu/36641946/An_integrated_procedure_to_control_for_common_method_variance_in_survey_data_using_random_intercept_factor_analysis_models.
  16. McCrae, R. R. (2018). Method biases in single-source personality assessments. Psychological Assessment, 30(9), 1160–1173.  https://doi.org/10.1037/pas0000566.CrossRefGoogle Scholar
  17. Meisenberg, G., & Williams, A. (2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44(7), 1539–1550.  https://doi.org/10.1016/j.paid.2008.01.010.CrossRefGoogle Scholar
  18. Mirowsky, J., & Ross, C. E. (1991). Eliminating defense and agreement bias from measures of the sense of control: A 2 × 2 index. Social Psychology Quarterly, 54(2), 127–145.  https://doi.org/10.2307/2786931.CrossRefGoogle Scholar
  19. Ozer, D. J., & Benet-Martínez, V. (2006). Personality and the prediction of consequential outcomes. Annual Review of Psychology, 57(1), 401–421.  https://doi.org/10.1146/annurev.psych.57.102904.190127.CrossRefGoogle Scholar
  20. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrighsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San Diego, CA: Academic Press.CrossRefGoogle Scholar
  21. Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322–338.  https://doi.org/10.1037/a0014996.CrossRefGoogle Scholar
  22. Poropat, A. E. (2014). A meta-analysis of adult-rated child personality and academic performance in primary education. British Journal of Educational Psychology, 84(2), 239–252.  https://doi.org/10.1111/bjep.12019.CrossRefGoogle Scholar
  23. Primi, R., De Fruyt, F., Santos, D., Antonoplis, S. & John, O. P. (2018). True or False? Keying direction and acquiescence influence the validity of socio-emotional skills items in predicting high school achievement. Submitted paper under review.Google Scholar
  24. Primi, R., Santos, D., De Fruyt, F., & John, O. P. (2019). Comparison of classical and modern methods for measuring and correcting for acquiescence. British Journal of Mathematical and Statistical Psychology.Google Scholar
  25. Primi, R., Santos, D., John, O. P., & De Fruyt, F. D. (2016). Development of an inventory assessing social and emotional skills in Brazilian youth. European Journal of Psychological Assessment, 32(1), 5–16.  https://doi.org/10.1027/1015-5759/a000343.CrossRefGoogle Scholar
  26. Savalei, V., & Falk, C. F. (2014a). Recovering substantive factor loadings in the presence of acquiescence bias: A comparison of three approaches. Multivariate Behavioral Research, 49(5), 407–424.  https://doi.org/10.1080/00273171.2014.931800.CrossRefGoogle Scholar
  27. Samuel, D. B., & Widiger, T. A. (2008). A meta-analytic review of the relationships between the five-factor model and DSM-IV-TR personality disorders: a facet level analysis. Clinical Psychology Review, 28(8), 1326–1342.  https://doi.org/10.1016/j.cpr.2008.07.002.CrossRefGoogle Scholar
  28. Savalei, V., & Falk, C. F. (2014b). Recovering substantive factor loadings in the presence of acquiescence bias: A comparison of three approaches. Multivariate Behavioral Research, 49, 407–424.  https://doi.org/10.1080/00273171.2014.931800.CrossRefGoogle Scholar
  29. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The developmental psychometrics of big five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Journal of Personality and Social Psychology, 94(4), 718–737.  https://doi.org/10.1037/0022-3514.94.4.718.CrossRefGoogle Scholar
  30. Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100(2), 330–348.  https://doi.org/10.1037/a0021717.CrossRefGoogle Scholar
  31. Soto, C. J., & John, O. P. (2019). Optimizing the length, width, and balance of a personality scale: How do internal characteristics affect external validity? Psychological Assessment, 31, 586–590.https://doi.org/10.1037/pas0000586.CrossRefGoogle Scholar
  32. Ten Berge, J. M. (1999). A legitimate case of component analysis of ipsative measures, and partialling the mean as an alternative to ipsatization. Multivariate Behavioral Research, 34(1), 89–102.  https://doi.org/10.1207/s15327906mbr3401_4.CrossRefGoogle Scholar
  33. Valentini, F. (2017). Editorial: Influência e controle da aquiescência na análise fatorial [Editorial: Acquiescence and factor analysis]. Avaliação Psicológica, 16, 120–121.  https://doi.org/10.15689/ap.2017.1602.CrossRefGoogle Scholar
  34. Weijters, B., Geuens, M., & Schillewaert, N. (2010). The stability of individual response styles. Psychological Methods, 15(1), 96–110.  https://doi.org/10.1037/a0018721.CrossRefGoogle Scholar
  35. Wetzel, E., Lüdtke, O., Zettler, I., & Böhnke, J. R. (2015). The Stability of extreme response style and acquiescence over 8 years. Assessment.  https://doi.org/10.1177/1073191115583714.CrossRefGoogle Scholar
  36. Zhang, J., & Ziegler, M. (2018). Why do personality traits predict scholastic performance? A three-wave longitudinal study. Journal of Research in Personality, 74, 182–193.  https://doi.org/10.1016/j.jrp.2018.04.006.CrossRefGoogle Scholar
  37. Ziegler, M. (2015). “F*** You, I Won’t Do What You Told Me!”—Response biases as threats to psychological assessment. European Journal of Psychological Assessment, 31(3), 153–158.  https://doi.org/10.1027/1015-5759/a000292.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ricardo Primi
    • 1
    Email author
  • Nelson Hauck-Filho
    • 2
  • Felipe Valentini
    • 2
  • Daniel Santos
    • 1
  • Carl F. Falk
    • 3
  1. 1.Ayrton Senna InstituteUniversidade São Francisco, and EduLab21CampinasBrazil
  2. 2.Universidade São FranciscoCampinasBrazil
  3. 3.McGill UniversityMontrealCanada

Personalised recommendations