Abstract
This study examines the relationship between rating scale format and instrument sensitivity to response style (RS). A multidimensional nominal response model (MNRM) that allows item discrimination to vary across items for both substantive and RS dimensions can be used for this purpose. We first conduct a set of simulations to examine the recovery of item slopes under the model. Then we apply the model to PISA and PIRLS survey items to investigate the item sensitivity to RS in relation to different rating scale types. Simulation results indicate good recovery of item slopes on both substantive and RS traits, and different rating scale formats (agreement-type versus frequency-type scales) from PISA and PIRLS are found to have varying sensitivities to RS, such that frequency-type scales are less affected.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
H. Baumgartner, J.B.E. Steenkamp, Response styles in marketing research: a cross-national investigation. J. Mark. Res. 38 (2), 143–156 (2001)
R.D. Bock, Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika 37 (1), 29–51 (1972)
D.M. Bolt, T.R. Johnson, Applications of a MIRT model to self-report measures: addressing score bias and DIF due to individual differences in response style. Appl. Psychol. Meas. 33 (5), 335–352 (2009)
D.M. Bolt, J.R. Newton, Multiscale measurement of extreme response style. Educ. Psychol. Meas. 71 (5), 814–833 (2011)
L. Cai, High-dimensional exploratory item factor analysis by a Metropolis–Hastings Robbins–Monro algorithm. Psychometrika 75 (1), 33–57 (2010a)
L. Cai, Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. J. Educ. Behav. Stat. 35 (3), 307–335 (2010b)
R.P. Chalmers, mirt: a multidimensional item response theory package for the R environment. J. Stat. Softw. 48 (6), 1–29 (2012)
J. Cohen, Statistical Power Analysis for the Behavior Science, 2nd edn. (Lawrence Erlbaum, Hillsdale, 1988)
M.G. De Jong, J.B.E. Steenkamp, J.P. Fox, H. Baumgartner, Using item response theory to measure extreme response style in marketing research: a global investigation. J. Mark. Res. 45 (1), 104–115 (2008)
C.F. Falk, L. Cai, A flexible full-information approach to the modeling of response styles. Pyschol. Methods 21 (3), 328–347 (2015)
F.J. Fowler, Improving Survey Questions: Design and Evaluation. Applied Social Research Methods Series, vol. 38 (Sage, Thousand Oaks, 1995)
T.R. Johnson, D.M. Bolt, On the use of factor-analytic multinomial logit item response models to account for individual differences in response style. J. Educ. Behav. Stat. 35 (1), 92–114 (2010)
Y. Lu, A multilevel multidimensional item response theory model to address the role of response style on measurement of attitudes in PISA 2006. PhD thesis, University of Wisconsin–Madison, Madison WI (2012)
Y. Lu, D.M. Bolt, Examining the attitude-achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style. Large Scale Assess. Educ. 3 (1), 1–18 (2015)
G. Meisenberg, A. Williams, Are acquiescent and extreme response styles related to low intelligence and education? Personal. Individ. Differ. 44 (7), 1539–1550 (2008)
G. Moors, The effect of response style bias on the measurement of transformational, transactional, and laissez-faire leadership. Eur. J. Work Organ. Psychol. 21 (2), 271–298 (2012)
H. Plieninger, Mountain or molehill? A simulation study on the impact of response styles. Educ. Psychol. Meas. 77 (1), 32–53 (2016)
D. Thissen, L. Steinberg, A taxonomy of item response models. Psychometrika 51 (4), 567–577 (1986)
Y. Van Vaerenbergh, T.D. Thomas, Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Publ. Opin. Res. 25 (2), 195–217 (2013)
B. Weijters, M. Geuens, N. Schillewaert, The stability of individual response styles. Psychol. Methods 15 (1), 96–110 (2010)
E. Wetzel, J.R. Böhnke, N. Rose, A simulation study on methods of correcting for the effects of extreme response style. Educ. Psychol. Meas. 76 (2), 304–324 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Deng, S., Bolt, D.M. (2017). Rating Scale Format and Item Sensitivity to Response Style in Large-Scale Assessments. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_30
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)