Skip to main content
SpringerLink
Log in

Item Response Theory Models for Multidimensional Ranking Items

  • Conference paper
  • First Online:
Book cover Quantitative Psychology Research

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 167))

Abstract

Multidimensional ranking items, in which different statements aim at different latent traits, are commonly used to measure noncognitive latent traits (e.g., career interests, attitudes, and personality). In this study, we developed two new item response theory models for multidimensional ranking items that yield statement utilities and person measures. Simulations were conducted to evaluate the parameter recovery of the two new models, and the results indicated that the parameters were recovered well by using the freeware Just Another Gibson Sampler (JAGS). Anempirical example of behaviors in workplaces was provided.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.00
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aitchison, J. A. (1986). The statistical analysis of compositional data. London, UK: The Blackburn Press.

    Book  MATH  Google Scholar 

  • Allison, P. D., & Christakis, N. A. (1994). Logit models for sets of ranked items. Sociological Methodology, 24, 199–228.

    Article  Google Scholar 

  • Barrett, L. F. (2004). Feelings or words? Understanding the content in self-report ratings of experienced emotion. Journal of Personality and Social Psychology, 87, 266–281.

    Article  Google Scholar 

  • Barrett, L. F., Gross, J., Christensen, T. C., & Benvenuto, M. (2001). Knowing what you are feeling and knowing what to do about it: Mapping the relation between emotion differentiation and emotion regulation. Cognition and Emotion, 15, 713–724.

    Article  Google Scholar 

  • Beggs, S., Cardell, S., & Hausman, J. (1981). Assessing the potential demand for electric cars. Journal of Econometrics, 17, 1–19.

    Article  Google Scholar 

  • Böckenholt, U. (2004). Comparative judgments as an alternative to ratings: Identifying the scale origin. Psychological Methods, 9, 453–465.

    Article  Google Scholar 

  • Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs I: The method of paired comparisons. Biometrika, 39, 324–345.

    MathSciNet  MATH  Google Scholar 

  • Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71, 460–502.

    Article  Google Scholar 

  • Brown, A., & Maydeu-Olivares, A. (2012). Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behavioral Research Methods, 44, 1135–1147.

    Article  Google Scholar 

  • Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18, 36–52.

    Article  Google Scholar 

  • Cattell, R. B. (1944). Psychological measurement: Normative, ipsative, interactive. Psychological Review, 51, 292–303.

    Article  Google Scholar 

  • Chapman, R. G., & Staelin, R. (1982). Exploiting rank ordered choice set data within the stochastic utility model. Journal of Marketing Research, 14, 288–301.

    Article  Google Scholar 

  • Chen, C.-W., & Wang, W.-C. (2013a, April). Item response theory models for ipsative tests. Paper presented at the annual meeting of National Council on Measurement in Education, San Francisco, CA.

    Google Scholar 

  • Chen, C.-W., & Wang, W.-C. (2013b, July). Computerized adaptive testing under the Rasch model for ipsative forced-choice items. Paper presented at the 78th annual meeting of the Psychometric Society, Arnhem, Netherlands.

    Google Scholar 

  • Chen, C.-W., & Wang, W.-C. (2014, April). Detecting differential statement functioning in ipsative tests using the logistic regression method. Paper presented at the annual meeting of National Council on Measurement in Education, Philadelphia, PA.

    Google Scholar 

  • Chen, C.-W., Wang, W.-C., & Ro, S. (2015a, July). A quick item selection method in computerized adaptive testing for ranking items. Paper presented at the 80th annual meeting of the Psychometric Society, Beijing, China.

    Google Scholar 

  • Chen, C.-W., Wang, W.-C., & Ro, S. (2015b, August). Controlling within-person exposure in computerized adaptive testing for ranking items. Paper presented at the Pacific Rim Objective Measurement Symposium, Fukuoka, Japan.

    Google Scholar 

  • De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer.

    MATH  Google Scholar 

  • de Vries, A. L. M., & van der Ark, A. (2008). Scoring methods for ordinal multidimensional forced-choice items. In J. Daunis-i-Estadella, & J. A. Martín-Fernández (Eds.), Proceedings of the 3rd Compositional Data Analysis Workshop Codawork ’08 (pp. 1–18). Girona, Spain: University of Girona.

    Google Scholar 

  • Dunlap, W. P., & Cornwell, J. M. (1994). Factor analysis of ipsative measures. Multivariate Behavioral Research, 29, 115–126.

    Article  Google Scholar 

  • Gelman, A., Meng, X.-L., & Stern, H. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733–807.

    MathSciNet  MATH  Google Scholar 

  • Hausman, J. A., & Ruud, P. A. (1987). Specifying and testing econometric models for ranked-ordered data. Journal of Econometrics, 34, 83–104.

    Article  MathSciNet  MATH  Google Scholar 

  • Hirschi, A. (2009). Development and criterion validity of differentiated and elevated vocational interests in adolescence. Journal of Career Assessment, 17, 384–401.

    Article  Google Scholar 

  • Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Kashdan, T. B., Barrett, L. F., & McKnight, P. E. (2015). Unpacking emotion differentiation: Transforming unpleasant experience by perceiving distinctions in negativity. Current Direction in Psychological Science, 24, 10–16.

    Article  Google Scholar 

  • Kopelman, R. E., Rovenpor, J. L., & Guan, M. (2003). The study of values: Construction of the fourth edition. Journal of Vocational Behavior, 62, 203–220.

    Article  Google Scholar 

  • Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. New York, NY: Wiley.

    MATH  Google Scholar 

  • Maydeu-Olivares, A., & Böckenholt, U. (2005). Structural equation modeling of paired-comparison and ranking data. Psychological Methods, 10, 285–304.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Brown, A. (2010). Item response modeling of paired comparison and ranking data. Multivariate Behavioral Research, 45, 935–974.

    Article  Google Scholar 

  • Meade, A. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531–552.

    Article  Google Scholar 

  • Mokken, R. J. (1971). A theory and procedure of scale analysis with applications in political research. Berlin: Walter de Gruyter, Mouton.

    Book  Google Scholar 

  • Plummer, M. (2003, March). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Paper presented at the 3rd International Workshop on Distributed Statistical Computing, Vienna, Austria.

    Google Scholar 

  • Punj, G. N., & Staelin, R. (1978). The choice process for graduate business schools. Journal of Marketing Research, 15, 588–598.

    Article  Google Scholar 

  • Qiu, X.-L., & Wang, W.-C. (2014, April). Computerized adaptive testing for forced-choice ipsative items. Paper presented at the annual meeting of American Educational Research Association, Philadelphia, PA.

    Google Scholar 

  • Qiu, X.-L., Wang, W.-C., & Ro, S. (2015, April). An IRT model for multidimensional ranking data in ipsative tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.

    Google Scholar 

  • Salgado, J. F., & Tauriz, G. (2014). The five-factor model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies. European Journal of Work and Organizational Psychology, 23, 3–30.

    Article  Google Scholar 

  • Saville, P., Sik, G., Nyfield, G., Hackston, J., & MacIver, R. (1996). A demonstration of the validity of the Occupational Personality Questionnaire (OPQ) in the measurement of job competencies across time and in separate organizations. Applied Psychology, 45, 243–262.

    Article  Google Scholar 

  • SHL. (2006). OPQ32 technical manual. Thames Ditton: SHL Group. Retrieved from https://www.cebglobal.com/shl/uk/solutions/products/docs/OPQ_Fact_Sheet_CEB%20v1.pdf.

  • Skrondal, A., & Rabe-Hesketh, S. (2003). Multilevel logistic regression for polytomous data and rankings. Psychometrika, 68, 267–287.

    Article  MathSciNet  MATH  Google Scholar 

  • Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linden, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society B, 64, 583–640.

    Article  MathSciNet  MATH  Google Scholar 

  • Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 79, 281–299.

    Google Scholar 

  • Thurstone, L. L. (1931). Rank order as a psychological method. Journal of Experimental Psychology, 14, 187–201.

    Article  Google Scholar 

  • Trull, T. T., Lan, S. P., Koval, P., & Ebner-Priemer, U. W. (2015). Affective dynamics in psychopathology. Emotion Review, 7, 355–361.

    Article  Google Scholar 

  • Witkin, H. A., Goodenough, D. R., & Oltman, P. K. (1979). Psychological differentiation: Current status. Journal of Personality and Social Psychology, 37, 1127–1145.

    Article  Google Scholar 

Download references

Acknowledgement

The study was supported by the General Research Fund, Hong Kong Research Grants Council (No. 845013).

Author information

Authors and Affiliations

  1. Department of Psychological Studies, The Education University of Hong Kong, 10 Lo Ping Road, Tai Po, New Territories, Hong Kong

    Wen-Chung Wang, Xuelan Qiu & Chia-Wen Chen

  2. IBM, 650 Third Avenue South, Minneapolis, MN, 55402, USA

    Sage Ro

Authors
  1. Wen-Chung Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuelan Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chia-Wen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sage Ro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Chung Wang .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    L. Andries van der Ark

  2. University of Wisconsin, Madison, Wisconsin, USA

    Daniel M. Bolt

  3. Education University of Hong Kong, Hong Kong, China

    Wen-Chung Wang

  4. University of Illinois, Champaign, Illinois, USA

    Jeffrey A. Douglas

  5. Umeå University, Umeå, Sweden

    Marie Wiberg

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Wang, WC., Qiu, X., Chen, CW., Ro, S. (2016). Item Response Theory Models for Multidimensional Ranking Items. In: van der Ark, L., Bolt, D., Wang, WC., Douglas, J., Wiberg, M. (eds) Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol 167. Springer, Cham. https://doi.org/10.1007/978-3-319-38759-8_5

Download citation

Publish with us

Policies and ethics

Search

Navigation