Advertisement

A Doubly Latent Space Joint Model for Local Item and Person Dependence in the Analysis of Item Response Data

Article
  • 24 Downloads

Abstract

Item response theory (IRT) is one of the most widely utilized tools for item response analysis; however, local item and person independence, which is a critical assumption for IRT, is often violated in real testing situations. In this article, we propose a new type of analytical approach for item response data that does not require standard local independence assumptions. By adapting a latent space joint modeling approach, our proposed model can estimate pairwise distances to represent the item and person dependence structures, from which item and person clusters in latent spaces can be identified. We provide an empirical data analysis to illustrate an application of the proposed method. A simulation study is provided to evaluate the performance of the proposed method in comparison with existing methods.

Keywords

latent space model multilayer network item response model local dependence cognitive assessment 

Notes

Acknowledgements

We appreciate the Editor, the Associate Editor, and three anonymous reviewers for their careful reading and detailed comments on the previous versions of our manuscript.

Supplementary material

11336_2018_9630_MOESM1_ESM.pdf (828 kb)
Supplementary material 1 (pdf 828 KB)

References

  1. Bishop, Y., Fienberg, S., & Holland, P. (1975). Discrete multivariate analysis. Cambridge: MIT Press.Google Scholar
  2. Borg, I., & Groenen, P. J. F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York, NY: Springer.Google Scholar
  3. Byrnes, J. P., & Overton, W. F. (1986). Reasoning about certainty and uncertainty in concrete, causal, and propositional context. Developmental Psychology, 22, 793–799.CrossRefGoogle Scholar
  4. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.CrossRefGoogle Scholar
  5. Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.CrossRefGoogle Scholar
  6. De Ayala, R. J., & Hertzog, M. A. (1991). The assessment of unidimensionality for use in item response theory. Multivariate Behavioral Research, 26, 765–792.CrossRefPubMedGoogle Scholar
  7. Draney, K. (2007). The Saltus model applied to proportional reasoning data. Journal of Applied Measurement, 8, 438–455.PubMedGoogle Scholar
  8. Draney, K., Wilson, M., Gluck, J., & Spiel, C. (2007). Mixture models in a developmental context. In R. Hancock & K. M. Samuelson (Eds.), Latent variable mixture models (pp. 199–216). Charlotte, NC: Information Age.Google Scholar
  9. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50, 195–212.CrossRefPubMedGoogle Scholar
  10. Evans, J. S. B. T., Newstead, S. E., & Byrne, R. M. J. (1993). Human reasoning: The psychology of deduction. Mahwah, NJ: Erlbaum.Google Scholar
  11. Fox, J., & Glas, C. A. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 271–288.CrossRefGoogle Scholar
  12. Friel, N., Rastelli, R., Wyse, J., & Raftery, A. E. (2016). Interlocking directorates in Irish companies using a latent space model for bipartite networks. Proceedings of the National Academy of Sciences of the United States of America, 113, 6629–6634.CrossRefPubMedPubMedCentralGoogle Scholar
  13. Glas, C. A., & Suarez-Falcon, J. C. (2003). A comparison of item-fit statistics for the three parameter logistic model. Applied Psychological Measurement, 27, 87–106.CrossRefGoogle Scholar
  14. Gollini, I., & Murphy, T. B. (2014). Mixture of latent trait analyzers for model-based clustering of categorical data. Statistics and Computing, 24, 569–588.CrossRefGoogle Scholar
  15. Gollini, I., & Murphy, T. B. (2016). Joint modeling of multiple network views. Journal of Computational and Graphical Statistics, 25, 246–265.CrossRefGoogle Scholar
  16. Handcock, M. S., Raftery, A. E., & Tantrum, J. M. (2007). Model-based clustering for social network. Journal of the Royal Statistical Society, Series A, 170, 301–354.CrossRefGoogle Scholar
  17. Hechenbichler, K., & Schliep, K. (2004). Weighted k-nearest-neighbor techniques and ordinal classification. Technical Report 399, Ludwig-Maximilians University, Munich.Google Scholar
  18. Hoff, P., Raftery, A., & Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97, 1090–1098.CrossRefGoogle Scholar
  19. Janveau-Brennan, G., & Markovits, H. (1999). The development of reasoning with causal conditionals. Developmental Psychology, 35, 904–911.CrossRefPubMedGoogle Scholar
  20. Kamata, A., & Bauer, D. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15, 136–153.CrossRefGoogle Scholar
  21. Krivitsky, P. N., Handcock, M. S., Raftery, A. E., & Hoff, P. D. (2009). Representing degree distributions, clustering, and homophily in social networks with latent cluster random network models. Social Networks, 31, 204–213.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Kruis, J., & Maris, G. (2016). Three representations of the ising model. Scientific Reports, 6(34175), 1–11.Google Scholar
  23. Liu, Y., & Maydeu-Olivares, A. (2012). Local dependence diagnostics in IRT modeling of binary data. Educational and Psychological Measurement, 73, 254–274.CrossRefGoogle Scholar
  24. Markovits, H., Fleury, M.-L., Quinn, S., & Venet, M. (1998). The development of conditional reasoning and the structure of semantic memory. Child Development, 69, 742–755.CrossRefPubMedGoogle Scholar
  25. Maydeu-Olivares, A., & Joe, H. (2005). Limited- and full-information estimation and goodness-of-fit testing in 2n contingency tables. Journal of the American Statistical Association, 100, 1009–1020.CrossRefGoogle Scholar
  26. Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732.CrossRefGoogle Scholar
  27. McDonald, R. P. (1982). Linear versus nonlinear models in item response theory. Applied Psychological Measurement, 6, 379–396.CrossRefGoogle Scholar
  28. Ng, A., Jordan, M., & Weiss, Y. (2002). On spectral clustering: Analysis and an algorithm. In T. Dietterich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural information processing systems (Vol. 14, pp. 849–856). Cambridge, MA: MIT Press.Google Scholar
  29. Oh, M.-S., & Raftery, A. (2001). Bayesian multidimensional scaling and choice of dimension. Journal of the American Statistical Association, 96, 1031–1044.CrossRefGoogle Scholar
  30. Overton, W. F. (1985). Scientific methodologies and the competence- moderator performance issue. In E. D. Neimark, R. de Lisi, & J. L. Newman (Eds.), Moderators of competence (pp. 15–41). Hillsdale: Erlbaum.Google Scholar
  31. Piaget, J. (1971). Biology and knowledge. Chicago: University of Chicago Press.Google Scholar
  32. Raftery, A., Niu, X., Hoff, P., & Yeung, K. (2012). Fast inference for the latent space network model using a case–control approximate likelihood. Journal of Computational and Graphical Statistics, 21, 909–919.CrossRefGoogle Scholar
  33. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.Google Scholar
  34. Rastelli, R., Friel, N., & Raftery, A. (2016). Properties of latent variable network models. Network Science, 4, 407–432.CrossRefGoogle Scholar
  35. Roberge, J. J., & Mason, E. J. (1978). Effects of negation on adolescents’ class and conditional reasoning abilities. The Journal of General Psychology, 98, 187–195.CrossRefPubMedGoogle Scholar
  36. Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282.CrossRefGoogle Scholar
  37. Shortreed, S., Handcock, M. S., & Hoff, P. (2006). Positional estimation within a latent space model for networks. Methodology, 2, 24–33.CrossRefGoogle Scholar
  38. Spiel, C., & Gluck, J. (2008). A model based test of competence profile and competence level in deductive reasoning. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts: State of the art and future prospects (pp. 41–60). Gottingen: Hogrefe.Google Scholar
  39. Spiel, C., Gluck, J., & Gossler, H. (2001). Stability and change of unidimensionality: The sample case of deductive reasoning. Journal of Adolescent Research, 16, 150–168.CrossRefGoogle Scholar
  40. Stout, W., Habing, B., Douglas, J., Kim, H., Roussos, L., & Zhang, J. (1996). Conditional covariance-based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331–354.CrossRefGoogle Scholar
  41. Takene, Y. (2007). Applications of multidimensional scaling in psychometrics. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26, pp. 359–400). Amsterdam: Elsevier.Google Scholar
  42. van Borkulo, C. D., Borsboom, D., Epskamp, S., Blanken, T. F., Boschloo, L., Schoevers, R. A., et al. (2014). A new method for constructing networks from binary data. Scientific Reports, 4(5918), 1–10.Google Scholar
  43. von Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17, 395–416.CrossRefGoogle Scholar
  44. Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–201.CrossRefGoogle Scholar
  45. Wilson, M., & Adams, R. J. (1995). Rasch models for item bundles. Psychometrika, 60, 181–198.CrossRefGoogle Scholar
  46. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three parameter logistic model. Applied Psychological Measurement, 8, 125–145.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2018

Authors and Affiliations

  1. 1.Department of Applied and Computational Mathematics and StatisticsUniversity of Notre DameNotre DameUSA
  2. 2.Graduate School of Education and Information StudiesUniversity of California, Los AngelesLos AngelesUSA

Personalised recommendations