Analyzing the Structure of Test Data

  • Mark D. Reckase
Part of the Statistics for Social and Behavioral Sciences book series (SSBS)


One of the common uses of MIRT is the analysis of the structure of the item response data that results from the administration of a set of test items to a sample of examinees. This type of analysis can be done in either an exploratory or confirmatory way. The exploratory analysis of item response data is used when either there is no clear hypothesis for the structure of the item response data, or when an unconstrained solution is seen as a strong test of the hypothesized structure. Confirmatory analyses require a clear hypothesis for the structure of the item response data. That is, there must be hypotheses about the number of coordinate dimensions needed to model the data and the relationship of the item characteristic surface to the coordinate axes. The types of confirmatory analyses that are typically done specify the relationship of the direction best measured by a test item (the direction of maximum discrimination) with the coordinate axes. It is also possible to have hypotheses about the difficulty of test items, but such hypotheses are seldom checked with the MIRT models. The difficulty parameters are typically left as parameters to be estimated without constraints.

In many cases, what is labeled as an exploratory analysis also has a confirmatory analysis component because the number of coordinate dimensions is selected prior to estimating the item and person-parameters. The resulting measures of fit of the model to the data are a test of a hypothesis about the number of dimensions. Because the number of coordinate dimensions is a critical component of both exploratory and confirmatory analyses, this issue will be addressed first. After consideration of the number-of-dimensions problem, procedures are described for determining the structure of item response data using exploratory and confirmatory procedures.


Test Item Item Response Item Parameter Confirmatory Analysis Story Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Apostle TM (1957) Mathematical analysis. Addison-Wesley, Reading, MAGoogle Scholar
  2. Bai J, Ng S (2002) Determining the number of factors in approximate factor models. Econometrica 70:191–221CrossRefMATHMathSciNetGoogle Scholar
  3. Haberman SJ (1977) Log-linear models and frequency tables with small expected cell counts. Annals of Statistics 5:1148–1169CrossRefMATHMathSciNetGoogle Scholar
  4. Harman HH (1976) Modern factor analysis (3rd edition revised). The University of Chicago Press, ChicagoGoogle Scholar
  5. Holzinger KJ, Harman HH (1941) Factor analysis: A synthesis of factorial methods. The University of Chicago Press, ChicagoGoogle Scholar
  6. Horn JL (1965) A rationale and test for the number of factors in factor analysis. Psychometrika 32:179–185CrossRefGoogle Scholar
  7. Jang EE, Roussos L (2007) An investigation into the dimensionality of TOEFL using conditional covariance-based nonparametric approach. Journal of Educational Measurement 44:1–21CrossRefMATHGoogle Scholar
  8. Kim HR (1994) New techniques for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation, University of Illinois, Champaign-Urbana, ILGoogle Scholar
  9. Kim J-P (2001) Proximity measures and cluster analyses in multidimensional item response theory. Unpublished doctoral dissertation, Michigan State University, East Lansing, MIGoogle Scholar
  10. Ledesma RD, Valero-Mora P (2007) Determining the number of factors to retain in EFA: an easy-to-use computer program for carrying out parallel analysis. Practical Assessment, Research & Evaluation 12:1–11Google Scholar
  11. Miller TR, Hirsch TM (1992) Cluster analysis of angular data in applications of multidimensional item response theory. Applied Measurement in Education 5:193–211CrossRefGoogle Scholar
  12. Mroch AA, Bolt DM (2006) A simulation comparison of parametric and nonparametric dimensionality detection procedures. Applied Measurement in Education 19:67–91CrossRefGoogle Scholar
  13. Reckase MD, Ackerman TA, Carlson JE (1988) Building a unidimensional test using multidimensional items. Journal of Educational Measurement 25:193–204CrossRefGoogle Scholar
  14. Reckase MD, Davey TC, Ackerman TA (1989) Similarity of the multidimensional space defined by parallel forms of a mathematics test. Paper presented at the meeting of the American Educational Research Association, San FranciscoGoogle Scholar
  15. Reckase MD, Stout W (1995) Conditions under which items that assess multiple abilities will be fit by unidimensional IRT models. Paper presented at the European meeting of the Psychometric Society, Leiden, The NetherlandsGoogle Scholar
  16. Reise SP, Waller NG, Comrey AL (2000) Factor analysis and scale revision. Psychological Assessment 12:287–297CrossRefGoogle Scholar
  17. Roussos LA, Ozbek OY (2006) Formulation of the DETECT population parameter and evaluation of DETECT estimation bias. Journal of Educational Measurement 43:215–243CrossRefGoogle Scholar
  18. Roussos LA, Stout WF, Marden JL (1998) Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement 35:1–30CrossRefGoogle Scholar
  19. Schilling S, Bock RD (2005) High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika 70:533–555MathSciNetGoogle Scholar
  20. Sokal RR, Michener CD (1958) A statistical method for evaluating systematic relationships. University of Kansas Science Bulletin 38:1409–1438Google Scholar
  21. Stout W (1987) A nonparametric approach for assessing latent trait dimensionality. Psychometrika 52:589–617CrossRefMATHMathSciNetGoogle Scholar
  22. Stout W, Douglas B, Junker B, Roussos L (1999) DIMTEST [Computer software]. The William Stout Institute for Measurement, Champaign, ILGoogle Scholar
  23. Tate R (2003) A comparison of selected empirical methods for assessing the structure of responses to test items. Applied Psychological Measurement 27:159–203CrossRefMathSciNetGoogle Scholar
  24. Thurstone LL (1947) Multiple-factor analysis: A development and expansion of The Vectors of Mind. The University of Chicago Press, ChicagoGoogle Scholar
  25. Wang M (1985) Fitting a unidimensional model to multidimensional item response data: The effect of latent space misspecification on the application of IRT (Research Report MW: 6-24-85). University of Iowa, Iowa City, IAGoogle Scholar
  26. Zhang JM, Stout W (1999a) Conditional covariance structure of generalized compensatory multidimensional items. Psychometrika 64:129–152CrossRefMathSciNetGoogle Scholar
  27. Zhang JM, Stout W (1999b) The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika 64:213–249CrossRefMathSciNetGoogle Scholar
  28. Zimowski MF, Muraki E, Mislevy RJ, Bock RD (2003) BILOG-MG for Windows. Scientific Software International, Inc., Lincolnwood, ILGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Counseling, Educational, Psychology, and Special Education DepartmentMichigan State UniversityEast LansingUSA

Personalised recommendations