Skip to main content
SpringerLink
Log in

GDM Software mdltm Including Parallel EM Algorithm

  • Chapter
  • First Online:
Handbook of Diagnostic Classification Models

Part of the book series: Methodology of Educational Measurement and Assessment ((MEMA))

Abstract

This chapter illustrates the use of the software mdltm (von Davier, A general diagnostic model applied to language testing data. ETS Research Report No. RR-05-16, Educational Testing Service, Princeton, 2005), for multidimensional discrete latent trait models. The software mdltm was designed to handle large data sets as well as complex test and sampling designs, providing high flexibility for operational analyses. It allows the estimation of many different latent variable models, includes different constraints for parameter estimation, and provides different model and item fit statistics as well as multiple methods for proficiency estimation. The software utilizes an computationally efficient parallel EM algorithm (von Davier, New results on an improved parallel EM algorithm for estimating generalized latent variable models. In van der Ark L, Wiberg M, Culpepper S, Douglas J, Wang WC (eds) Quantitative psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, New York, 2017) that allows estimation of high-dimensional diagnostic models for very large datasets. The software is illustrated by applying diagnostic models to data from the programme for international student assessment (PISA).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    PISA is a major international academic student survey that assesses the proficiencies of 15-year-old school populations (students in grade 7 or higher) in the domains of mathematics, reading, and science (sometimes accompanied by additional cognitive domains of interest such as collaborative problem solving and financial literacy). PISA is administered every 3 years since 2000 with the aim of monitoring students’ ability to use their knowledge and skills for meeting real-life challenges and to provide trend measures over time. In each cycle, one of the three domains is featured as major domain and consists of trend and new items, while the others serve as minor domains and consist of trend items only.

  2. 2.

    Plausible values are multiple imputations drawn from a posterior distribution obtained from a latent regression model (also referred to as population modeling or conditioning model) using IRT item parameters from the cognitive PISA assessment and principal components from the PISA Background Questionnaire. In PISA, each respondent receives 10 plausible values for each cognitive domain that can be used as test scores to produce group level statistics (never as individual test scores). For more information on plausible values and population modeling in large-scale assessments, see Mislevy and Sheehan (1987), von Davier, Gonzalez and Mislevy (2009), von Davier, Sinharay, Oranje, and Beaton (2006) or Yamamoto, Khorramdel, and von Davier (2013, updated 2016).

  3. 3.

    Note that decimals in the category frequency counts are due to the use of sample weights in the analyses.

  4. 4.

    For the details about adjacent category logit, including various types of parameterization for the polytomous responses, please refer to Agresti (2002).

  5. 5.

    The expected category frequencies (for multiple groups) and conditional proportions correct P(+|group) are statistics given separately for each group (e.g. a state, country or language). For latent class models, mixture IRT models and diagnostic models, the expected category frequencies are expected proportions correct per latent class, which are estimates of these proportions, given the classifications of respondents (proportionally assigned using posterior distribution of class membership given observed responses) into these classes.

References

  • Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23.

    Article  Google Scholar 

  • Agresti, A. (2002). Categorical data analysis. Hoboken, NJ: Wiley.

    Book  Google Scholar 

  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.

    Article  Google Scholar 

  • Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Cai, L. (2010a). High-dimensional exploratory item factor analysis by a Metropolis–Hastings Robbins–Monro algorithm. Psychometrika, 75, 33–57.

    Article  Google Scholar 

  • Cai, L. (2010b). Metropolis–Hastings Robbins–Monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics, 35, 307–335.

    Article  Google Scholar 

  • Gibbons, R. D., & Hedeker, D. (1992). Full-information item bi-factor analysis. Psychometrika, 57, 423–436.

    Article  Google Scholar 

  • Gilula, Z., & Haberman, S. J. (1994). Models for analyzing categorical panel data. Journal of the American Statistical Association, 89, 645–656.

    Article  Google Scholar 

  • Haberman, S. J., von Davier, M., & Lee, Y. (2008). Comparision of multidimensional item response models: Multivariate normal ability distributions versus multivariate polytomous ability distributions. ETS Research Report Series (pp. 1–25). https://doi.org/10.1002/j.2333-8504.2008.tb02131.x

  • Jeon, M., & Rijmen, F. (2014). Recent developments in maximum likelihood estimation of MTMM models for categorical data. Frontiers in Psychology, 5, 269. https://doi.org/10.3389/fpsyg.2014.00269

    Article  Google Scholar 

  • Jeon, M., Rijmen, F., & Rabe-Hesketh, S. (2013). Modeling differential item functioning using the multiple-group bifactor model. Journal of Educational and Behavioral Statistics, 38, 32–60.

    Article  Google Scholar 

  • Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.

    Article  Google Scholar 

  • Mazzeo, J., & von Davier, M. (2008). Review of the Programme for International Student Assessment (PISA) test design: Recommendations for fostering stability in assessment results (OECD Working Paper EDU/PISA/GB (2008) 28). Paris, France: OECD. Retrieved from https://edsurveys.rti.org/pisa/documents/mazzeopisa_test_designreview_6_1_09.pdf

    Google Scholar 

  • Mazzeo, J., & von Davier, M. (2013). Linking scales in international large-scale assessments. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. Boca Raton, FL: CRC Press.

    Google Scholar 

  • Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In A. E. Beaton (Ed.), Implementing the new design: The NAEP 1983–84 technical report (Report No. 15-TR-20). Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–177.

    Article  Google Scholar 

  • OECD. (2016). PISA 2015 Assessment and analytical framework: Science, reading, mathematic and financial Literacy. Paris, France: PISA, OECD Publishing.

    Book  Google Scholar 

  • Organisation for Economic Co-Operation and Development. (2013). Chapter 17: Technical report of the Survey of Adult Skills (PIAAC) (pp. 406–438). Retrieved from the OECD website: http://www.oecd.org/site/piaac/Technical%20Report_17OCT13.pdf

  • Organisation for Economic Co-Operation and Development. (2017). PISA 2015 technical report. Paris, France: OECD Publishing.

    Google Scholar 

  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Nielsen & Lydiche (Expanded Edition, Chicago, University of Chicago Press, 1980).

    Google Scholar 

  • Rijmen, F., & Jeon, M. (2013). Fitting an item response theory model with random item effects across groups by a variational approximation method. Annals of Operations Research, 206, 647–662.

    Article  Google Scholar 

  • Rijmen, F., Jeon, M., Rabe-Hesketh, S., & von Davier, M. (2014). A third order item response theory model for modeling the effects of domains and subdomains in large-scale educational assessment surveys. Journal of Educational and Behavioral Statistics, 38, 32–60.

    Google Scholar 

  • Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39(2), 142–151.

    Article  Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.

    Article  Google Scholar 

  • von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Report No. RR-05-16). Princeton, NJ: Educational Testing Service.

    Book  Google Scholar 

  • von Davier, M. (2008). The mixture general diagnostic model. In G. R. Hancock & K. M. Samuelson (Eds.), Advances in latent variable mixture models. Information Age Publishing.

    Google Scholar 

  • von Davier, M. (2010). Hierarchical mixtures of diagnostic models. Psychological Test and Assessment Modeling, 52, 8–28.

    Google Scholar 

  • von Davier, M. (2013). The DINA model as a constrained general diagnostic model – Two variants of a model equivalency. British Journal of Mathematical and Statistical Psychology, 67(1), 49–71. https://doi.org/10.1111/bmsp.12003

    Article  Google Scholar 

  • von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM) (Research Report No. ETS RR-14-40). Princeton, NJ: Educational Testing Service. https://doi.org/10.1002/ets2.12043

    Book  Google Scholar 

  • von Davier, M. (2016). High-performance psychometrics: The Parallel-E Parallel-M algorithm for generalized latent variable models. ETS Research Report Series ISSN, 2016, 2330–8516.

    Google Scholar 

  • von Davier, M. (2017). New results on an improved parallel EM algorithm for estimating generalized latent variable models. In L. van der Ark, M. Wiberg, S. Culpepper, J. Douglas, & W. C. Wang (Eds.), Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics (Vol. 196). New York, NY: Springer.

    Google Scholar 

  • von Davier, M., & Carstensen, C. H. (2006). Multivariate and mixture distribution rasch models: Extensions and applications. New York, NY: Springer.

    Google Scholar 

  • von Davier, M., Gonzalez, E. & Mislevy, R. (2009) What are plausible values and why are they useful? In IERI Monograph series: Issues and methodologies in large scale Assessments, vol. 2. Retrieved from: http://www.ierinstitute.org/fileadmin/Documents/IERI_Monograph/IERI_Monograph_Volume_02_Chapter_01.pdf

  • von Davier, M., González, J. B., & von Davier, A. A. (2013). Local equating using the Rasch Model, the OPLM, and the 2PL IRT Model—or—What is it anyway if the model captures everything there is to know about the test takers? Journal of Educational Measurement, 50(3), 295–303. https://doi.org/10.1111/jedm.12016

    Article  Google Scholar 

  • von Davier, M., & Rost, J. (2016). Logistic mixture-distribution response models. In W. van der Linden (Ed.), Handbook of item response theory (Vol. 1, 2nd ed., pp. 393–406). Boca Raton, FL: CRC Press.

    Google Scholar 

  • von Davier, M., Sinharay, S., Oranje, A., & Beaton, A. (2006). Statistical procedures used in the National Assessment of Educational Progress (NAEP): Recent developments and future directions. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26): Psychometrics. Amsterdam, The Netherlands: Elsevier.

    Google Scholar 

  • von Davier, M., & von Davier, A. (2007). A unified approach to IRT scale linking and scale transformations. Methodology, 3(3), 115–124.

    Article  Google Scholar 

  • Xu, X. & von Davier, M. (2008a). Linking with the General Diagnostic Model. ETS Research Report No. RR-08-08, Princeton, NJ: Educational Testing Service. http://onlinelibrary.wiley.com/doi/10.1002/j.2333-8504.2008.tb02094.x/full

  • Xu, X. & von Davier, M. (2008b). Fitting the structured general diagnostic model to NAEP data. ETS Research Report No. RR-08-27, Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Xu, X. & von Davier, M. (2008c). Comparing multiple-group multinomial loglinear models for multidimensional skill distributions in the general diagnostic model. ETS Research Report No. RR-08-35, Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Yamamoto, K., Khorramdel, L., & von Davier, M. (2013, updated 2016). Chapter 17: Scaling PIAAC cognitive data. In OECD (2013), Technical Report of the Survey of Adult Skills (PIAAC) (pp. 406–438), PIAAC, OECD Publishing. Retrieved from http://www.oecd.org/site/piaac/All%20PIACC%20Technical%20Report%20final.pdf

Download references

Author information

Authors and Affiliations

  1. National Board of Medical Examiners (NBME), Philadelphia, PA, USA

    Lale Khorramdel & Matthias von Davier

  2. Educational Testing Service, Princeton, NJ, USA

    Hyo Jeong Shin

Authors
  1. Lale Khorramdel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyo Jeong Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthias von Davier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthias von Davier .

Editor information

Editors and Affiliations

  1. National Board of Medical Examiners (NBME), Philadelphia, PA, USA

    Matthias von Davier

  2. Teachers College, Columbia University, New York, NY, USA

    Young-Sun Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Khorramdel, L., Shin, H.J., von Davier, M. (2019). GDM Software mdltm Including Parallel EM Algorithm. In: von Davier, M., Lee, YS. (eds) Handbook of Diagnostic Classification Models. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-030-05584-4_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05584-4_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05583-7

  • Online ISBN: 978-3-030-05584-4

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation