GPU-Accelerated Computing with Gibbs Sampler for the 2PNO IRT Model

  • Yanyan ShengEmail author
  • William S. Welling
  • Michelle M. Zhu
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)


Item response theory (IRT) is a popular approach used for addressing large-scale statistical problems in psychometrics as well as in other fields. The fully Bayesian approach for estimating IRT models is usually memory and computational expensive due to the large number of iterations. This limits the use of the procedure in many applications. In an effort to overcome such restrictions, previous studies proposed to tackle the problem using massive core-based graphic processing units (GPU), and demonstrated the advantage of this approach over the message passing interface (MPI) by showing that a single GPU card could achieve a speedup of up to 50×. Given that GPU is practical, cost-effective, and convenient, this study aims to seek further improvements using a single GPU card.


Item response theory Bayesian estimation MCMC Two-parameter IRT model High performance computing CUDA Optimization 


  1. Albert, J. H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational Statistics, 17(3), 251–269.CrossRefGoogle Scholar
  2. Bafumi, J., Gelman, A., Park, D. K., & Kaplan, N. (2005). Practical issues in implementing and understanding Bayesian ideal point estimation. Political Analysis, 13(2), 171–187.CrossRefGoogle Scholar
  3. Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques. (2nd ed.). New York: Dekker.Google Scholar
  4. Beseler, C. L., Taylor, L. A., & Leeman, R. F. (2010). An item-response theory analysis of DSM-IV alcohol-use disorder criteria and “binge” drinking in undergraduates. Journal of Studies on Alcohol and Drugs, 71(3), 418–423.CrossRefGoogle Scholar
  5. Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6(2), 258–276.CrossRefzbMATHGoogle Scholar
  6. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459.MathSciNetCrossRefGoogle Scholar
  7. Courvoisier, D., & Etter, J. F. (2008). Using item response theory to study the convergent and discriminant validity of three questionnaires measuring cigarette dependence. Psychology of Addictive Behaviors, 22(3), 391–401.CrossRefGoogle Scholar
  8. DiMarco, J., & Taufer. M. (2013). Performance impact of dynamic parallelism on different clustering algorithms. In SPIE Defense, Security, and Sensing, International Society for Optics and Photonics.Google Scholar
  9. Feske, U., Kirisci, L., Tarter, R. E., & Plkonis, P. A. (2007). An application of item response theory to the DSM-III-R criteria for borderline personality disorder. Journal of Personality Disorders, 21(4), 418–433.CrossRefGoogle Scholar
  10. Fienberg, S. E., Johnson, M. S., & Junker, B. W. (1999). Classical multilevel and Bayesian approaches to population size estimation using multiple lists. Journal of the Royal Statistical Society, Series A, 162(3), 383–392.CrossRefGoogle Scholar
  11. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–741.CrossRefzbMATHGoogle Scholar
  12. Georganas, E. (2013). High performance parallel Gibbs sampling for IRT models. Poster session presented at ParLab Winter Retreat, Berkeley, USA.Google Scholar
  13. Gilder, D. A., Gizer, I. R., & Ehlers, C. L. (2011). Item response theory analysis of binge drinking and its relationship to lifetime alcohol use disorder symptom severity in an American Indian community sample. Alcoholism: Clinical and Experimental Research, 35(5), 984–995.CrossRefGoogle Scholar
  14. Harris, M. (2007). Optimizing parallel reduction in CUDA. Presentation packaged with CUDA Toolkit, NVIDIA Corporation.Google Scholar
  15. Harwell, M., Stone, C. A., Hsu, H., & Kirisci, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20(2), 101–126.CrossRefGoogle Scholar
  16. Hoberock, J., & Bell, N. (2010). Thrust: A parallel template library.
  17. Karunadasa, N. P., & Ranasinghe, D. N. (2009). Accelerating high performance applications with CUDA and MPI. In 2009 International Conference on Industrial and Information Systems (ICIIS) (pp. 331–336).Google Scholar
  18. Kirk, D. B., & Hwu, W. W. (2013). Programming massively parallel processors: A hands-on approach (2nd ed.). Burlington, MA: Addison-Wesley.Google Scholar
  19. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Boston: Addison-Wesley.zbMATHGoogle Scholar
  20. Luitjens, J. (2014). Faster parallel reduction on Kepler. Retrieved from Scholar
  21. Martin, C. S., Chung, T., Kirisci, L., & Langenbucher, J. W. (2006). Item response theory analysis of diagnostic criteria for alcohol and cannabis use disorders in adolescents: Implications for DSM-V. Journal of Abnormal Psychology, 115(4), 807–814.CrossRefGoogle Scholar
  22. Mislevy, R. J. (1985). Estimation of latent group effects. Journal of the American Statistical Association, 80(392), 993–997.MathSciNetCrossRefGoogle Scholar
  23. Molenaar, I. W. (1995). Estimation of item parameters. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 39–51). New York: Springer.CrossRefGoogle Scholar
  24. NVIDIA. (2010). CUDA CURAND library. Santa Clara, CA: NVIDIA Corporation.Google Scholar
  25. Oancea, B., & Andrei, T. (2013). Developing a high performance software library with MPI and CUDA for matrix computations. Computational Methods in Social Sciences, 1(2), 1–10.Google Scholar
  26. Orlando, M., Sherbourne, C. D., & Thissen, D. (2000). Summed-score linking using item response theory: Application to depression measurement. Psychological Assessment, 12(3), 354–359.CrossRefGoogle Scholar
  27. Panter, A. T., & Reeve, B. B. (2002). Assessing tobacco beliefs among youth using item response theory models. Drug and Alcohol Dependence, 68(1), 21–39.CrossRefGoogle Scholar
  28. Pastias, K., Rahimi, M., Sheng, Y., & Rahimi, S. (2012). Parallel computing with a Bayesian item response model. American Journal of Computational Mathematics, 2(2), 65–71.CrossRefGoogle Scholar
  29. Patz, R. J., & Junker, B. (1999). A straightforward approach to Markov chain Monte Carlo methods for item response model. Journal of Educational and Behavioral Statistics, 24(2), 146–178.CrossRefGoogle Scholar
  30. Reiser, M. (1989). An application of the item-response model to psychiatric epidemiology. Sociological Methods and Research, 18(1), 66–103.MathSciNetCrossRefGoogle Scholar
  31. Rose, J. S., & Dierker, L. C. (2010). An item response theory analysis of nicotine dependence symptoms in recent onset adolescent smokers. Drug and Alcohol Dependence, 110(12), 70–79.CrossRefGoogle Scholar
  32. Sheng, Y., & Headrick, T. C. (2007). An algorithm for implementing Gibbs sampling for 2PNO IRT models. Journal of Modern Applied Statistical Methods, 6(1), 341–349.Google Scholar
  33. Sheng, Y., & Rahimi, M. (2012). High performance Gibbs sampling for IRT models using row-wise decomposition. ISRN Computational Mathematics, 2012(264040), 1–9.CrossRefGoogle Scholar
  34. Sheng, Y., Welling, W. S., & Meng, M. M. (2014). A GPU-based Gibbs sampler for a unidimensional IRT model. ISRN Computational Mathematics, 2014 (368149), 1–11.Google Scholar
  35. Smith, A. F. M., & Roberts, G. O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods (with discussion). Journal of the Royal Statistical Society, Series B, 55(1), 3–23.MathSciNetzbMATHGoogle Scholar
  36. Tierney, L. (1994). Markov chains for exploring posterior distributions. The Annals of Statistics, 22(4), 1701–1728.MathSciNetCrossRefzbMATHGoogle Scholar
  37. Tsutsumi, A., Iwata, N., Watanabe, N., de Jonge, J., Pikhart, H., Fernndez-Lpez, J. A., et al. (2009). Application of item response theory to achieve cross-cultural comparability of occupational stress measurement. International Journal of Methods in Psychiatric Research, 18(1), 58–67.CrossRefGoogle Scholar
  38. Tutakawa, R. K., & Lin, H. Y. (1986). Bayesian estimation of item response curves. Psychometrika, 51(2), 251–267.MathSciNetCrossRefGoogle Scholar
  39. Yang, C.-T., Huang, C.-L., & Lin, C.-F. (2011). Hybrid CUDA, OpenMP, and MPI parallel programming on multicore GPU clusters. Computer Physics Communications, 182, 266–269.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Yanyan Sheng
    • 1
    Email author
  • William S. Welling
    • 2
  • Michelle M. Zhu
    • 2
  1. 1.Quantitative Methods, Department of Counseling, Quantitative Methods & Special EducationSouthern Illinois UniversityCarbondaleUSA
  2. 2.Department of Computer ScienceSouthern Illinois UniversityCarbondaleUSA

Personalised recommendations