Quality of Life Research

, Volume 16, Supplement 1, pp 187–194 | Cite as

Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment




We review the papers presented at the NCI/DIA conference, to identify areas of controversy and uncertainty, and to highlight those aspects of item response theory (IRT) and computer adaptive testing (CAT) that require theoretical or empirical research in order to justify their application to patient reported outcomes (PROs).


IRT and CAT offer exciting potential for the development of a new generation of PRO instruments. However, most of the research into these techniques has been in non-healthcare settings, notably in education. Educational tests are very different from PRO instruments, and consequently problematic issues arise when adapting IRT and CAT to healthcare research.


Clinical scales differ appreciably from educational tests, and symptoms have characteristics distinctly different from examination questions. This affects the transferring of IRT technology. Particular areas of concern when applying IRT to PROs include inadequate software, difficulties in selecting models and communicating results, insufficient testing of local independence and other assumptions, and a need of guidelines for estimating sample size requirements. Similar concerns apply to differential item functioning (DIF), which is an important application of IRT. Multidimensional IRT is likely to be advantageous only for closely related PRO dimensions.


Although IRT and CAT provide appreciable potential benefits, there is a need for circumspection. Not all PRO scales are necessarily appropriate targets for this methodology. Traditional psychometric methods, and especially qualitative methods, continue to have an important role alongside IRT. Research should be funded to address the specific concerns that have been identified.


Quality of life Item response theory Patient reported outcomes Health outcomes measurement 


  1. 1.
    Berkson, J. (1944). Application of the logistic function to bio-assay. Journal of the American Statistical Society, 39, 357–365.Google Scholar
  2. 2.
    Rasch, G. (1960). Probabilistic models for some intelligence attainment tests. Copenhagen: Danish Institute for Educational Research.Google Scholar
  3. 3.
    Cox, D. R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society series B, 34, 187–220.Google Scholar
  4. 4.
    McCullagh, P. (1980). Regression models for ordinal data (with discussion). Journal of the Royal Statistical Society series B, 42, 109–142.Google Scholar
  5. 5.
    Van der Linden, W. J., & Hambleton, R. K. (1996). Item response theory: brief history, common models, and extensions. In W. J. Van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (p. 23). New York: Springer.Google Scholar
  6. 6.
    Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. (Psychometric Monograph No. 17). Iowa City: Psychometric Society.Google Scholar
  7. 7.
    Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.CrossRefGoogle Scholar
  8. 8.
    Fayers, P. M., & Hand, D. J. (1997). Factor analysis, causal indicators and quality of life. Quality of Life Research, 6, 139–150.PubMedGoogle Scholar
  9. 9.
    Fayers, P. M, & Hand, D. J. (2002). Causal variables, indicator variables and measurement scales: An example from quality of life. Journal of the Royal Statistical Society series A, 165, 233–261.Google Scholar
  10. 10.
    Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists (p. 123). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  11. 11.
    Wright, B. D. (1992). IRT in the 1990s: Which models work best? Rasch Measurement Transactions, 6, 196–200.Google Scholar
  12. 12.
    Donaldson, M. (2006). Using patient-reported outcomes in clinical oncology practice: benefits, challenges, and next steps. Expert Review Pharmacoeconomics Outcomes Research, 6, 87–95.CrossRefGoogle Scholar
  13. 13.
    De Boer, A. G. E. M., Van Lanschot, J. J. B., Stalmeier, P. F. M. et al. (2004). Is a single-item visual analogue scale as valid, reliable and responsive as multi-item scales in measuring quality of life? Quality of Life Research, 13, 311–320.PubMedCrossRefGoogle Scholar
  14. 14.
    Collins, S. L., Edwards, J., Moore, R. A et al. (2001). Seeking a simple measure of analgesia for mega-trials: Is a single global assessment good enough? Pain, 91, 189–194.PubMedCrossRefGoogle Scholar
  15. 15.
    Bernhard, J., Sullivan M., Hurny C. et al. (2001). Clinical relevance of single item quality of life indicators in cancer clinical trials. British Journal of Cancer, 84, 1156–1165.PubMedCrossRefGoogle Scholar
  16. 16.
    Petersen, M. Aa., Groenvold, M., Aaronson, N., Fayers, P. M., Sprangers, M. A., & Bjorner, J. B. (2006). Multidimensional computerized adaptive testing of the EORTC QLQ-C30: Basic developments and evaluation. Quality of Life Research, 15, 315–329.PubMedCrossRefGoogle Scholar
  17. 17.
    Haley, S. M., Ni, P. S., Ludlow, L. H., & Fragala-Pinkham, M. A. (2006). Measurement precision and efficiency of multidimensional computer adaptive testing of physical functioning using the pediatric evaluation of disability inventory. Archives of Physical Medicine and Rehabilitation, 87, 1223–1229.PubMedCrossRefGoogle Scholar
  18. 18.
    Wang, W.-C., Chen, P.-H., & Cheng, T.-Y. (2004). Improving measurement precision of test batteries using multidimensional item response models. Psychological Methods, 9, 116–136.PubMedCrossRefGoogle Scholar
  19. 19.
    Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists (p. 278). Mahwah: Lawrence Erlbaum Associates.Google Scholar
  20. 20.
    Bair, M. J., Robinson, R. L., Katon, W., & Kroenke, K. (2003). Depression and pain co-morbidity: a literature review. Archives of Internal Medicine, 163, 2433–2445.PubMedCrossRefGoogle Scholar
  21. 21.
    Ruoff, G. E. (1996). Depression in the patient with chronic pain. Journal of Family Practice, 43, S25–S33.PubMedGoogle Scholar
  22. 22.
    Alonso, J., Angermeyer, M. C., Bernert, S. et al. (2004). Prevalence of mental disorders in Europe: Results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatrica Scandinavica Suppl 420, 21–27.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.Department of Public Health, Institute of Applied Health SciencesUniversity of Aberdeen Medical SchoolAberdeenUK
  2. 2.Faculty of Medicine, Pain and Palliation Research GroupNorwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations