Quality of Life Research

, Volume 27, Issue 4, pp 1041–1054 | Cite as

Modernizing quality of life assessment: development of a multidimensional computerized adaptive questionnaire for patients with schizophrenia

  • Pierre Michel
  • Karine Baumstarck
  • Christophe Lancon
  • Badih Ghattas
  • Anderson Loundou
  • Pascal Auquier
  • Laurent Boyer



Quality of life (QoL) is still assessed using paper-based and fixed-length questionnaires, which is one reason why QoL measurements have not been routinely implemented in clinical practice. Providing new QoL measures that combine computer technology with modern measurement theory may enhance their clinical use. The aim of this study was to develop a QoL multidimensional computerized adaptive test (MCAT), the SQoL-MCAT, from the fixed-length SQoL questionnaire for patients with schizophrenia.


In this multicentre cross-sectional study, we collected sociodemographic information, clinical characteristics (i.e., duration of illness, the PANSS, and the Calgary Depression Scale), and quality of life (i.e., SQoL). The development of the SQoL-CAT was divided into three stages: (1) multidimensional item response theory (MIRT) analysis, (2) multidimensional computerized adaptive test (MCAT) simulations with analyses of accuracy and precision, and (3) external validity.


Five hundred and seventeen patients participated in this study. The MIRT analysis found that all items displayed good fit with the multidimensional graded response model, with satisfactory reliability for each dimension. The SQoL-MCAT was 39% shorter than the fixed-length SQoL questionnaire and had satisfactory accuracy (levels of correlation >0.9) and precision (standard error of measurement <0.55 and root mean square error <0.3). External validity was confirmed via correlations between the SQoL-MCAT dimension scores and symptomatology scores.


The SQoL-MCAT is the first computerized adaptive QoL questionnaire for patients with schizophrenia. Tailored for patient characteristics and significantly shorter than the paper-based version, the SQoL-MCAT may improve the feasibility of assessing QoL in clinical practice.


Schizophrenia Quality of life Patient-reported outcome Multidimensional computerized adaptive testing Item response theory Psychometrics 

Supplementary material

11136_2017_1553_MOESM1_ESM.docx (15 kb)
Supplementary material 1 (DOCX 14 KB)
11136_2017_1553_MOESM2_ESM.docx (17 kb)
Supplementary material 2 (DOCX 17 KB)


  1. 1.
    Awad, A. G., Voruganti, L. N., & Heslegrave, R. J. (1997). Measuring quality of life in patients with schizophrenia. PharmacoEconomics, 11(1), 32–47.CrossRefPubMedGoogle Scholar
  2. 2.
    Bobes, J., García-Portilla, P., Sáiz, P. A., Bascarán, T., & Bousoño, M. (oct 2005). Quality of life measures in schizophrenia. European Psychiatry, 20(Suppl 3), S313–S317.CrossRefPubMedGoogle Scholar
  3. 3.
    Boyer, L., Baumstarck, K., Boucekine, M., Blanc, J., Lançon, C., & Auquier, P. (2013). Measuring quality of life in patients with schizophrenia: An overview. Expert Review of Pharmacoeconomics & Outcomes Research, 13(3), 343–349.CrossRefGoogle Scholar
  4. 4.
    Greenhalgh, J., Long, A. F., & Flynn, R. (1982). The use of patient reported outcome measures in routine clinical practice: Lack of impact or lack of theory? Social Science & Medicine, 60(4), 833–843.CrossRefGoogle Scholar
  5. 5.
    Morris, J., Perez, D., & McNoe, B. (1998). The use of quality of life data in clinical practice. Quality of Life Research, 7(1), 85–91.CrossRefPubMedGoogle Scholar
  6. 6.
    Awad, A. G., & Voruganti, L. N. P. (2012). Measuring quality of life in patients with schizophrenia: An update. PharmacoEconomics, 30(3), 183–195.CrossRefPubMedGoogle Scholar
  7. 7.
    Halyard, M. Y., Frost, M. H., Dueck, A., & Sloan, J. A. (2006). Is the use of QOL data really any different than other medical testing? Current Problems in Cancer, 30(6), 261–271.CrossRefPubMedGoogle Scholar
  8. 8.
    Halyard, M. Y., Frost, M. H., & Dueck, A. (2006). Integrating QOL assessments for clinical and research purposes. Current Problems in Cancer, 30(6), 319–330.CrossRefPubMedGoogle Scholar
  9. 9.
    Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., & Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.CrossRefPubMedCentralPubMedGoogle Scholar
  10. 10.
    Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., & Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.CrossRefPubMedCentralPubMedGoogle Scholar
  11. 11.
    Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists (1 ed). Mahwah, NJ: Psychology Press.Google Scholar
  12. 12.
    Fayers, P., & Machin, D. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes (2e éd). New York: Wiley.CrossRefGoogle Scholar
  13. 13.
    Weiss. (2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37, 70–84.CrossRefGoogle Scholar
  14. 14.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., & Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.CrossRefPubMedGoogle Scholar
  15. 15.
    Hill, C. D., Edwards, M. C., Thissen, D., Langer, M. M., Wirth, R. J., & Burwinkle, T. M., et al. (2007). Practical issues in the application of item response theory: A demonstration using items from the pediatric quality of life inventory (PedsQL) 4.0 generic core scales. Medical Care, 45(5 Suppl 1), S39–S47.CrossRefPubMedGoogle Scholar
  16. 16.
    Gardner, W., Kelleher, K. J., & Pajer, K. A. (2002). Multidimensional adaptive testing for mental health problems in primary care. Medical Care, 40(9), 812–823.CrossRefPubMedGoogle Scholar
  17. 17.
    Petersen, M. A., Groenvold, M., Aaronson, N., Fayers, P., Sprangers, M., Bjorner, J. B. (2006). Multidimensional computerized adaptive testing of the EORTC QLQ-C30: Basic developments and evaluations. Quality Life Research, 15(3), 315–329.CrossRefGoogle Scholar
  18. 18.
    Haley, S. M., Ni, P., Dumas, H. M., Fragala-Pinkham, M. A., Hambleton, R. K., & Montpetit, K., et al. (2009) Measuring global physical health in children with cerebral palsy: Illustration of a multidimensional bi-factor model and computerized adaptive testing. Quality Life Research, 18(3), 359–370.CrossRefGoogle Scholar
  19. 19.
    Auquier, P., Simeoni, M. C., Sapin, C., Reine, G., Aghababian, V., & Cramer, J., et al. (2003). Development and validation of a patient-based health-related quality of life questionnaire in schizophrenia: The S-QoL. Schizophrenia Research, 63(1–2), 137–149.CrossRefPubMedGoogle Scholar
  20. 20.
    Baumstarck, K., Boyer, L., Boucekine, M., Aghababian, V., Parola, N., & Lançon, C., et al. Self-reported quality of life measure is reliable and valid in adult patients suffering from schizophrenia with executive impairment. Schizophrenia Research, 147(1), 58–67.Google Scholar
  21. 21.
    Baumstarck, K., Boucekine, M., Boyer, L., Aghababian, V., Parola, N., & Reuter, F., et al. (2014). Quantification of relevance of quality of life assessment for patients with cognitive impairment: The suitability indices. BMC Neurology, 14(1), 78.CrossRefPubMedCentralPubMedGoogle Scholar
  22. 22.
    Auquier, P., Tinland, A., Fortanier, C., Loundou, A., Baumstarck, K., & Lancon, C., et al. (2013). Toward meeting the needs of homeless people with schizophrenia: The validity of quality of life measurement. PloS ONE, 8(10), e79677.CrossRefPubMedCentralPubMedGoogle Scholar
  23. 23.
    Boyer, L., Lançon, C., Baumstarck, K., Parola, N., Berbis, J., & Auquier, P. (2013). Evaluating the impact of a quality of life assessment with feedback to clinicians in patients with schizophrenia: Randomised controlled trial. The British Journal of Psychiatry, 202, 447–453.CrossRefPubMedGoogle Scholar
  24. 24.
    Catherine, F.-A., Boyer, L., Jonathan, W., Jean-Philippe, R., Raphaelle, R., Elisabeth, S., et al. (2015). Neural substrate of quality of life in patients with schizophrenia: A magnetisation transfer imaging study. Scientific Reports, 5, 17650.PubMedCentralPubMedGoogle Scholar
  25. 25.
    Boyer, L., Richieri, R., Faget, C., Padovani, R., Vaillant, F., Mundler, O., et al. (2012). Functional involvement of superior temporal sulcus in quality of life of patients with schizophrenia. Psychiatry Research, 202(2), 155–160.CrossRefPubMedGoogle Scholar
  26. 26.
    McKenna, S. P. (1997) Measuring quality of life in schizophrenia. European Psychiatry, 12(Suppl 3), 267s–274s.CrossRefPubMedGoogle Scholar
  27. 27.
    Calman, K. C. (1984). Quality of life in cancer patients–an hypothesis. Journal of medical ethics, 10(3), 124–127.CrossRefPubMedCentralPubMedGoogle Scholar
  28. 28.
    Caqueo-Urízar, A., Boyer, L., Boucekine, M., & Auquier, P. (2014). Spanish cross-cultural adaptation and psychometric properties of the Schizophrenia Quality of Life short-version questionnaire (SQoL18) in 3 middle-income countries: Bolivia, Chile and Peru. Schizophrenia Research, 159(1), 136–143.CrossRefPubMedGoogle Scholar
  29. 29.
    Chou, C.-Y., Ma, M.-C., Yang, T.-T., Chen-Sea, M.-J. (2011). Psychometric validation of the S-QoL Chinese (Taiwan) version for patients with schizophrenia. Quality Life Research, 20(5), 763–767.CrossRefGoogle Scholar
  30. 30.
    American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: American Psychiatric Association.Google Scholar
  31. 31.
    Loi n° 78–17 du 6 janvier. (1978). relative à l’informatique, aux fichiers et aux libertés.Google Scholar
  32. 32.
    WMA Declaration of Helsinki - Ethical principles for medical research involving human subjects. (2013). Retrieved March 18, 2016, from
  33. 33.
    Kay, S. R., Fiszbein, A., & Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for Schizophrenia. Schizophrenia Bulletin, 13(2), 261–276.CrossRefPubMedGoogle Scholar
  34. 34.
    Lançon, C., Reine, G., Llorca, P. M., & Auquier, P. (1999). Validity and reliability of the French-language version of the Positive and Negative Syndrome Scale (PANSS). Acta Psychiatrica Scandinavica, 100(3), 237–243.CrossRefPubMedGoogle Scholar
  35. 35.
    Addington, D, Addington, J, & Schissel, B. (1990) A depression rating scale for schizophrenics. Schizophrenia Research, 3(4), 247–251.CrossRefPubMedGoogle Scholar
  36. 36.
    Michel, P., Baumstarck, K., Ghattas, B., Pelletier, J., Loundou, A., & Boucekine, M., et al. A multidimensional computerized adaptive short-form quality of life questionnaire developed and validated for multiple sclerosis. The MusiQoL-MCAT. Medicine (Baltimore). 2016.Google Scholar
  37. 37.
    Fayers, P. M., & Hand, D. J.. (1997). Factor analysis, causal indicators and quality of life. Quality Life Research, 6(2), 139–150.CrossRefGoogle Scholar
  38. 38.
    Costa, D. S. J. (2015). Reflective, causal, and composite indicators of quality of life: A conceptual or an empirical distinction? Quality Life Research, 24(9), 2057–2065.CrossRefGoogle Scholar
  39. 39.
    Muthén, L., & Muthén, B. (2012). MPlus user’s guide (7th ed). Los Angeles: Muthén & Muthén.Google Scholar
  40. 40.
    Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.CrossRefGoogle Scholar
  41. 41.
    Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Iowa City: Psychometric Society.Google Scholar
  42. 42.
    Samejima, F. (1974). Normal ogive model on the continuous response level in the multidimensional latent space. Psychometrika, 39(1), 111–121.CrossRefGoogle Scholar
  43. 43.
    Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23.CrossRefGoogle Scholar
  44. 44.
    Hays, R. D., Morales, L. S., & Reise, S. P. (2000). Item response theory and health outcomes measurement in the 21st century. Medical Care, 38(9 Suppl), II28-II42.PubMedCentralGoogle Scholar
  45. 45.
    Cai L. (2010). Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics, 35(3), 307–335.CrossRefGoogle Scholar
  46. 46.
    Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29.CrossRefGoogle Scholar
  47. 47.
    Kang, T., & Chen, T. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45(4), 391–406.CrossRefGoogle Scholar
  48. 48.
    Raju, N. S., Price, L. R., Oshima, T. C., & Nering, M. L. (2007). Standardized conditional SEM: A case for conditional reliability. Applied Psychological Measurement, 31(3), 169–80.CrossRefGoogle Scholar
  49. 49.
    Wang, T, & Others, A. (1996). Conditional standard errors, reliability and decision consistency of performance levels using polytomous IRT. Retrieved February 18, 2015, from
  50. 50.
    Wright, B. D., & Linacre, J. M. (1994) Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.Google Scholar
  51. 51.
    Molenaar, I. W. (1991). A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Kwant Methoden, 12(37), 97–117.Google Scholar
  52. 52.
    van der Ark, L. A. (2007). Mokken scale analysis in R. Journal of Statistical Software, 39(8), 1–30.Google Scholar
  53. 53.
    Stochl, J., Jones, P. B., & Croudace, T. J. (2012). Mokken scale analysis of mental health and well-being questionnaire item responses: A non-parametric IRT method in empirical research for applied health researchers. BMC Medical Research Methodology, 12, 74.CrossRefPubMedCentralPubMedGoogle Scholar
  54. 54.
    Siu, C. O., Harvey, P. D., Agid, O., Waye, M., Brambilla, C., & Choi, W.-K., et al. (2015). Insight and subjective measures of quality of life in chronic schizophrenia. Schizophrenia Research: Cognition, 2(3), 127–132.CrossRefGoogle Scholar
  55. 55.
    Boyer, L., Cermolacce, M., Dassa, D., Fernandez, J., Boucekine, M., & Richieri, R., et al. (2012). Neurocognition, insight and medication nonadherence in schizophrenia: A structural equation modeling approach. PloS ONE, 7(10), e47655.CrossRefPubMedCentralPubMedGoogle Scholar
  56. 56.
    Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). Lordif: An R Package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39(8), 1–30.CrossRefPubMedCentralPubMedGoogle Scholar
  57. 57.
    Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (ordinal) item scores. Ottawa, Directorate of Human Resources Research and Evaluation, Department of National Defense.Google Scholar
  58. 58.
    Anatchkova, M. D., Saris-Baglama, R. N., Kosinski, M., & Bjorner, J. B. (2009). Development and preliminary testing of a computerized adaptive assessment of chronic pain. The Journal of Pain, 10(9), 932–943.CrossRefPubMedCentralPubMedGoogle Scholar
  59. 59.
    Huisman, M., & Molenaar, I. W. (2001). Imputation of missing scale data with item response models. In: Boomsma, A., van Duijn, M. A. J. & Snijders, T. A. B. (Eds.) Essays on item response theory, pp. 221–244. New York: Springer. (Lecture Notes in Statistics). Retrieved September 20, 2013, from
  60. 60.
    Peyre, H., Leplège, A., & Coste, J. (2010). Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the SF-36 in the French 2003 decennial health survey. Quality of Life Research, 20(2), 287–300.CrossRefPubMedGoogle Scholar
  61. 61.
    Mulder, J, & van der Linden, W. J.. (2009). Multidimensional adaptive testing with Kullback–Leibler information item selection. In: van der Linden, W. J. & Glas, C. A. W. (Eds.) Elements of adaptive testing, pp. 77–101. New York: Springer. (Statistics for Social and Behavioral Sciences). Retrieved January 28, 2015, from
  62. 62.
    Harvill, L. M. (1991). Standard error of measurement. Educational Measurement: Issues and Practice, 10(2), 33–41.CrossRefGoogle Scholar
  63. 63.
    Babcock, B., & Weiss, D. J. (2013). Termination criteria in computerized adaptive tests: Do variable-length CATs provide efficient and effective measurement? Journal of Computerized Adaptive Testing, 1, 1–18.Google Scholar
  64. 64.
    Choi, S. W., Reise, S. P., Pilkonis, P. A., Hays, R. D., & Cella, D. (2010). Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Quality of Life Research, 19(1), 125–136.CrossRefPubMedGoogle Scholar
  65. 65.
    R Core Team. (2016). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved from
  66. 66.
    Alessandrini, M., Lançon, C., Fond, G., Faget-Agius, C., Richieri, R., & Faugere, M., et al. (2016). A structural equation modelling approach to explore the determinants of quality of life in schizophrenia. Schizophrenia Research, 171(1–3), 27–34.CrossRefPubMedGoogle Scholar
  67. 67.
    Ruggeri, M., Nosè, M., Bonetto, C., Cristofalo, D., Lasalvia, A., Salvi, G., et al. (2005). Changes and predictors of change in objective and subjective quality of life: Multiwave follow-up study in community psychiatric practice. The British Journal of Psychiatry, 187, 121–130.CrossRefPubMedGoogle Scholar
  68. 68.
    Reine, G., Simeoni, M.-C., Auquier, P., Loundou, A., Aghababian, V., & Lancon, C. (2005). Assessing health-related quality of life in patients suffering from schizophrenia: A comparison of instruments. European Psychiatry, 20(7), 510–519.CrossRefPubMedGoogle Scholar
  69. 69.
    Kemmler, G., Holzner, B., Neudorfer, C., Meise, U., & Hinterhuber, H. (1997). General life satisfaction and domain-specific quality of life in chronic schizophrenic patients. Quality of Life Research, 6(3), 265–273.CrossRefPubMedGoogle Scholar
  70. 70.
    Folsom, D. P., Depp, C., Palmer, B. W., Mausbach, B. T., Golshan, S., & Fellows, I., et al. (2009). Physical and mental health-related quality of life among older people with schizophrenia. Schizophrenia Research, 108(1–3), 207–213.CrossRefPubMedCentralPubMedGoogle Scholar
  71. 71.
    Smits, N. (2016). On the effect of adding clinical samples to validation studies of patient-reported outcome item banks: A simulation study. Quality of Life Research, 25(7), 1635–1644.CrossRefPubMedGoogle Scholar
  72. 72.
    Yao, L. (2014). Multidimensional CAT item selection methods for domain scores and composite scores with item exposure control and content constraints. Journal of Educational Measurement, 51(1), 18–38.CrossRefGoogle Scholar
  73. 73.
    Petersen, M. A., Giesinger, J. M., Holzner, B., Arraras, J. I., Conroy, T., & Gamper, E.-M., et al. (2013). Psychometric evaluation of the EORTC computerized adaptive test (CAT) fatigue item pool. Quality of Life Research, 22(9), 2443–2454.CrossRefPubMedGoogle Scholar
  74. 74.
    Zhang, J. (2012). Calibration of response data using MIRT models with simple and mixed structures. Applied Psychological Measurement, 36(5), 375–398.CrossRefGoogle Scholar
  75. 75.
    Baumstarck, K., Boyer, L., Boucekine, M., Aghababian, V., Parola, N., & Lançon, C., et al. (2013). Self-reported quality of life measure is reliable and valid in adult patients suffering from schizophrenia with executive impairment. Schizophrenia Research, 147(1), 58–67.CrossRefPubMedGoogle Scholar
  76. 76.
    Girard, V., Tinland, A., Boucekine, M., Loubière, S., Lancon, C., & Boyer, L., et al. (2016). Validity of a common quality of life measurement in homeless individuals with bipolar disorder and schizophrenia. Journal of Affective Disorders, 204, 131–137.CrossRefPubMedGoogle Scholar
  77. 77.
    Woods, C. M. (2014). Estimating the latent density in unidimensional IRT to permit non-normality. In Handbook of item response theory modeling: Applications to typical performance assessment. New York: Routledge.Google Scholar
  78. 78.
    Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347–360.CrossRefGoogle Scholar
  79. 79.
    Nunnally, J. C, & Bernstein, I. H. (1994) Psychometric theory, p. 736 (3rd ed). New York: McGraw-Hill.Google Scholar
  80. 80.
    Carpenter, J. R., & Kenward, M. G. (2008). Missing data in randomised controlled trials—a practical guide. Birmingham: The National Institute for Health Research. RM03/JH17/MK.Google Scholar
  81. 81.
    Fielding, S., Fayers, P. M., & Ramsay, C. R. (2009). Investigating the missing data mechanism in quality of life outcomes: A comparison of approaches. Health and Quality of Life Outcomes, 7, 57.CrossRefPubMedCentralPubMedGoogle Scholar
  82. 82.
    Nortvedt, M. W., Riise, T., Myhr, K. M., & Nyland, H. I. (2000). Quality of life as a predictor for change in disability in MS. Neurology, 55(1), 51–54.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Pierre Michel
    • 1
    • 2
  • Karine Baumstarck
    • 1
  • Christophe Lancon
    • 1
  • Badih Ghattas
    • 1
    • 2
  • Anderson Loundou
    • 1
  • Pascal Auquier
    • 1
  • Laurent Boyer
    • 1
  1. 1.Aix-Marseille University, EA 3279 – Public Health, Chronic Diseases and Quality of Life - Research UnitMarseilleFrance
  2. 2.Aix-Marseille University – I2M UMR 7373 – Mathematics Institute of MarseilleMarseilleFrance

Personalised recommendations