Advertisement

I SURVIVE: inter-rater reliability of three physical functional outcome measures in intensive care unit survivors

  • F. Aileen Costigan
  • Bram Rochwerg
  • Alexander J. Molloy
  • Magda McCaughan
  • Tina Millen
  • Julie C. Reid
  • Chris Farley
  • Laurel Patterson
  • Michelle E. KhoEmail author
Reports of Original Investigations

Abstract

Purpose

We prospectively assessed inter-rater reliability of three physical function measures in Canadian intensive care unit (ICU) survivors in the inpatient setting.

Methods

We enrolled patients who had an ICU length of stay of ≥ three days, were mechanically ventilated for ≥ 24 hr, and were ambulating independently before hospital admission. Weekly from ICU discharge to hospital discharge, two trained frontline clinicians, blinded to each other’s findings, independently performed the Physical Function ICU Test-scored (PFIT-s; score out of 10), 30-sec sit-to-stand (30STS; # of stands), and two-minute walk test (2MWT; distance in m), all within 24 hr. We calculated the intraclass correlation coefficient (ICC), standard error of measurement (SEM), and minimal detectable change (MDC90).

Results

We enrolled 42 patients. PFIT-s: in 36 patients with 66 paired scores, the ICC was 0.78 (95% confidence interval [CI], 0.66 to 0.86), the SEM was 1.04, and the MDC90 was 2.42. 30STS: in 35 patients with 67 paired scores, the ICC was 0.85 (95% CI, 0.76 to 0.90), the SEM was 1.91, and the MDC90 was 4.45. 2MWT: in 35 patients with 58 paired scores, the ICC was 0.78 (95% CI, 0.66 to 0.87), the SEM was 20.87, and the MDC90 was 48.69.

Conclusion

These three measures show good inter-rater reliability when used by trained frontline clinicians to assess physical function in ICU survivors in the inpatient setting.

I SURVIVE : Fidélité interobservateurs de trois mesures de la capacité physique fonctionnelle chez des survivants de l’unité de soins intensifs

Résumé

Objectif

Nous avons évalué de manière prospective la fidélité interobservateurs de trois mesures de la capacité physique fonctionnelle auprès de survivants des unités de soins intensifs (USI) canadiennes dans un cadre hospitalier.

Méthode

Nous avons recruté des patients qui avaient séjourné à l’USI ≥ trois jours, avaient été sous ventilation mécanique ≥ 24 h et se déplaçaient de façon autonome avant leur admission à l’hôpital. Entre le congé de l’USI et le congé de l’hôpital, deux cliniciens de première ligne formés ont réalisé chaque semaine et de manière indépendante trois tests : le PFIT (Test de la capacité physique fonctionnelle à l’USI - Physical Function ICU Test-scored; score sur 10), le test de 30 sec pour se lever d’une position assise (30STS; # de fois debout), et le test de marche pendant deux minutes (2MWT; distance en m). Les cliniciens n’avaient pas accès aux résultats de leur collègue. Nous avons calculé le coefficient de corrélation intraclasse (CCI), l’erreur type sur la mesure (ETM), et le changement minimal détectable (CMD90).

Résultats

Nous avons recruté 42 patients, et observé les résultats suivants : pour le PFIT : chez 36 patients avec 66 scores appariés, le CCI était de 0,78 (intervalle de confiance [IC] 95 %, 0,66 à 0,86), l’ETM de 1,04, et le CMD90 de 2,42; pour le test 30STS : chez 35 patients avec 67 scores appariés, le CCI était de 0,85 (IC 95 %, 0,76 à 0,90), l’ETM de 1,91, et le CMD90 de 4,45; pour le 2MWT : chez 35 patients avec 58 scores appariés, le CCI était de 0,78 (IC 95 %, 0,66 à 0,87, l’ETM de 20,87, et le CMD90 de 48,69.

Conclusion

Ces trois mesures affichent une bonne fidélité interobservateurs lorsqu’elles sont utilisées par des cliniciens de première ligne formés afin d’évaluer la capacité physique fonctionnelle des survivants de l’USI dans un cadre hospitalier.

Notes

Acknowledgements

The authors would like to thank the following people for their roles in this study: St. Joseph’s Healthcare: Trained assessors: Sarah Couch, Caitlin Kuzyk, Frances Garner, Brittany McCulloch, Christine Neilson, Miranda Prince. Juravinksi Hospital: Trained assessors: Daniel Aranda, Cortney Armstrong, Mel Auclair, Helen Bishop, Tania Brittain, Rachel Consoli, Shivaun Davidson, Chelsea Hale, Jessie Koopman, Gillian Manson, Leigh Ann Niven, Kyla Soucie, Jessica Temesy; those who collected outcome measures or helped facilitate measurements; Lisa Buckingham for database support; Professor Paul Stratford and Dr. Lawrence Mbuagbaw for statistical consultation.

Declarations of interest

None declared.

Editorial responsibility

This submission was handled by Dr. Sangeeta Mehta, Associate Editor, Canadian Journal of Anesthesia.

Author contributions

F Aileen Costigan, Bram Rochwerg, and Michelle E. Kho contributed to all aspects of this manuscript, including study conception and design; acquisition, analysis, and interpretation of data; and drafting the article. Alexander J. Molloy contributed to acquisition, analysis, and interpretation of the data. Julie C. Reid contributed to the analysis and the interpretation of data. Magda McCaughanTina M. Millen, Chris Farley, and Laurel Patterson contributed to the acquisition of data.

Funding

This work was supported by the Canadian Respiratory Health Professionals grant from the Lung Association and the Ontario Ministry of Research, Innovation, and Science Early Researcher Award. Michelle Kho holds a Canada Research Chair in Critical Care Rehabilitation and Knowledge Translation from the Canadian Institutes of Health Research (CIHR). The funders had no influence on the design, conduct, analysis, or decision to submit this manuscript for publication.

Supplementary material

12630_2019_1411_MOESM1_ESM.pdf (104 kb)
Supplementary material 1 (PDF 103 kb)

References

  1. 1.
    Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA 2010; 304: 1787-94.CrossRefGoogle Scholar
  2. 2.
    Esteban A, Frutos-Vivar F, Muriel A, et al. Evolution of mortality over time in patients receiving mechanical ventilation. Am J Respir Crit Care Med 2013; 188: 220-30.CrossRefGoogle Scholar
  3. 3.
    Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012. JAMA 2014; 311: 1308-16.CrossRefGoogle Scholar
  4. 4.
    Needham DM, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders’ conference. Crit Care Med 2012; 40: 502-9.CrossRefGoogle Scholar
  5. 5.
    Parry SM, Huang M, Needham DM. Evaluating physical functioning in critical care: considerations for clinical practice and research. Crit Care 2017; 21: 249.CrossRefGoogle Scholar
  6. 6.
    Needham DM, Sepulveda KA, Dinglas VD, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An International modified Delphi consensus study. Am J Respir Crit Care Med 2017; 196: 1122-30.CrossRefGoogle Scholar
  7. 7.
    Parry SM, Granger CL, Berney S, et al. Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med 2015; 41: 744-62.CrossRefGoogle Scholar
  8. 8.
    Jones CJ, Rikli RE, Beam WC. A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res Q Exerc Sport 1999; 70: 113-9.CrossRefGoogle Scholar
  9. 9.
    Bohannon RW, Wang YC, Gershon RC. Two-minute walk test performance by adults 18 to 85 years: normative values, reliability, and responsiveness. Arch Phys Med Rehabil 2015; 96: 472-7.CrossRefGoogle Scholar
  10. 10.
    Pin TW. Psychometric properties of 2-minute walk test: a systematic review. Arch Phys Med Rehabil 2014; 95: 1759-75.CrossRefGoogle Scholar
  11. 11.
    Reid JC, Kho ME, Stratford PW. Outcome measures in clinical practice: five questions to consider when assessing patient outcome. Curr Phys Med Rehabil Rep 2015; 3: 248-54.CrossRefGoogle Scholar
  12. 12.
    Calvo-Ayala E, Khan BA, Farber MO, Ely EW, Boustani MA. Interventions to improve the physical function of ICU survivors: a systematic review. Chest 2013; 144: 1469-80.CrossRefGoogle Scholar
  13. 13.
    Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011; 64: 96-106.CrossRefGoogle Scholar
  14. 14.
    Denehy L, Skinner EH, Edbrooke L, et al. Exercise rehabilitation for patients with critical illness: a randomized controlled trial with 12 months of follow-up. Crit Care 2013; 17: R156.CrossRefGoogle Scholar
  15. 15.
    Denehy L, de Morton NA, Skinner EH, et al. A physical function test for use in the intensive care unit: validity, responsiveness, and predictive utility of the physical function ICU test (scored). Phys Ther 2013; 93: 1636-45.CrossRefGoogle Scholar
  16. 16.
    Skinner EH, Berney S, Warrillow S, Denehy L. Development of a physical function outcome measure (PFIT) and a pilot exercise training protocol for use in intensive care. Crit Care Resusc 2009; 11: 110-5.Google Scholar
  17. 17.
    Zanni JM, Korupolu R, Fan E, et al. Rehabilitation therapy and outcomes in acute respiratory failure: an observational pilot project. J Crit Care 2010; 25: 254-62.CrossRefGoogle Scholar
  18. 18.
    Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985; 13: 818-29.CrossRefGoogle Scholar
  19. 19.
    Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987; 40: 373-83.CrossRefGoogle Scholar
  20. 20.
    Groll DL, To T, Bombardier C, Wright JG. The development of a comorbidity index with physical function as the outcome. J Clin Epidemiol 2005; 58: 595-602.CrossRefGoogle Scholar
  21. 21.
    Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. the index of ADL: a standardized measure of biological and psychosocial function. JAMA 1963; 185: 914-9.CrossRefGoogle Scholar
  22. 22.
    Bagshaw SM, Stelfox HT, McDermid RC, et al. Association between frailty and short- and long-term outcomes among critically ill patients: a multicentre prospective cohort study. CMAJ 2014; 186: E95-102.CrossRefGoogle Scholar
  23. 23.
    Stratford PW, Spadoni G. Sample size estimation for the comparison of competing measures’ reliability coefficients. Physiotherapy Canada 2003; 55: 225-9.CrossRefGoogle Scholar
  24. 24.
    Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Stat Med 1998; 17: 101-10.CrossRefGoogle Scholar
  25. 25.
    IBM. SPSS Statistics for Windows. 25.0th ed. Armonk, NY: IBM Corp; 2017 .Google Scholar
  26. 26.
    Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-8.CrossRefGoogle Scholar
  27. 27.
    Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 2005; 19: 231-40.Google Scholar
  28. 28.
    Riddle DL, Stratford P. Is This Change Real?: Interpreting Patient Outcomes in Physical Therapy. Paul Stratford, PA: FA Davis Co.; 2013 .Google Scholar
  29. 29.
    Koo TK, Li MY. A Guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155-63.CrossRefGoogle Scholar
  30. 30.
    Bodilsen AC, Juul-Larsen HG, Petersen J, Beyer N, Andersen O, Bandholm T. Feasibility and inter-rater reliability of physical performance measures in acutely admitted older medical patients. PloS One 2015; 10: e0118248.CrossRefGoogle Scholar
  31. 31.
    Lyders Johansen K, Derby Stistrup R, Skibdal Schjott C, Madsen J, Vinther A. Absolute and relative reliability of the timed ‘up & go’ test and ‘30second chair-stand’ test in hospitalised patients with stroke. PloS One 2016; 11: e0165663.CrossRefGoogle Scholar
  32. 32.
    Leung AS, Chan KK, Sykes K, Chan KS. Reliability, validity, and responsiveness of a 2-min walk test to assess exercise capacity of COPD patients. Chest 2006; 130: 119-25.CrossRefGoogle Scholar
  33. 33.
    Unnanuntana A, Ruangsomboon P, Keesukpunt W. Validity and responsiveness of the two-minute walk test for measuring functional recovery after total knee arthroplasty. J Arthroplasty 2018; 33: 1737-44.CrossRefGoogle Scholar
  34. 34.
    Corner EJ, Wood H, Englebretsen C, et al. The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study. Physiotherapy 2013; 99: 33-41.CrossRefGoogle Scholar

Copyright information

© Canadian Anesthesiologists' Society 2019

Authors and Affiliations

  • F. Aileen Costigan
    • 1
  • Bram Rochwerg
    • 2
    • 3
  • Alexander J. Molloy
    • 1
  • Magda McCaughan
    • 1
  • Tina Millen
    • 4
  • Julie C. Reid
    • 5
  • Chris Farley
    • 4
  • Laurel Patterson
    • 1
  • Michelle E. Kho
    • 1
    • 5
    • 6
    Email author
  1. 1.Physiotherapy DepartmentSt. Joseph’s HealthcareHamiltonCanada
  2. 2.Department of Medicine (Division of Critical Care)McMaster UniversityHamiltonCanada
  3. 3.Department of Health Research Methods, Evidence and ImpactMcMaster UniversityHamiltonCanada
  4. 4.Department of Critical CareJuravinski HospitalHamiltonCanada
  5. 5.Faculty of Health Sciences, School of Rehabilitation ScienceMcMaster UniversityHamiltonCanada
  6. 6.Institute of Applied Health Science, School of Rehabilitation ScienceMcMaster UniversityHamiltonCanada

Personalised recommendations