INTRODUCTION
Clinical complexity (CC) represents one of the most relevant challenges of modern medicine, and its quantification is crucial both to stratify the clinical risk and for a fair hospital reimbursement policy.1,2,3 CC is a multifaceted condition that encompasses biological (e.g., age, multimorbidity, frailty) and non-biological (e.g., socioeconomic, cultural, environmental, and behavioral) components.4, 5 On this basis, a vectorial model of CC has been set up in which each vector expresses the dynamic changes over time of each component.5 Although this model cannot but be an approximation of CC, it is able to take into account its determining factors all at once. To allow its use, each vector has been graded following a consensus meeting during which the five most representative variables of each CC domain were selected.6 However, a score made of a series of variables could be a source of variability. For this reason, we have performed an interobserver agreement study.
METHODS
In June–August 2018, three healthcare professionals conducted the present study: a senior physician (observer #1, gastroenterology consultant), a research nurse (observer #2), and a young physician (observer #3, internal medicine resident). The first two observers were preliminarily trained by testing the CC index with roughly 100 patients, whereas only its theoretical bases were taught to the third observer. The CC index (Fig. 1) was administered by each observer to 30 consecutive adult patients (mean age 68 years, range 31–88; 16 females) admitted to an academic internal medicine ward. The three observers followed a randomization list, so that each of them administered the CC index to the same patient for the first, second, and third times in three consecutive days. Patients with a poor prognosis (< 48 h) were excluded. Each observer timed her/himself with an electronic chronometer during the CC index administration.
We rescaled each variable (Fig. 1) to have a score of 0 if the answer was “no” and a score of 2 if the answer was “yes”; these were summed up within each domain (range 0–10) and over all domains (range 0–50). Agreement of the scores computed for each domain and overall between each of the three pairs of observers was assessed with the intraclass correlation coefficient (ICC) with 95% confidence intervals (CI). The strength of agreement was interpreted as follows: poor, < 0.50; moderate, 0.50–0.75; good, 0.75–0.90; excellent, > 0.90.
This study represents a sub-analysis of the San MAtteo Complexity study (NCT03439410) that was approved by the local Ethics Committee.
RESULTS
Table 1 reports the results of the interobserver agreement. Overall agreement was excellent (> 0.90) between observers #1 and #2, and good (≥ 0.80) between observers #1/#2 and #3. Domain agreement showed some weaknesses regarding the cultural domain (> 0.50, moderate agreement) between any two observers, and generally showed poorer results for all domains when comparing observers #1/#2 vs #3. The agreement between observers #1 and #2 was good (≥ 0.80), particularly for the biological and socioeconomic domains.
The mean times needed to complete the CC index were 21 ± 7 (observer #1), 24 ± 6 (observer #2), and 22 ± 8 min (observer #3).
DISCUSSION
The CC index studied has a high interobserver agreement among different healthcare professionals who were trained to administer this tool (observers #1 and #2). Observer #3 had a high agreement rate when considering the total CC index. However, the assessment of environmental and cultural domains showed lower agreement, possibly because some of their variables could be open to diverging interpretations. Hence, a training period consisting of becoming familiar with the meaning of the variables and practicing with this tool is warranted. A larger study involving more observers is needed to confirm our findings.
The completion of the CC index takes approximately 20 min, including the time needed to collect relevant patient medical history. We believe that this amount of time is acceptable, particularly in a research setting, and could be potentially reduced by computer-assisted data collection.
To conclude, the results of the present study support the feasibility of the use of the CC index in an internal medicine setting, and could provide background for its use in future studies. A simplification of the CC index, i.e., with fewer, but more specific, variables, will be considered at the end of its validation phase (NCT03439410).
Abbreviations
- CC:
-
Clinical complexity
- CI:
-
Confidence interval
- ICC:
-
Intraclass correlation coefficient
References
Whittle J, Bosworth H. Studying complexity is complex. J Gen Intern Med 2007;22:379–81.
Zullig LL, Whitson HE, Hastings SN, et al. A systematic review of conceptual frameworks of medical complexity and new model development. J Gen Intern Med 2016;31:329–37.
Corazza GR, Formagnana P, Lenti MV. Bringing complexity into clinical practice: An internistic approach. Eur J Intern Med 2019;61:9–14.
Hong CS, Atlas SJ, Ashburner JM, et al. Evaluating a model to predict primary care physician-defined complexity in a large academic primary care practice-based research network. J Gen Intern Med 2015;30:1741–7.
Safford MM, Allison JJ, Kiefe CI. Patient complexity: more than comorbidity. The vector model of complexity. J Gen Intern Med 2007;22:382–90.
Corazza GR, Klersy C, Formagnana P, Lenti MV, Padula D; Consensus Panel. A consensus for the development of a vector model to assess clinical complexity. Intern Emerg Med 2017;12:1313–8.
Acknowledgements
We thank Mr. Sturgeon for having proofread the paper.
Funding
This research is part of a project for the study of clinical complexity funded by San Matteo Hospital Foundation - Italian Ministry of Health (Progetto di Ricerca Corrente 2017 - PI Prof. Gino Roberto Corazza).
Author information
Authors and Affiliations
Contributions
All authors participated in the drafting of the manuscript or critical revision of the manuscript for important intellectual content, and provided approval of the final submitted version. Individual contributions are as follow: GRC, MVL designed the study, organized data collection, and drafted the manuscript; MVL, ASB, IB conducted the study and enrolled patients; MC, GB, CK contributed to data collection and analysis; GRC made the final critical revision for important intellectual content. All authors approved the final version of the paper.
Corresponding author
Ethics declarations
This study represents a sub-analysis of the San MAtteo Complexity study (NCT03439410) that was approved by the local Ethics Committee.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lenti, M.V., Klersy, C., Brera, A.S. et al. Reproducibility in the Assessment of the Components of a Clinical Complexity Index. J GEN INTERN MED 34, 2316–2318 (2019). https://doi.org/10.1007/s11606-019-05228-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-019-05228-8