Advertisement

Predictive ability of perioperative atrial fibrillation risk indices in cardiac surgery patients: a retrospective cohort study

  • Nathan H. WaldronEmail author
  • Mary Cooter
  • Jonathan P. Piccini
  • Kevin J. Anstrom
  • Rebecca Y. Klinger
  • Miklos D. Kertai
  • Mihai V. Podgoreanu
  • Mark Stafford-Smith
  • Mark F. Newman
  • Joseph P. Mathew
Reports of Original Investigations
  • 218 Downloads

Abstract

Purpose

The Multicenter Study of Perioperative Ischemia (McSPI) AFRisk index predicts postoperative atrial fibrillation (POAF) after cardiac surgery, but requires pre-, intra-, and postoperative data. Other more abbreviated risk indices exist, but there is no consensus on which risk index is optimal. We compared the discriminatory capacity of the McSPI AFRisk index with three indices containing only preoperative data (the CHA2DS2Vasc score, POAF score, and Kolek clinical risk prediction model), hypothesizing that the McSPI AFRisk index would have superior predictive capacity.

Methods

We retrospectively evaluated 783 patients undergoing cardiac surgery using cardiopulmonary bypass. The predictive capacity of each index was assessed by comparing receiver-operating characteristic (ROC) curves, scaled Brier scores, net reclassification indices, and the integrated discrimination indices.

Results

The incidence of POAF was 32.6%. The area under the curve (AUC) of the ROC curve were 0.77, 0.58, 0.66, and 0.66 for the McSPI AFRisk index, CHA2DS2Vasc score, POAF score, and Kolek clinical risk prediction model, respectively. The McSPI AFRIsk index had the highest AUC (P < 0.0001). The scaled Brier scores for the McSPI AFRisk index, CHA2DS2Vasc score, POAF score, and Kolek clinical risk prediction model were 0.23, 0.02, 0.08, and 0.07, respectively. Both net reclassification indices and integrated discrimination indices showed that the McSPI AFRisk index more appropriately identified patients at high risk of POAF.

Conclusions

The McSPI AFRisk index showed superior ability to predict POAF after cardiac surgery compared with three other indices. When clinicians and investigators wish to measure the risk of POAF after cardiac surgery, they should consider using the McSPI AFRisk index.

Capacité de prédiction des indices de risque de fibrillation auriculaire périopératoire chez les patients de chirurgie cardiaque : une étude de cohorte rétrospective

Résumé

Objectif

L’indice AFRisk de l’étude multicentrique McSPI sur l’ischémie périopératoire prédit la fibrillation auriculaire postopératoire (POAF) après une chirurgie cardiaque, mais nécessite des données pré-, per- et postopératoires. Il existe d’autres indices de risques moins extensifs, mais il n’y a pas de consensus sur le choix de l’indice optimal. Nous avons comparé la capacité de discrimination de l’indice McSPI AFRisk à celle de trois autres indices ne contenant que des données préopératoires (le score CHA2DS2Vasc, le score POAF et le modèle de prédiction de risque clinique de Kolek) en énonçant l’hypothèse que la capacité de prédiction de l’indice McSPI AFRisk serait supérieure.

Méthodes

Nous avons évalué rétrospectivement 783 patients ayant subi une chirurgie cardiaque avec circulation extracorporelle. La capacité de prédiction de chaque indice a été évaluée en comparant les courbes d’efficacité du récepteur (ROC), les scores gradués de Brier, les indices de reclassement net et les indices de discrimination intégrée.

Résultats

L’incidence de la POAF était de 32,6 %. L’aire sous la courbe (ASC) de la courbe ROC était de 0,77, 0,58, 0,66 et 0,66 pour, respectivement, l’indice McSPI AFRisk, le score CHA2DS2Vasc, le score POAF et le modèle de prédiction de risque clinique de Kolek. L’indice McSPI AFRIsk avait l’ASC la plus élevée (P < 0,0001). Les scores gradués de Brier pour l’indice McSPI AFRisk, le score CHA2DS2Vasc, le score POAF et modèle de prédiction de risque clinique de Kolek étaient, respectivement, de 0,23, 0,02, 0,08 et 0,07. Les indices de reclassement net et de discrimination intégrée ont montré tous les deux que l’indice McSPI AFRisk identifiait mieux les patients à risque élevé de POAF.

Conclusions

L’indice McSPI AFRisk a démontré une capacité de prédiction de la POAF après chirurgie cardiaque supérieure à celle des trois autres indices. Quand les cliniciens et investigateurs souhaiteront mesurer le risque de POAF après chirurgie cardiaque, ils devraient envisager d’utiliser l’indice McSPI AFRisk.

Postoperative atrial fibrillation (POAF) is the most common complication after cardiac surgery,1 affecting 27-40% of patients and typically occurring in the first week after surgery.2 Postoperative atrial fibrillation after cardiac surgery is associated with an increased risk for postoperative stroke, recurrent atrial fibrillation, myocardial infarction, longer intensive care unit (ICU) and hospital length of stay (LOS), higher healthcare costs, higher readmission rates, and increased mortality.2-5 Risk factors for POAF are well established and include advanced age,2,6-10 history of atrial fibrillation (AF),2,6,8,9 valve surgery,2,9 chronic obstructive pulmonary disease (COPD),2,9 left ventricular dysfunction,6,8,9 left atrial enlargement,10 P-wave abnormalities,6,8 and medication withdrawal.2 To characterize individual risk of POAF, a number of risk indices incorporating multiple risk factors have been developed.

The Multicenter Study of Perioperative Ischemia (McSPI) AFRisk index, which was introduced in 20042 and modified in 2009 to include statin use,11 has been shown to reliably predict the occurrence of POAF after coronary artery bypass graft (CABG) surgery, but requires pre-, intra-, and postoperative data. Recently, POAF risk scores that include only preoperative data have been proposed to be sufficient. Introduced in 2010 to predict the risk of thromboembolism in patients with atrial fibrillation,12 the CHA2DS2VASc score has shown promise in predicting POAF after cardiac surgery in small- to moderate-size cohorts.13-15 Similarly, the recently developed POAF score9 and clinical risk-prediction model developed by Kolek et al.8 have shown predictive capacity in preliminary studies. While the POAF score has recently been found to outperform the CHA2DS2VASc score,16 there is still no consensus on which index best predicts POAF. Determining which index has the best discriminative capacity would allow for improved individual risk stratification for POAF and the potential to better assess risk reduction in interventional studies. Additionally, identification of an accurate risk index would facilitate implementation of prophylactic measures, such as amiodarone administration, to reduce the incidence of POAF after cardiac surgery.1

Because POAF increases morbidity, mortality, and healthcare costs, accurate risk assessment is important for instituting appropriate risk-reduction strategies. As such, we aimed to compare the predictive ability of the modified McSPI AFRisk index with three abbreviated indices containing only preoperative data, hypothesizing that the modified McSPI AFRisk index would have superior predictive capacity.

Methods

After obtaining approval from the Duke University School of Medicine Institutional Review Board (IRB Pro00006745, initially approved 13 March 2008), we retrospectively evaluated patients undergoing CABG ± valvular surgery in the Duke University Health System utilizing cardiopulmonary bypass (CPB) from 1994 to 2007 for whom comprehensive comorbidities, medications, and adverse outcomes had been prospectively collected. All included patients had consented to inclusion in a prospective observational database aimed at investigating phenotypic and genotypic differences that may be associated with adverse outcomes after cardiothoracic surgery. This database was collected with a target sample size of n = 1004 patients undergoing various cardiac surgical procedures and has been detailed in previous manuscripts.17 A STROBE diagram detailing cohort selection is shown in Fig. 1.
Fig. 1

STROBE diagram detailing cohort selection. ECG = electrocardiogram; POAF = postoperative atrial fibrillation

Prior to surgery, all patients were in sinus rhythm and were not being treated with Vaughan Williams class I or III antiarrhythmic agents. In all patients, anesthesia was induced and maintained with midazolam, fentanyl, propofol, and isoflurane or sevoflurane. Cardiopulmonary bypass management was standardized, with mean arterial pressure maintained between 50 and 80 mmHg, temperature during CPB maintained at 30-35°C as per institutional practice, and alpha-stat blood gas management utilized in all cases. Patients initially recovered in the Cardiothoracic Intensive Care Unit prior to transfer to a Stepdown Unit. Throughout the entire length of stay, cardiac rhythm was continuously monitored via telemetry.

Postoperative atrial fibrillation was defined as any atrial fibrillation/flutter episode lasting > 30 sec, as detected by continuous telemetry, electrocardiogram (ECG) analysis, or any AF that required treatment or was recorded in daily notes/discharge summaries. The modified McSPI AFRisk index,2,11 CHA2DS2Vasc score,13 POAF score,9 and Kolek clinical prediction model8 were calculated for all patients as defined in Table 1. Patients were classified as high risk for POAF if they had: a CHA2DS2Vasc score > 3, a POAF score of ≥ 3, or a Kolek clinical prediction score of ≥ 5. The modified McSPI score designates three risk categories, with patients classified as low risk for POAF if their modified McSPI AFRisk score is < 7, medium risk if their score is 7-24, and high risk if their score is > 24.11
Table 1

Demographic factors in each risk score

Demographic factor

Point value

Kolek clinical prediction model

 Age > 60 yr

2 points

 Male sex

1 point

 Diabetes, n (%)

1 point

 HTN, n (%)

1 point

 LVEF < 40%, n (%)

1 point

 PR interval > 200 msec, n (%)

1 point

 History of atrial fibrillation, n (%)

3 points

Modified McSPI AFRisk index

 Age

< 30: 0 points, 30-39: 6 points, 40-49: 12 points, 50-59: 18 points, 60-69: 24 points, 70-79: 30 points, ≥ 80: 36 points

 History of atrial fibrillation

7 points

 COPD

4 points

 Valvular surgery

5 points

 Beta-blocker withdrawal

7 points

 ACE-inhibitor withdrawal

5 points

 Pre- and postoperative beta blocker

−7 points

 Pre- and postoperative ACE Inhibitor

−5 points

 Pre- and postoperative statins

−5 points

 Postoperative beta blocker

−11 points

 Postoperative K+ supplementation

−5 points

 Postoperative NSAID use

−7 points

POAF score

 Age

60-69: 1 point, 70-79: 2 points, > 80: 3 points

 COPD

1 point

 GFR < 15*

1 point

 Dialysis*

 Emergency surgery

1 point

 Preoperative IABP

1 point

 LVEF < 30%

1 point

 Valvular surgery

1 point

*The composite renal variable for the POAF score receives 1 point and is composed of GFR < 15 or dialysis

CHA2DS2Vasc score

 Congestive heart failure

1 point

 HTN

1 point

 Age

≥ 75: 2 points, 65-74: 1 point

 Diabetes

1 point

 Stroke/TIA/thromboembolism

2 points

 Vascular disease*

1 point

 Prior MI*

 Sex

1 point if female

*”Vascular disease” receives 1 points and is described as prior MI, peripheral artery disease, or aortic plaque

Patients were defined as high risk for POAF if they had: a CHA2DS2Vasc score > 3, a POAF score ≥ 3, or a Kolek clinical prediction score ≥ 5. The McSPI score designates three risk categories, with patients defined as low risk of POAF if their McSPI AFRisk score is < 7, medium risk if their score is 7-24, and high risk if their score is > 24

ACE = angiotensin-converting enzyme; COPD = chronic obstructive pulmonary disease; GFR = glomerular filtration rate; HTN = hypertension; IABP = intra-aortic balloon pump; LVEF = left ventricular ejection fraction; MI = myocardial infarction; NSAID = non-steroidal anti-inflammatory drug; POAF = postoperative atrial fibrillation; TIA = transient ischemic attack

Statistical analysis

The predictive capacity of each risk score was assessed using comparison of receiver-operating characteristic (ROC) curves,18 scaled Brier scores,19 as well as calculation of a net reclassification index (NRI) and integrated discrimination index (IDI).20 For each ROC curve, the area under the curve (AUC) and the confidence intervals were estimated with generalized Mann-Whitney U statistics, and the differences between the AUCs were tested with Chi-squared tests. As point estimates and confidence intervals of risk index performance were of primary interest, the P values for the test of difference in the AUC were not corrected for the multiple pairwise comparisons. Brier scores, a measure of the accuracy of a probabilistic prediction, are defined as the average squared difference between the predicted probability and the patient’s observed outcome.21 Brier scores were calculated for each risk score and then scaled by the maximum possible score, given the observed incidence of POAF in this cohort, to create a scaled Brier score.

As each of the risk scores under consideration has defined risk strata,2,8,9,12 we used net reclassification indices, a measure of improvement in risk classification with transition from one model to another,22 to compare risk classification for each score relative to the modified AFRisk index. The calculation requires defining four probabilities based on reclassification to a higher risk class (moving “up”) and reclassification to a lower risk class (moving “down”) among patients who either did or did not experience POAF. We defined patients as reclassified “up” if they moved from low risk in the comparison index to either moderate or high risk by the modified McSPI AFRisk index and reclassified “down” if they moved from high risk to either moderate or low risk by the modified McSPI AFRisk index. The NRI is estimated as the difference in proportion of events (patients with POAF) that were reclassified “up” vs “down” minus the difference in proportion of non-events (patients without POAF) reclassified “up” vs “down”. In addition to comparing the risk classification we also calculated the IDI to assess the difference in discrimination between models across the possible range of cut points.20 The IDI corresponds to the difference in discrimination slopes between the risk scores being compared and may be interpreted as the average improvement in risk values for events and non-events.23 All analyses were performed using SAS™ version 9.4 (SAS Institute Inc, Cary, NC, USA) or R version 3.1.1. P < 0.05 was considered statistically significant.

Results

The initial cohort included 972 patients undergoing cardiac surgery, but due to lack of availability of ECG data used to calculate the Kolek score, this was reduced to 783 patients whose demographics are depicted in Table 2. The overall incidence of POAF was 32.6%, and the incidence of POAF in each risk group is depicted in Table 3. The mean [standard deviation (SD)] modified McSPI AFRisk index was 14.5 (13.0) and was higher in patients with POAF [23.2 (13.0)] than in patients without POAF [10.4 (10.7); P < 0.001]. The mean CHA2DS2Vasc score was 2.5 (1.5) and was higher in patients with POAF [2.8 (1.5)] than in patients without POAF [2.4 (1.5); P < 0.001]. The mean POAF score was 1.4 (1.1) and was higher in patients with POAF [1.8 (1.1)] than in patients without POAF [1.2 (1.0); P < 0.001]. The mean Kolek clinical risk prediction score was [3.4 (1.6)] and was higher in patients with POAF [4.0 (1.6)] than in patients without POAF [3.1 (1.6); P < 0.001].
Table 2

Demographics of included patients

 

POAF

P value

No POAF

n = 528 (67.4%)

Yes POAF

n = 255 (32.6%)

Baseline characteristics

Age

62 [55-69]

68 [62-74]

< 0.001

Race (white)

523 (99.1)

253 (99.2)

0.82

Sex (male)

383 (72.7)

197 (77.6)

0.14

Weight (kg)

85.4 [74.6-96.0]

83.8 [74.4-95.2]

0.42

EuroScore (additive sum)

4.0 [3.0-6.0]

6.0 [4.0-8.0]

< 0.001

Cardiac comorbidities

   

History of AF

15 (2.8)

26 (10.2)

< 0.001

Hypertension

333 (63.2)

175 (68.9)

0.12

History of MI

244 (46.2)

110 (43.1)

0.42

Congestive heart failure

53 (10.1)

35 (13.8)

0.13

Preop heart block

63 (12.9)

45 (19.1)

0.03

LVEF (%)

55 (45, 65)

55 (45, 70)

0.94

Prior cardiac surgery

37 (7.0)

9 (3.5)

0.05

COPD

49 (9.3)

44 (17.3)

0.001

Cerebrovascular disease

26 (5.0)

19 (7.7)

0.15

Prior stroke

17 (3.2)

9 (3.5)

0.82

Diabetes

174 (33.3)

79 (31.0)

0.58

Peripheral vascular disease

52 (9.9)

32 (12.6)

0.25

Renal disease: dialysis dependent

7 (1.3)

6 (2.4)

0.29

Severe renal disease

7 (1.3)

6 (2.4)

0.28

Intraoperative characteristics

CPB time

112 [92-133]

118 [97-146]

0.007

Aortic crossclamp time

63 [46-81]

68 [50-87]

0.03

Valvular procedure

11 (2.1)

9 (3.5)

0.23

Aortic valve replacement

8 (1.5)

5 (2.0)

NT

Mitral valve repair/replacement

5 (0.9)

4 (1.6)

NT

Tricuspid valve repair

0

1 (0.4)

NT

IABP used

28 (5.3)

14 (5.5)

0.91

Emergent

3 (0.6)

3 (1.2)

0.36

Perioperative medications

Beta blockers

   

Received preop and postop

397 (75.2)

109 (42.7)

< 0.001

Received postop only

80 (15.2)

24 (9.4)

0.03

Received preop and NOT postop

39 (7.4)

93 (36.5)

< 0.001

ACE inhibitors

   

Received preop and postop

76 (14.4)

21 (8.2)

0.01

Received preop and NOT postop

211 (40.0)

136 (53.3)

<0.001

Statins preop and postop

183 (34.7)

63 (24.7)

0.005

Postop potassium supplementation

456 (86.4)

169 (66.3)

< 0.001

Postop NSAID use

120 (22.7)

23 (9.0)

< 0.001

Data presented as median [interquartile range] or n (%). Severe renal disease was defined as serum creatinine > 3 g·dL−1, dialysis, or renal transplant. Heart block included ≥ second-degree atrioventricular blockade. NT = not tested

ACE = angiotensin-converting enzyme; COPD = chronic obstructive pulmonary disease; CPB = cardiopulmonary bypass; IABP = intra-aortic balloon pump; LVEF = left ventricular ejection fraction; MI = myocardial infarction; NSAID = non-steroidal anti-inflammatory drug; POAF = postoperative atrial fibrillation

Table 3

Incidence of POAF in each risk group

Risk score

Risk category

Incidence of POAF

Modified McSPI AFRisk index

High

69.1%

Moderate

29.3%

Low

11.6%

CHA 2 DS 2 Vasc score

High

41.1%

Low

29.9%

POAF score

High

59.2%

Low

27.5%

Kolek clinical risk model

High

50.3%

Low

27.6%

McSPI = Multicenter Study of Perioperative Ischemia; POAF = postoperative atrial fibrillation

Receiver-operating characteristic curves for each index are depicted in Fig. 2. The area under the ROC curve for the modified McSPI AFRisk index (0.77; 95% CI, 0.74 to 0.81) was significantly higher than for the CHA2DS2Vasc score (0.58; 95% CI, 0.54 to 0.62), POAF score (0.66; 95% CI, 0.62 to 0.70), and Kolek clinical risk prediction model (0.66; 95% CI, 0.62 to 0.70; P < 0.001 for all comparisons). In addition, the AUC for the CHA2DS2Vasc score was significantly lower than for the POAF score (P < 0.001) and the Kolek clinical risk prediction model score (P < 0.001). No difference was found between the AUC of the POAF and Kolek score (P = 0.79). A post-hoc power calculation utilizing the observed incidence of POAF (32.6%) and modified McSPI AFRisk AUC of 0.77 revealed 80% power to detect an AUC difference as small as 0.05 and 90% power for an AUC difference of 0.06. In our sample of 783 patients where 32.6% developed POAF, we estimate > 99% power to detect the observed difference between the modified McSPI AFRisk index AUC and the AUC of all other comparator scores.
Fig. 2

Receiver-operating characteristic curves for each risk index. McSPI = Multicenter Study of Perioperative Ischemia; POAF = postoperative atrial fibrillation

Adjusted Brier scores as well as R2 values suggest greater accuracy for the modified McSPI AFRisk index than the CHA2DS2Vasc score, POAF score, or Kolek clinical risk prediction model. In addition, the Hosmer-Lemeshow goodness of fit test did not reveal evidence of model misfit in our cohort (Table 4). Agreement on risk categorization between the modified McSPI AFRisk index, CHA2DS2Vasc score, POAF risk score, and Kolek clinical risk prediction model is depicted graphically in Fig. 3. The modified McSPI AFRisk index and CHA2DS2Vasc score classified 35.2% of patients in the same POAF risk category (8.8% as high risk, 26.4% as low risk). The modified McSPI AFRisk index and POAF risk score classified 36.7% of patients in the same POAF risk category (8.6% as high risk, 28.1% as low risk). The modified McSPI AFRisk index and Kolek clinical prediction risk model classified 36.1% of patients in the same POAF risk category (9.2% as high risk, 27.0% as low risk).
Table 4

Discrimination and calibration values for each risk index

Parameter

McSPI AFRisk score

CHA2DS2VASc score

POAF score

Kolek clinical risk model

AUC (95% CI)

0.77 (0.74 to 0.81)

0.58 (0.54 to 0.62)

0.66 (0.62 to 0.70)

0.66 (0.62-0.70)

Adjusted Brier score

0.228

0.020

0.077

0.067

R2 value

0.206

0.016

0.071

0.068

H-L GOF P value

0.21

0.84

0.60

0.27

AUC = area under the curve; CI = conficence interval; H-L GOF = Hosmer-Lemeshow goodness of fit; McSPI = Multicenter Study of Perioperative Ischemia; POAF = postoperative atrial fibrillation

Fig. 3

Scatter plots of comparison risk index predicted probability of postoperative atrial fibrillation (POAF) outcome by Multicenter Study of Perioperative Ischemia (McSPI) predicted probability of POAF outcome. The non-events appear in the left column with blue circles, and events appear in the right column with red crosses. The threshold for risk classification for each risk index is indicated with dashed lines

Reclassification data for the use of the modified McSPI AFRisk index compared with other risk scores are presented in Table 5. Application of the modified McSPI AFRisk index to the CHA2DS2Vasc score resulted in a total NRI = 0.209, with the NRI for patients with POAF = 0.492 and the NRI for patients without POAF = -0.283. Application of the modified McSPI AFRisk index to the POAF score resulted in a total NRI = 0.034, with the NRI for patients with POAF = 0.508 and the NRI for patients without POAF = -0.474. Application of the modified McSPI AFRisk index to the Kolek clinical risk prediction model resulted in a total NRI = 0.120, with the NRI for patients with POAF = 0.467 and the NRI for patients without POAF = -0.347. As seen in Table 5, the observed IDI was highest for the modified McSPI AFRisk index, indicating superior discrimination. By AUC difference and the NRI and IDI compared with the modified McSPI AFRisk index, the CHA2DS2Vasc score performed worst, and the POAF score was closest in performance although the modified McSPI AFRisk index had superior performance by every metric.
Table 5

Reclassification performance of the McSPI AFRisk Index vs CHA2DS2VASc score, POAF score, and Kolek clinical model

Parameter

McSPI AFRisk score vs CHA2DS2VASc score

McSPI AFRisk score vs POAF score

McSPI AFRisk score vs

Kolek clinical model

AUC difference (95% CI)

0.20 (0.15 to 0.24)

0.11 (0.08 to 0.15)

0.12 (0.08 to 0.16)

NRI for patients with POAF

0.492

0.508

0.467

NRI for patients without POAF

−0.283

−0.474

−0.347

Final NRI

0.209

0.034

0.120

IDI

0.208

0.151

0.156

AUC = area under the curve; CI = confidence interval; IDI = integrated discrimination index; McSPI = Multicenter Study of Perioperative Ischemia; NRI = net reclassification index; POAF = postoperative atrial fibrillation

Discussion

In this retrospective study of 783 cardiac surgery (including CPB) patients, the modified McSPI AFRisk index showed superior capacity to predict POAF compared with three other indices. The POAF score and Kolek clinical risk prediction model showed moderate predictive ability, while the CHA2DS2Vasc had the least predictive capacity for POAF. Given these results, we encourage investigators and clinicians to use the modified McSPI AFRisk index to predict risk of POAF after cardiac surgery.

The incidence of POAF in our study (32.6%) was similar to that seen in multiple previous cardiac surgical cohorts.4,8-10,24 Of note, POAF has previously been reported as a relatively rare phenomenon (13% of patients),13 perhaps related to regional variability in incidence.2 Though not measured in our study, POAF has consistently been associated with increased in-hospital morbidity,2,5,25,26 resource utilization,5,26,27 and short-term mortality2,26 after cardiac surgery. In addition, patients experiencing POAF have an increased long-term risk of chronic atrial fibrillation28 and decreased long-term survival after cardiac surgery.7,29 Importantly, strategies to reduce POAF have been associated with reduced hospital LOS,4 healthcare costs,4,30 and incidence of stroke,31 suggesting the importance of POAF as a major morbid event after cardiac surgery. Nevertheless, strategies to prevent POAF have not appreciably changed the incidence of POAF over the past decade. Better identification of at-risk patients may improve the efficiency and outcomes associated with preventive interventions.

Assessing the predictive capacity of multiple risk prediction models can be complex. The cornerstone of these comparisons is often based on ROC curves, which assess discrimination—the ability to determine who will develop the condition of interest.20 Nevertheless, investigators have also encouraged the use of novel ways to evaluate the predictive capacity of risk models.32 One such technique is the use of the NRI, which sums the correct vs incorrect risk reclassifications in moving from one model to another.22 Based upon the derivation of the formula,23 the NRI is flexible in quantifying differences between two-category (e.g., CHA2DS2Vasc score, Kolek clinical risk prediction model, and POAF score) and three-category (e.g., modified McSPI AFRisk index) risk indices. To address a category-free measure of discrimination improvement, we also present the IDI, which can be used to quantify model improvement that may be missed by comparing AUCs.20 Additionally, the overall performance of a model can be assessed by using Brier scores, which may capture both the discrimination and calibration of a model.21 By combining multiple model performance metrics, we sought to create a more complete comparison of multiple POAF risk models.

In our analysis, the modified McSPI AFRisk index was superior to the CHA2DS2Vasc score, Kolek clinical risk prediction model, and POAF score in all metrics, as evidenced by its higher AUC, higher R2 value, higher adjusted Brier score, higher IDI, and NRI consistent with correctly reclassifying high-risk patients. The AUCs of the modified McSPI AFRisk index and Kolek clinical risk prediction model were similar to originally published values,2,8 and AUC values for the CHA2DS2Vasc and POAF score were very similar to those in a recently published comparison of preoperative POAF risk indices.16 There is much interest in identifying a highly predictive preoperative POAF risk index, and these data indicate that adding perioperative medication information significantly improves the predictive capacity of the modified McSPI AFrisk index. The NRI and IDI23 indicate that the modified McSPI AFRisk index frequently reclassifies POAF risk compared with the CHA2DS2Vasc score, Kolek clinical risk prediction model, and POAF score. Given that NRIs for POAF events were consistently higher than the NRIs for no POAF events, the modified McSPI AFRisk index improves reclassification by more often designating patients as high risk compared with other indices.

Many of the well-known risk factors for POAF, including age,6-10 left ventricular dysfunction,6,8,9 history of AF,2,6,8,9 valvular surgery,2,9 and COPD,2,9 are present in multiple POAF risk indices. Additional included risk factors, such as a prolonged PR interval8 or end-stage renal disease,9 likely serve as markers of vulnerable atrial substrate6 and increased comorbidity burden, respectively. The modified McSPI AFRisk index is unique in its inclusion of comprehensive medication-related data. The advent of complex electronic medical record systems can facilitate automated risk score derivation and provide timely risk stratification information to healthcare providers. While the literature has been mixed regarding the effects of perioperative angiotensin-converting enzyme (ACE) inhibitors,33 both perioperative beta blockers4 and statins34 protect against the development of POAF. Interestingly, withdrawal of preoperative beta blockers or ACE-inhibitor therapy is associated with POAF,2 a finding replicated in this cohort. It is likely that addition of these relevant medication-based factors improves the discriminative power of the modified McSPI AFRisk index.

The primary strength of this study is the prospective data collection and definition of POAF. Additionally, this represents an external validation of each risk index. Limitations of this study include the use of a relatively homogeneous cohort of primarily Caucasian patients undergoing non-emergent cardiac surgery at a large tertiary medical center. These characteristics may limit the generalizability of our results to other populations. In addition, the overall approach of this study is a retrospective cohort analysis of patients undergoing surgery over a 13-year period, during which time cardiac surgical care may have advanced. Nevertheless, perioperative care as well as data collection strategies were standardized throughout this time period. Moreover, this manuscript does not account for post-discharge POAF, which occurs in the minority of patients after cardiac surgery but is less common than in-hospital POAF.35 While this investigation does not address the negative impact of POAF on postoperative recovery profiles, this link has been well established in previous literature, with POAF associated with increased ICU and hospital LOS, higher costs, more frequent readmissions, higher perioperative and long-term mortality, and higher long-term risks of atrial dysrhythmias.5,26,27 Finally, no single performance metric provides a comprehensive assessment of model performance, and both ROC curves22 and the NRI36 have been criticized for providing incomplete measures of performance. Nevertheless, the lack of overlap between AUC confidence intervals of the modified McSPI vs other indices provides a compelling indicator of superiority. In addition, we combined multiple measures to strengthen our ability to assess the performance of each risk index.

In conclusion, the modified McSPI AFRisk index showed improved ability to predict POAF compared with the CHA2DS2Vasc score, POAF score, and Kolek clinical risk prediction model. This may be due to the inclusion of medication-related information, which is unique to the modified McSPI AFRisk index. In future studies, investigators are encouraged to account for perioperative management of common cardiovascular medications, including beta blockers, ACE inhibitors, statins, and potassium supplementation, as these data may better elucidate the risk of POAF. Moreover, as electronic medical records become increasingly complex, physicians could integrate the modified McSPI AFRisk index into perioperative care of cardiac surgical patients, thereby potentially reducing medication withdrawal and so decreasing the incidence of POAF. Future research is necessary to determine the generalizability of our results to larger, more heterogeneous populations. As cardiovascular care advances and the surgical population continues to include older, frailer patients, prevention of POAF may prove to be an effective strategy to reduce perioperative morbidity and costs. Paramount to prevention is risk stratification or individual risk prediction, and further work should focus on optimizing the performance of these various POAF risk-prediction scores.

Notes

Conflicts of interest

None declared.

Editorial responsibility

This submission was handled by Dr. Steven Backman, Associate Editor, Canadian Journal of Anesthesia.

Author contributions

Nathan H. Waldron and Joseph P. Mathew contributed substantially to all aspects of this manuscript, including conception and design; acquisition, analysis, and interpretation of data; and drafting the article. Mary Cooter, Jonathan P. Piccini, Kevin J. Anstrom, and Miklos D. Kertai contributed substantially to the conception and design of the manuscript. Rebecca Y. Klinger, Mihai V. Podgoreanu, Mark Stafford-Smith, and Mark F. Newman contributed substantially to the interpretation of data. All authors approved the final version of this article.

Funding

This work was supported by an American Heart Association Mentored Clinical & Population Research Award to the first author, NHW.

References

  1. 1.
    Mitchell LB, Exner DV, Wyse DG, et al. Prophylactic oral amiodarone for the prevention of arrhythmias that begin early after revascularization, valve replacement, or repair: PAPABEAR: a randomized controlled trial. JAMA 2005; 294: 3093-100.CrossRefPubMedGoogle Scholar
  2. 2.
    Mathew JP, Fontes ML, Tudor IC, et al. A multicenter risk index for atrial fibrillation after cardiac surgery. JAMA 2004; 291: 1720-9.CrossRefPubMedGoogle Scholar
  3. 3.
    Kertai MD, Li YW, Li YJ, et al. G protein-coupled receptor kinase 5 gene polymorphisms are associated with postoperative atrial fibrillation following coronary artery bypass graft surgery in patients receiving beta-blockers. Circ Cardiovasc Genet 2014; 7: 625-33.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Arsenault KA, Yusuf AM, Crystal E, et al. Interventions for preventing post-operative atrial fibrillation in patients undergoing heart surgery. Cochrane Database Syst Rev 2013; 1: CD003611.Google Scholar
  5. 5.
    LaPar DJ, Speir AM, Crosby IK, et al. Postoperative atrial fibrillation significantly increases mortality, hospital readmission, and hospital costs. Ann Thorac Surg 2014; 98: 527-33; discussion 533.Google Scholar
  6. 6.
    Amar D, Shi W, Hogue CW Jr, et al. Clinical prediction rule for atrial fibrillation after coronary artery bypass grafting. J Am Coll Cardiol 2004; 44: 1248-53.CrossRefPubMedGoogle Scholar
  7. 7.
    El-Chami MF, Kilgo P, Thourani V, et al. New-onset atrial fibrillation predicts long-term mortality after coronary artery bypass graft. J Am Coll Cardiol 2010; 55: 1370-6.CrossRefPubMedGoogle Scholar
  8. 8.
    Kolek MJ, Muehlschlegel JD, Bush WS, et al. Genetic and clinical risk prediction model for postoperative atrial fibrillation. Circ Arrhythm Electrophysiol 2015; 8: 25-31.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Mariscalco G, Biancari F, Zanobini M, et al. Bedside tool for predicting the risk of postoperative atrial fibrillation after cardiac surgery: the POAF score. J Am Heart Assoc 2014; 3: e000752.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Tran DT, Perry JJ, Dupuis JY, Elmestekawy E, Wells GA. Predicting new-onset postoperative atrial fibrillation in cardiac surgery patients. J Cardiothorac Vasc Anesth 2015; 29: 1117-26.CrossRefPubMedGoogle Scholar
  11. 11.
    Klinger RY, Thunberg CA, White WD, et al. Intraoperative magnesium administration does not reduce postoperative atrial fibrillation after cardiac surgery. Anesth Analg 2015; 121: 861-7.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Lip GY, Nieuwlaat R, Pisters R, Lane DA, Crijns HJ. Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the Euro Heart Survey on atrial fibrillation. Chest 2010; 137: 263-72.CrossRefPubMedGoogle Scholar
  13. 13.
    Borde D, Gandhe U, Hargave N, Pandey K, Mathew M, Joshi S. Prediction of postoperative atrial fibrillation after coronary artery bypass grafting surgery: is CHA 2 DS 2 -VASc score useful? Ann Card Anaesth 2014; 17: 182-7.CrossRefPubMedGoogle Scholar
  14. 14.
    Chua SK, Shyu KG, Lu MJ, et al. Clinical utility of CHADS2 and CHA2DS2-VASc scoring systems for predicting postoperative atrial fibrillation after cardiac surgery. J Thorac Cardiovasc Surg 2013; 146: 919-926.e1.CrossRefPubMedGoogle Scholar
  15. 15.
    Yin L, Ling X, Zhang Y, et al. CHADS2 and CHA2DS2-VASc scoring systems for predicting atrial fibrillation following cardiac valve surgery. PLoS One 2015; 10: e0123858.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Cameron MJ, Tran DT, Abboud J, Newton EK, Rashidian H, Dupuis JY. Prospective external validation of three preoperative risk scores for prediction of new onset atrial fibrillation after cardiac surgery. Anesth Analg 2018; 126: 33-8.CrossRefPubMedGoogle Scholar
  17. 17.
    Kertai MD, Li YJ, Ji Y, et al. Genome-wide association study of new-onset atrial fibrillation after coronary artery bypass grafting surgery. Am Heart J 2015; 170: 580-90.e28.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Harrell FE Jr, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med 1984; 3: 143-52.CrossRefPubMedGoogle Scholar
  19. 19.
    Brier GW. Verification of forecasts expressed in terms of probability. Monthly Weather Review 1950; 78: 1-3.CrossRefGoogle Scholar
  20. 20.
    Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157-72; discussion 207-12.Google Scholar
  21. 21.
    Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21: 128-38.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Tzoulaki I, Liberopoulos G, Ioannidis JP. Use of reclassification for assessment of improved prediction: an empirical evaluation. Int J Epidemiol 2011; 40: 1094-105.CrossRefPubMedGoogle Scholar
  23. 23.
    Pepe MS, Feng Z, Gu JW. Comments on ‘Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI:  https://doi.org/10.1002/sim.2929). Stat Med 2008; 27: 173-81.
  24. 24.
    Mariscalco G, Engstrom KG. Postoperative atrial fibrillation is associated with late mortality after coronary surgery, but not after valvular surgery. Ann Thorac Surg 2009; 88: 1871-6.CrossRefPubMedGoogle Scholar
  25. 25.
    Ahlsson A, Fengsrud E, Bodin L, Englund A. Postoperative atrial fibrillation in patients undergoing aortocoronary bypass surgery carries an eightfold risk of future atrial fibrillation and a doubled cardiovascular mortality. Eur J Cardiothorac Surg 2010; 37: 1353-9.CrossRefPubMedGoogle Scholar
  26. 26.
    Steinberg BA, Zhao Y, He X, et al. Management of postoperative atrial fibrillation and subsequent outcomes in contemporary patients undergoing cardiac surgery: insights from the Society of Thoracic Surgeons CAPS-Care Atrial Fibrillation Registry. Clin Cardiol 2014; 37: 7-13.CrossRefPubMedGoogle Scholar
  27. 27.
    Hravnak M, Hoffman LA, Saul MI, Zullo TG, Whitman GR. Resource utilization related to atrial fibrillation after coronary artery bypass grafting. Am J Crit Care 2002; 11: 228-38.PubMedPubMedCentralGoogle Scholar
  28. 28.
    Melduni RM, Schaff HV, Bailey KR, et al. Implications of new-onset atrial fibrillation after cardiac surgery on long-term prognosis: a community-based study. Am Heart J 2015; 170: 659-68.CrossRefPubMedGoogle Scholar
  29. 29.
    Sigurdsson MI, Longford NT, Heydarpour M, et al. Duration of postoperative atrial fibrillation after cardiac surgery is associated with worsened long-term survival. Ann Thorac Surg 2016; 102: 2018-26.CrossRefPubMedGoogle Scholar
  30. 30.
    Zebis LR, Christensen TD, Kristiansen IS, Hjortdal VE. Amiodarone cost effectiveness in preventing atrial fibrillation after coronary artery bypass graft surgery. Ann Thorac Surg 2008; 85: 28-32.CrossRefPubMedGoogle Scholar
  31. 31.
    Aasbo JD, Lawrence AT, Krishnan K, Kim MH, Trohman RG. Amiodarone prophylaxis reduces major cardiovascular morbidity and length of stay after cardiac surgery: a meta-analysis. Ann Intern Med 2005; 143: 327-36.CrossRefPubMedGoogle Scholar
  32. 32.
    Greenland P, O’Malley PG. When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk. Arch Intern Med 2005; 165: 2454-6.CrossRefPubMedGoogle Scholar
  33. 33.
    Mangieri A. Renin-angiotensin system blockers in cardiac surgery. J Crit Care 2015; 30: 613-8.CrossRefPubMedGoogle Scholar
  34. 34.
    Kuhn EW, Liakopoulos OJ, Stange S, et al. Preoperative statin therapy in cardiac surgery: a meta-analysis of 90,000 patients. Eur J Cardiothorac Surg 2014; 45: 17-26; discussion 26.Google Scholar
  35. 35.
    Bidar E, Maesen B, Nieman F, Verheule S, Schotten U, Maessen JG. A prospective randomized controlled trial on the incidence and predictors of late-phase postoperative atrial fibrillation up to 30 days and the preventive value of biatrial pacing. Heart Rhythm 2014; 11: 1156-62.CrossRefPubMedGoogle Scholar
  36. 36.
    Kerr KF, Wang Z, Janes H, McClelland RL, Psaty BM, Pepe MS. Net reclassification indices for evaluating risk prediction instruments: a critical review. Epidemiology 2014; 25: 114-21.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Canadian Anesthesiologists' Society 2018

Authors and Affiliations

  • Nathan H. Waldron
    • 1
    • 2
    Email author
  • Mary Cooter
    • 1
  • Jonathan P. Piccini
    • 2
    • 3
  • Kevin J. Anstrom
    • 2
  • Rebecca Y. Klinger
    • 1
  • Miklos D. Kertai
    • 1
    • 4
  • Mihai V. Podgoreanu
    • 1
  • Mark Stafford-Smith
    • 1
  • Mark F. Newman
    • 1
  • Joseph P. Mathew
    • 1
  1. 1.Department of AnesthesiologyDuke University Medical CentreDurhamUSA
  2. 2.Duke Clinical Research InstituteDuke University School of MedicineDurhamUSA
  3. 3.Division of CardiologyDuke University Medical CenterDurhamUSA
  4. 4.Department of AnesthesiologyVanderbilt University Medical CenterNashvilleUSA

Personalised recommendations