1 Introduction

Measurement of cardiac output (CO) has been a mainstay of hemodynamic monitoring in the perioperative management of cardiovascular surgery, and a pulmonary artery catheter (PAC) has been considered as a gold standard for measurement of CO. However, the frequency of use of PAC is gradually decreasing due to the lack of studies demonstrating clinical benefits [1, 2]. Alternative and minimally invasive techniques for measuring CO, such as arterial pressure-based CO (APCO) [3], transthoracic bioimpedance [4], and transthoracic echocardiography [5], have been developed and are widely used to avoid risks associated with PAC use.

Recently, the estimated continuous CO (esCCO) system, which can continuously monitor CO using the pulse wave transit time (PWTT) technique, has been developed as a non-invasive technique and its efficacy evaluated in various types of patients, including surgical [6, 7] and critically-ill patients in the intensive care unit (ICU) [8]. The principle of CO measurement using esCCO is based on the inverse relationship between PWTT, which is the interval between the peak of the R wave on the electrocardiogram (ECG) and the rising phase of the percutaneous oxygen saturation (SpO2) pulse wave, and stroke volume (SV). Due to this relationship, CO can be calculated using the following formula with PWTT, heart rate (HR), and three numeric constants (K, α, and β), using an ECG monitor and SpO2 waveform only without additional devices [9]:

$${\text{esCCO}}=K \times (a \times {\text{PWTT}}+\beta ) \times {\text{HR}}$$

Among the minimally invasive hemodynamic monitoring devices, APCO has been widely used for CO and stroke volume variation (SVV) measurement, and previous clinical studies have demonstrated that the APCO system can perform precise CO measurements and SVV can be used as a reliable variable to predict fluid responsiveness [10, 11]. Recently, esCCO has also been ameliorated to continuously measure SVV, but its efficacy has not yet been evaluated in the clinical setting. Furthermore, studies directly comparing esCCO and APCO, despite the frequent use of APCO among surgical ICU patients, including cardiovascular surgery patients, have been limited. This study aimed to evaluate the accuracy, precision, and interchangeability of CO and SVV measured using esCCO in the post-operative period among cardiovascular surgery patients, compared with that of APCO.

2 Patients and methods

2.1 Patients

This prospective observational study was conducted in the 10-bed capacity ICU of Keio University Hospital, which is a 1044-bed capacity teaching hospital. Intensivists treat all patients in cooperation with attending physicians and medical specialists in our ICU. This study was approved by the ethics committee of Keio University School of Medicine (approval number: 20130515) and registered at the UMIN Clinical Trials Registry (http://www.umin.ac.jp/, registration number UMIN000013984) before enrollment of patients. Patients who underwent cardiovascular surgeries under hemodynamic monitoring using the APCO system (Flo Trac system: Edwards Lifesciences, CA, USA) were enrolled in this study. All participants provided written informed consent before surgery. Patients younger than 20 years and with arrhythmias, pacemaker, and circulatory support devices before surgery and those who presented continuous arrhythmias were excluded from the study.

2.2 Study protocol

Anesthetic management during the surgical procedure was entrusted to the attending anesthesiologists. All patients were admitted to the ICU after operation with invasive mechanical ventilation. Measurement of CO and SVV by two different methods, esCCO and APCO systems, were initiated as soon as patients were admitted to the ICU. CO and SVV were simultaneously and continuously monitored with two devices after the esCCO calibration using the APCO value. CO values were recorded every 6 h after initiation of measurement (6, 12, 18, 24 h after admission) until discharge from the ICU, and SVV values were recorded every hour while patients were mechanically ventilated under assist/control ventilation mode or synchronized intermittent mandatory ventilation mode with respiratory rate of ≥ 10/min.

Doses of sedative and analgesic drugs (propofol, dexmedetomidine, and fentanyl) were adjusted by the intensivists or attending physicians to maintain Richmond Agitation Sedation Scale scores from − 3 to 0 according to the patient’s condition. Delirium, which was evaluated by the attending nurses using Confusion Assessment Method of ICU (CAM-ICU), was treated by intravenous administration of haloperidol as the first choice, and atypical antipsychotics as the second choice. Ventilator setting was also adjusted by the intensivists or attending physicians, and the timing of weaning from mechanical ventilation was determined between the intensivists and attending physicians. Hemodynamic monitoring by means of APCO and esCCO were continued until patients were transferred to the high care unit.

2.3 Measurement procedure of esCCO and APCO

ECG was monitored using lead II, and a pulse oximeter probe was placed on the fingertip of an upper limb into which an arterial line for APCO was not inserted. The side for placement of the arterial line was determined by the cardiologist before surgery. ECG and pulse oximetry wave, which are required to calculate pulse wave transit time, were obtained from a BSM-9101 bedside monitor (Nihon Kohden, Tokyo, Japan) and transmitted to a personal computer equipped with a C-compiled program for continuous esCCO calculation. esCCO was calculated using the following formula: esCCO = K × (α × PWTT + β) × HR (K, α, and β are numeric constants). SVV based on the esCCO system (esSVV) was also calculated simultaneously. Invasive blood pressure was continuously monitored through the cannula inserted into the radial artery, through which APCO and SVV based on APCO (APSVV) were measured using an Edwards Vigileo Monitor (Edwards Lifesciences, CA, USA). Calibration had to be performed before the continuous monitoring of esCCO. While α is a numerical constant determined before calibration, β and K are calculated after calibration using four values: CO measured by the alternative method (APCO in this study), and 3–10 min averaged PWTT, HR, and PP (pulse pressure: systolic pressure − diastolic pressure). After the calibration period, esCCO was initiated to monitor CO and SVV continuously.

2.4 Statistics

Based on a previous study, which compared esCCO and thermodilution CO (TDCO) measurement [12], 136 points were considered to be required after calculating the sample size to detect a difference of 0.3 L/min between the two devices at the significance level of 0.05 with a statistical power of 80%, assuming that the standard deviation of the differences was 1.25 L/min.

Correlation and precision analyses were performed using Spearman’s and Bland–Altman analyses, respectively. Multiple measurements per patient were performed for these analyses. Regarding the Bland–Altman analysis, we defined a reliable percentage error as less than 45%, following a previous study [13]. Welch’s analysis of variance was conducted at every measurement point to evaluate the changes of biases between the two devices according to the time course. The trending ability of esCCO was evaluated using the four-quadrant plot and polar plot analyses with the exclusion zone being 15% of the mean APCO [14]. Based on the exclusion zone as 15% of the mean APCO, we analyzed the data in which APCO changes between the two consecutive measurements were > 15% of the mean APCO value to evaluate the trending ability more precisely. A concordance rate of > 80% was defined as an acceptable value based on a previous report that compared TDCO and APCO [15], which was evaluated at 30° in the polar plot analysis [16]. The results were presented as mean ± standard deviation or median (interquartile range) when appropriate. P-values < 0.05 were considered statistically significant.

3 Results

3.1 Patient characteristics

This study was conducted from July 2014 to September 2015. A total of 21 patients who underwent cardiovascular surgeries were included in this study. Table 1 shows the patient characteristics. The mean age of the participants was 64 ± 15 years, and most patients (17/21) were men. Nineteen patients underwent operation for aortic aneurysm, while only two patients were operated for mitral valve diseases. The durations of invasive mechanical ventilation and measurements of CO and SVV were 24 ± 32 and 55 ± 36 h, respectively.

Table 1 Patient characteristics

3.2 Analysis of CO measured by esCCO and APCO

A total of 143 measurement points were used for CO analysis. Figure 1 shows the results of correlation and precision analyses between esCCO and APCO evaluated using Spearman’s and Bland–Altman analyses, respectively. The correlation analysis demonstrated a significantly strong correlation between esCCO and APCO with a 0.62 correlation coefficient (p < 0.01). Bland–Altman analysis revealed that mean APCO, bias, and precision were 5.68, 0.14, and 1.96 L/min respectively, and the 95% limits of agreement (bias ± 2 × precision) was − 3.78 to 4.05 L/min. The percentage error of 69%, which was twice the percentage of precision to the mean APCO value and served as an indicator of interchangeability, was higher than the reliable percentage error of 45%.

Fig. 1
figure 1

The results of CO analysis between esCCO and APCO. The values of 143 measurement points were collected and analyzed to compare CO measured by esCCO and APCO. A The result of Spearman analysis. B The result of Bland–Altman analysis

3.3 Analysis of SVV measured by esCCO and APCO

To evaluate SVV, a total of 146 measurement points were gathered. Figure 2 shows the results of the correlation and precision analyses. Correlation analysis revealed that the correlation coefficient between esSVV and APSVV was 0.4 (p < 0.01), which was lower compared with that of CO. Precision analysis due to Bland–Altman analysis showed that the mean APSVV, bias, and precision were 10.3, − 3.79, and 5.08% respectively, and the 95% limits of agreement (bias ± 2 × precision) were − 13.94 to 6.37%. The percentage error was 99%, which was high, indicating that the interchangeability was not acceptable.

Fig. 2
figure 2

The results of SVV analysis between esCCO and APCO. The values of 146 measurement points were collected and analyzed to compare SVV measured by esCCO and APCO. A The result of Spearman analysis. B The result of Bland–Altman analysis

3.4 The changes of biases in CO and SVV measured by esCCO and APCO according to the time course

Figure 3 presents the changes of biases between esCCO and APCO according to the time course after calibration. No significant differences in CO and SVV biases were observed during the measurement time course after calibration (p = 0.95 for CO and p = 0.94 for SVV). These results indicate that the precision of esCCO is not affected by the time.

Fig. 3
figure 3

The changes of biases during the study period. These figures represented the changes of biases during the study period. A The change of CO bias between esCCO and APCO. Measurement points more than 48 h apart were compiled at 48 h. B The change of SVV bias between esCCO and APCO

3.5 The ability of esCCO to trend with changes in CO

The results of the four-quadrant plot and polar plot analyses, which were performed to evaluate the trending ability of esCCO, are shown in Fig. 4. Regarding the four-quadrant plot analysis, the number of measurement points was 71, and the concordance rate was 80.3%, which was acceptable. The polar plot analysis with 37 measurement points showed a mean polar angle of − 4.4% and a concordance rate of 75.7% at 30°, which showed that the trending ability of esCCO was almost acceptable [16].

Fig. 4
figure 4

Evaluation of the trending ability of esCCO. A The result of four-quadrant plot analysis. B The result of polar plot analysis

4 Discussion

This is the first study to compare CO and SVV between the esCCO system based on PWTT analysis and APCO system based on arterial waveform analysis directly in postoperative patients in the ICU. Regarding CO, esCCO was significantly correlated with APCO, as shown by the correlation coefficient of 0.62. The Bland–Altman analysis showed that the bias was 0.14 L/min, which was comparable to the value of reference, and the precision of 1.96 L/min was slightly higher than the reference [12]. The percentage error of 69% was slightly higher compared with those reported in previous studies. Regarding the trending ability, the four-quadrant plot and polar plot analyses revealed that the ability of esCCO to trend with changes in CO could be justified. However, with regard to SVV, esSVV did not correlate so well with APSVV (r = 0.4) as compared with esCCO. The bias of − 3.79%, precision of 5.08%, and the percentage error of 99% indicate that SVV evaluated by esCCO was not reliable and acceptable. These results suggest that continuous CO measurement using the esCCO system is a potentially reliable non-invasive hemodynamic monitoring method, while further studies are warranted for esSVV as a reliable hemodynamic monitoring tool in the clinical setting.

4.1 Agreement, accuracy, and interchangeability between esCCO and APCO, compared with previous studies

Although few studies compared esCCO and APCO simultaneously and directly among postoperative patients, some previous studies have evaluated the agreement, accuracy, and interchangeability between esCCO and other CO measurement techniques among various types of patients. Among studies which compared esCCO and TDCO measurement, Ishihara et al. compared esCCO with continuous TDCO measured by a pulmonary artery catheter in patients scheduled for elective cardiac surgery and showed that the correlation coefficient was 0.8 and the bias ± precision was − 0.06 ± 0.82 L/min [9]. Another multicenter study conducted by Yamada et al. [12], which included both ICU and intraoperative patients, showed that the correlation between esCCO and bolus TDCO measured by pulmonary artery catheterization was good (r = 0.79), and the bias ± precision was 0.13 ± 1.15 L/min. In the study which included only post off-pump coronary artery bypass grafting surgery patients [6], postoperative evaluation of esCCO compared with intermittent TDCO revealed that the bias was 0.4 L/min with a precision of 1.15 L/min and percentage error of 41%.

Regarding studies that compared esCCO and APCO, Terada et al. evaluated esCCO and APCO simultaneously in 15 patients who underwent kidney transplant surgery, compared with intermittent bolus TDCO, and revealed that the difference (bias ± precision) and percentage error between esCCO and TDCO were − 0.39 ± 1.15 L/min and 35.6%, respectively, while those between APCO and TDCO were 0.04 ± 1.37 L/min and 42.4%, respectively [17]. They concluded that the trending ability of esCCO was comparable with APCO. In another study that compared esCCO and APCO simultaneously [18], the bias between the two systems was reported to be 0.6 L/min, which was a higher value compared with those reported in other studies.

Although the bias of 0.14 L/min in our study was quite comparable to those of previous studies, the correlation coefficient of 0.62 and the precision of 1.96 L/min were slightly inferior to those reported in previous studies. Furthermore, regarding the percentage error (2 × precision/mean APCO), which is an indicator of interchangeability, the value of 69% in our study was higher than 41, 54, and 48.5% reported in previous studies [6, 12, 19], which compared esCCO and TDCO, and was also higher than the upper limit of the 45% percentage error defined as a reliable value [13]. However, considering that TDCO measurement is the gold standard method and there is a deviation between TDCO and APCO, it is quite natural that a larger deviation exists between esCCO and APCO. Since the bias between esCCO and APCO did not change during the study procedure (42 h after ICU admission) in our study, CO monitoring by esCCO could be used for a long period after a single calibration.

4.2 Reliability of SVV measured by esCCO (esSVV), compared with SVV based on APCO (APSVV), in evaluating intra-vascular volume status

To our knowledge, studies evaluating SVV based on PWTT analysis in clinical situations were limited. As shown in previous studies, SVV has been used as a reliable hemodynamic variable to predict fluid responsiveness in critically ill patients, even though several limitations were pointed out [10, 11]. Thus, if SVV can be measured continuously, precisely, and noninvasively by using esCCO, it is very ideal. However, the correlation coefficient of 0.4, the bias ± precision of -3.79 ± 5.08%, and percentage error of 99% in our study suggest that the SVV measurement system based on PWTT may require further amelioration of the measurement algorithm.

The discrepancy between SVV measured by esCCO and APCO systems may be attributable to the difference in the underlying principles between the two devices. The APCO system, which measures CO based on an arterial waveform analysis, calculates single values of SV, and SVV using maximum, minimum, and average SV during the preceding 20 s. In contrast, the esCCO system, which measures CO based on PWTT analysis, calculates single values of SV, and SVV using the same three values as used in the APCO system during each breath. Thereafter, the esCCO system presents a mean SVV value over 32 breaths. This difference in measurement principles could contribute to the disagreement between SVV values measured by esCCO and APCO systems. In order to evaluate intravascular volume status and accurately predict responsiveness to volume challenge using the esCCO system, further improvement might be necessary for this device to be able to track acute preload changes.

4.3 The ability of esCCO to trend with changes in CO

Regarding the esCCO response to fluid challenge, some previous studies evaluated the ability of esCCO to track changes in CO induced by the changes in preload. In a study which evaluated the changes in esCCO and intermittent bolus TDCO after change of body position, volume challenge, and the Pringle maneuver in patients undergoing partial hepatectomy, Tsutsui et al. showed that the direction of CO change between esCCO and TDCO represented a good concordance rate of 96% and concluded that esCCO could provide good trending abilities despite the wide range of limits of agreement [20]. Feissel et al. assessed the abilities of esCCO to detect CO changes after fluid infusion in septic shock patients as a secondary outcome compared with transthoracic echocardiography, and showed that a threshold of 11% increase in esCCO discriminated responders from non-responders with a sensitivity of 83% and a specificity of 77% [21]. These results purported that esCCO can be used as a reliable hemodynamic monitoring system for intravascular volume change in critically-ill patients. However, in a study [22] that investigated whether esCCO could track changes in CO induced by volume expansion and passive leg raising in critically-ill patients as compared with transthoracic echocardiography, Biais et al. concluded that esCCO could not track preload-induced CO change accurately by showing the concordance rate of 83% and angular bias of − 20°.

In this study, we evaluated the trending ability of esCCO by the four-quadrant plot and polar plot analyses, even though the number of measurement points per analysis was reduced due to the effect of the 15% exclusion zone. In the analysis of the trending ability, a concordance rate more than 90% is usually considered to indicate reliable trending ability when thermodilution is the reference method [14]. However, considering that the concordance rate of APCO compared with TDCO was reported to be 81% [15], the concordance rates of this study (80.3 and 75.7%, respectively) were comparable to the value reported in the previous report. Although a prospective study to examine the change in esCCO induced by volume resuscitation is required for evaluation of the trending ability of esCCO, the results of this study have shown the possibility of esCCO to trend with changes in CO accurately.

4.4 Limitations of this study

Our study has several limitations that should be considered. First, CO was not evaluated with a pulmonary artery catheter based on the thermodilution technique, which is the gold standard method for measuring CO. However, hemodynamic monitoring using a pulmonary catheter is not routinely used during perioperative management for patients undergoing cardiovascular surgeries in our hospital, and simultaneous monitoring using both TDCO and APCO is not practical in the clinical setting. Since APCO has been a reliable perioperative hemodynamic monitoring system in many previous studies [10, 11], it is quite a valid method to compare esCCO with APCO directly in this study. Second, whether the APCO system can precisely and continuously monitor CO is questionable, given that APCO values, which were calculated based on the wave of invasive arterial pressure line, were sometimes unreliable due to the flexion of the joint where an invasive arterial line was inserted. Considering that the nursing staff taking care of patients in our ICU were always paying attention to ensure that the arterial pressure wave was not affected by the movement of joints, the accuracy of APCO must be guaranteed. Third, we did not evaluate the effects of systemic vascular resistance, which has been shown to affect the esCCO value in previous studies [7, 8, 20]. Although the effects of systemic vascular resistance should be examined in future studies before esCCO application in the clinical setting, we believe that we accomplished the aim of this study to evaluate the accuracy of esCCO in post-cardiovascular surgery patients compared with APCO. Fourth, patients with continuous arrhythmias after surgery were excluded from this study, which might be an obstacle to the generalization of the study results. However, considering that there was only one patient who presented continuous arrhythmias after surgery in this study, the generalizability of the study results should not be denied. Finally, the results of this study are not applicable to other types of ICU patients, since only post-cardiovascular surgery patients were enrolled in this study and the majority of patients underwent operation for aortic aneurysm (19/21). Furthermore, the replaced vascular graft may affect PWTT measurement in comparison with the normal aorta. Further studies seem necessary to evaluate the accuracy and interchangeability of esCCO in other types of critically-ill patients.

4.5 Conclusion

In this study, we evaluated the accuracy, precision, and interchangeability of CO and SVV measured by the esCCO system, which can calculate these values based on PWTT, compared with APCO. The correlation and precision analysis revealed that while continuous CO measurement using esCCO was almost acceptable, SVV measured by esCCO was not so reliable. A further improved algorithm may be required for esSVV to predict an intravascular volume status.