Advertisement

Delphi Study to Reach International Consensus Among Vascular Surgeons on Major Arterial Vascular Surgical Complications

  • S. M. L. de Mik
  • F. E. Stubenrouch
  • D. A. Legemate
  • R. Balm
  • D. T. UbbinkEmail author
  • the DISCOVAR Study Group
Open Access
Original Scientific Report
  • 2 Downloads

Abstract

Background

The complications discussed with patients by surgeons prior to surgery vary, because no consensus on major complications exists. Such consensus may improve informed consent and shared decision-making. This study aimed to achieve consensus among vascular surgeons on which complications are considered ‘major’ and which ‘minor,’ following surgery for abdominal aortic aneurysm (AAA), carotid artery disease (CAD) and peripheral artery disease (PAD).

Methods

Complications following vascular surgery were extracted from Cochrane reviews, national guidelines, and reporting standards. Vascular surgeons from Europe and North America rated complications as major or minor on five-point Likert scales via an electronic Delphi method. Consensus was reached if ≥ 80% of participants scored 1 or 2 (minor) or 4 or 5 (major).

Results

Participants reached consensus on 9–12 major and 6–10 minor complications per disease. Myocardial infarction, stroke, renal failure and allergic reactions were considered to be major complications of all three diseases. All other major complications were treatment specific or dependent on disease severity, e.g., spinal cord ischemia, rupture following AAA repair, stroke for CAD or deep wound infection for PAD.

Conclusion

Vascular surgeons reached international consensus on major and minor complications following AAA, CAD and PAD treatment. This consensus may be helpful in harmonizing the information patients receive and improving standardization of the informed consent procedure. Since major complications differed between diseases, consensus on disease-specific complications to be discussed with patients is necessary.

The complications surgeons choose to discuss with their patients prior to surgery differ [1, 2]. According to informed consent guidelines, surgeons are obliged to discuss the major and most frequently occurring complications with their patient [3, 4]. However, no consensus exists on which specific complications are considered to be major.

Evidently, there are no objective criteria to determine whether complications are major or minor. In these situations, the Delphi method can help fill the knowledge gap by reaching consensus using the knowledge and personal opinions of experts. The RAND corporation originally developed the Delphi method to predict the impact of technology on warfare in the 1950s [5]. Since then, its use has expanded into other areas, including healthcare. For example, using the Delphi method, experts have reached consensus on which complications to discuss with patients prior to treatment for varicose veins, colorectal cancer and skin cancer [6, 7, 8].

Consensus on which specific major complications to discuss with patients will harmonize how surgeons inform their patients and obtain informed consent. Moreover, informing patients about potential major complications allows them to assess their values and preferences regarding available treatment options.

Patients with vascular surgical diseases may particularly benefit from this assessment as there is usually a conservative, endovascular and/or open surgical treatment option available, each with their own benefits and complications [9].

Thus, the aim of this study was to reach consensus on which complications are considered ‘major’ and which ‘minor’ following treatment for abdominal aortic aneurysm (AAA), carotid artery disease (CAD) and peripheral artery disease (PAD) using the knowledge and personal opinions of vascular surgeons from Europe and North America.

Methods

Study design

The Delphi method typically presents participants with 3–5 rounds of a fixed set of questions [10]. After each round, participants receive a summary of responses from the previous round. Based on this summary, participants may adjust their answers in the following round. This process continues until participants reach consensus or if no additional consensus is expected. The Delphi method allows researchers to include a large number of participants, anonymously and throughout the world. It also prevents one expert from dictating consensus [10].

Participants

Participants in a Delphi study are usually experts on the topic on which consensus is sought. Their scientific and practical knowledge of potential complications, and their experience in the consulting room, equip vascular surgeons with an expert opinion on whether a complication is major or minor. This study focused on complications following elective treatment for AAA, CAD and/or PAD. Thus, vascular surgeons were eligible for participation if they treated patients with one of these diseases and had either published articles about the disease or performed ≥5 interventions for this disease during the previous year.

With permission from the organisers, vascular surgeons who had attended the 2015 VEITH symposium were contacted via personal email and asked to participate. Dutch vascular surgeons were contacted via the Dutch Society of Vascular Surgery. Participants were selected in alphabetical order until the email addresses of at least 50 vascular surgeons per disease had been obtained.

Complications

One researcher extracted all reported complications from reference articles used in Cochrane systematic reviews on AAA and CAD [11, 12]. The Dutch peripheral vascular disease guideline was used for PAD [13]. Complications included 30-day and long-term complications as well as complications from open and endovascular surgery. Death was not included as a complication in this study as the authors presumed all participants would consider death to be a major complication. A second researcher verified the extraction. From the literature, 24–30 complications per disease were extracted. Following the first round, participating experts were asked to suggest complications they deemed missing from the survey. These were added in the second round.

Complications can have differing consequences that affect whether a complication is viewed as major or minor. Therefore, complications were extended by a description based on the three-tiered severity scoring of the Society for Vascular Surgery (SVS) reporting standards [14, 15, 16]. An example of this is the consequences of spinal cord ischemia which are mild if resolved within 24 h. However, consequences can also be regarded as moderate if the ability to walk without support is regained within one month, and as severe if paraplegia remains permanent. If a complication was not addressed by the SVS reporting standards, the Clavien–Dindo classification combined with information from reference articles or daily practice was used to present the differing consequences [17]. In the surveys, complications were presented in alphabetical order to avoid ranking bias.

Delphi method modifications

The researchers made several modifications to the classic Delphi method [10]. First, an electronic Delphi method was used, i.e., the surveys were developed online and sent via email using the online tool SurveyMonkey (San Mateo, California, USA). Second, each disease had its specific list of complications and its own survey. Hence, we actually carried out three separate studies. Third, to avoid lengthy surveys containing 30 complications, each with three differing consequences, the first Delphi round started by presenting the moderate severity levels of each complication. If participants deemed this complication as major, it was assumed that they would also rate the corresponding severe level as major. In the next round, participants were asked to rate the mild level of this complication. Similarly, if they deemed the moderate complication to be minor, the mild level was also considered as minor. In the next round, participants were asked to rate the severe level of the same complication. Figure 1 provides an overview of this process.
Fig. 1

Overview of the modified Delphi method used in this study

Consensus

In every round, participants rated each complication on a five-point Likert scale, ranging from 1 (definitely minor) to 5 (definitely major). No strict guidelines exist within the Delphi method concerning the appropriate level of consensus. This level depends on the severity of the issue on which consensus is sought. A 51% level of consensus may be acceptable for decisions about a new hospital logo, whereas a 100% level of consensus is required for life or death situations. The researchers decided that an 80% level of consensus should suffice for this study [10]. Thus, if ≥80% of participants gave a complication a Likert score of 1 or 2, it was deemed minor. Likewise, if ≥80% of participants gave a complication a Likert score of 4 or 5, it was deemed major.

Number of rounds

Delphi studies have a minimum of two rounds. The maximum number of rounds depends on the number necessary to reach either consensus on all questions or the stage at which no additional consensus is expected. This Delphi study originally comprised four rounds. Post hoc, we added a fifth round to limit the possibility of missing major complications for which participants had not reached consensus on the moderate level. Participants had two weeks to complete each round.

Data analysis

The researchers assessed the reliability of the surveys by calculating the internal consistency of the first Delphi round expressed by the Cronbach’s alpha, using IBM SPSS Statistics v.23 (Armonk, New York, USA). A Cronbach’s alpha value of 0.7 or higher is regarded as an acceptable level of internal consistency, while a value of 0.9 or higher is considered excellent [18].

Following each round, the Likert scores from all participants for each complication were collected. The online survey tool automatically turned these scores into percentages. In the next round, the survey reported these percentages back to participants anonymously. Figure 2 shows an example of this feedback.
Fig. 2

Example of feedback presented in Delphi rounds

Results

Sixty-three vascular surgeons were invited for the AAA survey, 50 for CAD and 52 for PAD. Nine vascular surgeons participated in surveys for more than one of these diseases.

In the first round, 19 of 63 surgeons accepted our invitation for AAA, 21 of 50 for CAD and 17 of 52 for PAD. The response rates were 30%, 42% and 33%, respectively. Annually, these participants performed a median number of 48, 50 and 175 interventions for AAA, CAD and PAD, respectively. In the final round, 11 experts in AAA, 16 in CAD and 14 in PAD continued participation, resulting in a 25% total response rate.

Vascular surgeons from 13 different countries participated in this study. Thirty-two surgeons were affiliated to European centers and 14 to North American centers (see Fig. 3). Table 1 shows the characteristics of the participating surgeons. These characteristics did not show statistically significant differences between those surgeons who did or did not participate.
Fig. 3

International distribution of participants

Table 1

Characteristics of the participating vascular surgeons

Characteristics

Participants*

(N = 46)

Experience in years (mean, SD)

29.31 (8.98)

Interventions performed annually (median, IQR)

58 (30–100)

Male vascular surgeons (%)

90.7

Affiliation with a University Hospital (%)

83.7

N number, SD standard deviation, IQR interquartile range

*Some vascular surgeons participated in surveys for more than one vascular disease

In round 1, Cronbach’s alpha scores were 0.96, 0.97 and 0.95 for AAA, CAD and PAD, respectively. Experts suggested the following additional complications: access site pseudoaneurysm, ureteral and bowel lesion for AAA; patch infection, re-stenosis or occlusion, and contrast encephalopathy for CAD; decubitus ulcers, loss of sensibility in the leg, re-stenosis or occlusion, and aneurysm formation for PAD.

Table 2 shows the major and minor complications following intervention in which the surgeons reached consensus per disease. The surgeons reached consensus on 12 major AAA complications, nine CAD complications and nine PAD complications. They also agreed upon nine, six and ten minor complications for AAA, CAD and PAD, respectively. Appendix A provides the complete lists of complications including severity levels. The experts did not reach consensus on some complications, including incisional hernia requiring surgical repair (AAA), pulmonary embolism requiring anticoagulant therapy (CAD), decubitus requiring surgical debridement (PAD).
Table 2

Major and minor complications following treatment on which the vascular surgeons reached consensus per vascular disease

AAA

Major

Endovascular

1

Allergic reaction (due to contrast allergy) requiring to abort treatment and ventilator support

2

Aneurysm rupture requiring endovascular surgical re-intervention or open surgical repair

3

Vascular graft technical deployment problem requiring conversion to open surgery or development of a permanent disability and inability to live independently

Open

4

Bowel lesion requiring primary closure or anastomosis or a colostomy

Both

5

Acute myocardial infarction requiring percutaneous coronary intervention or resuscitation

6

Stroke with permanent deficit but the ability to walk without support or with permanent severe impairment and inability to live independently

7

Renal failure requiring temporary dialysis with permanently reduced renal function requiring surveillance by a nephrologist or permanent dialysis

8

Thrombo-embolic event requiring minor toe or foot amputation, lytic therapy or fem-fem crossover surgery

9

Vascular graft infection controlled with antibiotics or requiring graft removal with in situ repair

10

Congestive heart failure requiring permanent medication or resuscitation

11

Pulmonary embolism requiring anticoagulant therapy or surgical therapy due to hemodynamic instability

12

Spinal cord ischemia regaining the ability to walk without support within 1 month or permanent paraplegia

AAA

Minor

Endovascular

1

Endoleak type 2 with no evidence of aneurysm expansion requiring additional follow-up imaging or is resolved after 6 months

2

Vascular graft technical deployment problem without conversion

3

Arterial access site pseudoaneurysm requiring no additional intervention

4

Vascular graft migration requiring no additional intervention

Open

5

Urinary retention requiring one-time catheterization

Both

6

Superficial wound infection requiring opening of the wound at bedside or oral antibiotics

7

Bowel ischemia requiring no additional support

8

Hemorrhage requiring 1 blood transfusion

9

Wound hematoma with spontaneous resolution

CAD

Major

Endovascular

1

Allergic reaction requiring to abort treatment and ventilator support

2

Renal failure requiring temporary dialysis with permanently reduced renal function requiring surveillance by a nephrologist or permanent dialysis

3

Retroperitoneal hemorrhage requiring both blood transfusion and limited surgical or endovascular intervention for control or massive transfusion and major open surgical intervention

Open

4

Respiratory distress due to neck hematoma requiring surgical evacuation or nerve compression or cervical blow-out requiring arterial repair

Both

5

Acute myocardial infarction requiring percutaneous coronary intervention or resuscitation

6

Ipsilateral non-disabling stroke causing prolonged hospital stay, with permanent deficit with mild impairment or with permanent severe impairment and inability to live independently

7

Carotid artery re-stenosis or occlusion with TIA/minor stroke requiring endovascular or open re-intervention or with major stroke

8

Embolic or hemodynamic stroke resulting in mild or temporary cognitive function impairment

9

Hyperperfusion syndrome requiring intensive monitor surveillance on brain care unit

CAD

Minor

Endovascular

1

Arterial access site pseudoaneurysm requiring thrombin injection or no additional treatment

Both

2

Bradycardia requiring short-term medicinal support or is resolved within 24 hours

3

Wound hematoma requiring prolonged compression

4

Hypertension requiring medicinal support or is resolved within 24 hours

5

Pulmonary infection requiring IV antibiotics or oral antibiotics

6

Cardiac arrhythmias with spontaneous resolution

PAD

Major

Endovascular

1

Renal failure requiring temporary dialysis with permanently reduced renal function requiring surveillance by a nephrologist or permanent dialysis

2

Allergic reaction (due to contrast allergy) requiring to abort treatment and ventilator support

Both

3

Acute limb ischemia due to distal embolization requiring lytic therapy, below-knee amputation or above-knee amputation

4

Acute myocardial infarction requiring percutaneous coronary intervention or resuscitation or coronary artery bypass graft surgery

5

Stroke with permanent deficit but the ability to walk without support or with permanent severe impairment inability to live independently

6

Worsened peripheral artery disease requiring above-knee amputation

7

Pulmonary embolism requiring anticoagulant therapy or surgical therapy due to hemodynamic instability

8

Vascular graft infection controlled with antibiotics or requiring vascular reconstruction

9

Deep wound infection requiring surgical debridement

PAD

Minor

Endovascular

1

Arteriovenous fistula at puncture site requiring no additional intervention

2

Dissection requiring no additional intervention

3

Femoral pseudo-aneurysm at puncture site requiring no additional intervention

Open

4

Transient loss of sensibility on medial side or lower leg

5

Lymphocele or lymphorrhea resolution without aspiration

Both

6

Superficial wound infection requiring opening of the wound at bedside or oral antibiotics

7

Elevated troponin levels requiring surveillance by a cardiologist

8

Pulmonary infection requiring IV antibiotics or oral antibiotics

9

Cardiac arrhythmias with spontaneous resolution

10

Wound hematoma with spontaneous resolution

AAA abdominal aortic aneurysm, CAD carotid artery disease, PAD peripheral artery disease

Discussion

This Delphi study helped vascular surgeons to reach consensus on 9 to 12 major complications following surgery for AAA, CAD and PAD. Additionally, they reached consensus on 6 to 10 minor complications. This will give surgeons the ability to base the complications they discuss with their patients prior to surgery on these sets of major complications.

Understandably, having to discuss up to 12 major complications may lead to some resistance. This continues to make the discussion about which complications to discuss with patients difficult but important. Surgeons want their patients to feel fully informed about their treatment options; however, they do not want to overburden or scare patients by listing all complications, particularly rare but severe complications. Thus, surgeons may not always discuss the risk of paraplegia after aortic repair. Nevertheless, it should be noted that most patients want more information than they currently receive [19]. This was also evident in two previous Delphi studies, which compared the patients’ and physicians’ viewpoints regarding information that should be discussed prior to treatment [7, 8].

Close examination of legislation and guidelines on the informed consent procedure shows that differences exist between countries. Legislation in the UK requires physicians to discuss those risks that a reasonable person in the patient’s position would deem of significance, or that their specific patient would deem significant [4]. The Royal Dutch Medical Association states that physicians must discuss complications occurring in more than 1% of patients as well as less frequently occurring major complications [3]. Unfortunately, neither country provides specific information about the severity and number of major complications surgeons should discuss with patients.

In general, major complications may require additional surgery or endovascular treatment, ICU monitoring, or cause permanent changes. This is why, our participants rated myocardial infarction, stroke, renal failure and allergic reactions as major complications for all three diseases. However, when observing the other major complications, it becomes evident that defining a complication as major depends on the specific disease and treatment. Moreover, it depends on the outcome of weighing the benefits of treating the disease against the severity of a complication, for example, aneurysm rupture following endovascular aortic repair and stroke after CAD and deep wound infection for PAD. Evaluation of these diseases and the complications from Delphi studies on varicose veins, colorectal and skin cancer shows that every disease requires its own list of complications [6, 7, 8]. Therefore, the authors recommend that all specialties develop sets of major complications for each disease they treat to discuss with their patients.

It is also important to realize that the sets of major complications obtained in this study are by no means final. The patients’ viewpoint is currently lacking. Another study to investigate whether the major complications from this study match those that patients consider to be major is ongoing. Adding this knowledge may close the surgeon–patient information gap and empower vascular patients to engage in shared decision-making (SDM) [20]. Previous studies have shown that SDM has a beneficial effect on quality of care and patient satisfaction [21, 22, 23].

However, engaging patients in SDM requires more from surgeons than just discussing major and frequently occurring complications. This first step toward harmonizing the complications under discussion is to ensure that all patients are informed equally about potential complications, which reduces unwarranted variation. Next, surgeons should help patients understand the risks involved and explicitly ask them about their concerns regarding these complications. Decision support tools for SDM are available to help patients grasp relevant information concerning the occurrence and timing of potential complications, while also encouraging patients to contemplate their concerns and preferences [22]. Surgeons must then help patients weigh the benefits and harms of their own situation. This allows surgeons to advise their patients about the treatment option that best fits the patient’s considered opinion and preferences.

The strengths of this study are first that study participation was made as easy as possible by using an electronic questionnaire participants could fill out at any time. In addition, participants received a maximum of 30 complications per round and frequent reminders about completing the questionnaire. To promote adherence, participants received the summary of the previous round and the next questionnaire within two days following each round. Second, all participants were experienced vascular surgeons who reflected the opinions of a number of countries in Europe and North America. This suggests that the consensus reached in this study is valid for a wide range of Western countries. Third, our first Delphi round had high internal consistency. This implies that the items used belonged to a single construct, which is likely because all items were known potential complications following vascular surgical intervention. Fourth, the sets of complications used were comprehensive, as the experts suggested only few additional complications.

Limitations of the study are first the relatively small number of experts per disease who participated fully. Evidence suggests that the more participants, the lower the possibility of reaching consensus. Panel sizes of 5–30 participants are recommended, while 5–10 participants per category are necessary and 15–30 participants in all [24]. Hence, our panel size was considered satisfactory throughout the study. Second, most participating surgeons were affiliated to a university hospital. The frequency and severity with which complications occur may differ between university medical centers and other medical centers, due to a different case mix. Thus, all participants were asked to rate these complications as if they occurred in a ‘regular’ patient. Third, as the researchers decided to start the survey with the moderate level of severity of each complication, where participants did not reach consensus on the moderate level they did not obtain information on the severe level. Therefore, consensus on some major complications may not have been reached. This, however, holds for a minority of complications.

In conclusion, by means of this Delphi study, an international panel of vascular surgeons reached consensus on major complications following treatment for AAA, CAD and PAD. Vascular surgeons should base the complications they discuss with their patients prior to surgery on these sets of major complications. As complications of individual diseases differ substantially, all specialties should have sets of complications available for the diseases they treat to discuss with their patients. The next step is to finalize these lists by including the patients’ viewpoint and for surgeons to help patients weigh the benefits against possible complications of each treatment option.

Notes

Acknowledgements

R. Balm: Department of Vascular Surgery, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands.; J. P. Becquemin: Department of Vascular Surgery, Henri Mondor University, Creteil, France; J. D. Blankensteijn: Department of Surgery, Amsterdam University Medical Centers, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; G. J. de Borst: Department of Vascular Surgery, University Medical Center Utrecht, Utrecht, The Netherlands; L. Capoccia: Department of Surgery “Paride Stefanini”, Policlinico Umberto I, Rome, Italy; D. G. Clair: Department of Vascular Surgery, University of South Carolina, Columbia, USA; J. L. Cronenwett: Division of Vascular Surgery, Dartmouth-Hitchcock Medical Center, Lebanon, USA; A. H. Davies: Department of Surgery and Cancer, Charing Cross Hospital, London, United Kingdom, B. H. P. Elsman: Department of Surgery, Deventer Hospital, Deventer, The Netherlands; M. A. Farber: Department of Surgery, UNC Aortic Center, Chapel Hill, USA; T. L. Forbes: Division of Vascular Surgery, Toronto General Hospital, Toronto, Canada; P. C. J. M. Goverde: Department of Surgery, Vascular Clinic ZNA, Antwerp, Belgium; I. van Herzeele: Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium; R. J. Hinchliffe: Bristol Centre for Surgical Research, University of Bristol, Bristol, United Kingdom; D. L. Jacobs: Department of Surgery, Saint Louis University School of Medicine, St. Louis, USA; V. Jongkind: Department of Surgery, Amsterdam University Medical Centers, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; C. D. Liapis: Department of Vascular Surgery, Athens Medical Center, Athens, Greece; L. Lönn: Department of Vascular Surgery, Rigshospitalet, Copenhagen, Denmark; M. Montero-Baker: Division of Vascular Surgery, USA Tucson Medical Center, Tucson, USA; W. S. Moore: Division of Vascular Surgery, UCLA Medical Center, Los Angeles, USA; A. R. Naylor: Department of Vascular Surgery, Leicester Royal Infirmary, Leicester, United Kingdom; K. Overbeck: Department of Surgery, City Hospitals Sunderland, Sunderland, United Kingdom; T. A. Resch: Department of Thoracic and Vascular Surgery, Skane University Hospital, Malmo, Sweden; S. Ronchey: Department of Vascular Surgery, San Filippo Neri Hospital, Rome, Italy; N. Sakalihasan: Department of Cardiovascular Surgery, University Hospital of Liege, Liege, Belgium; T. P. Sarac: Department of Vascular and Endovascular Surgery, Yale New Haven Hospital, New Haven, USA; C. Setacci: Department of Vascular Surgery, University of Siena, Siena, Italy; H. Sillesen: Department of Vascular Surgery, Athens Medical Center, Athens, Greece; F. J. Veith: Department of Vascular Surgery, University of South Carolina, Columbia, USA; Division of Vascular Surgery, New York University Langone Medical Center, New York, USA; H. J. Verhagen: Department of Vascular Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands; F. Verzini: Vascular Surgery Unit, S. Maria della Misericordia Hospital, Perugia, Italy; A. M. Wiersema: Department of Surgery, Westfriesgasthuis, Hoorn, The Netherlands.

Funding

This study was funded by the AMC Foundation, which was not involved in the study design, data analysis or interpretation of results.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

Only vascular surgeons participated in this study, and thus, no informed consent was obtained.

Supplementary material

268_2019_5038_MOESM1_ESM.pdf (129 kb)
Supplementary file1 (PDF 128 kb)

References

  1. 1.
    McManus PL, Wheatley KE (2003) Consent and complications: risk disclosure varies widely between individual surgeons. Ann R Coll Surg 85:79–82CrossRefGoogle Scholar
  2. 2.
    Knops AM, Ubbink DT, Legemate DA et al (2010) Information communicated with patients in decision making about their abdominal aortic aneurysm. Eur J Vasc Endovasc Surg 39:708–713CrossRefGoogle Scholar
  3. 3.
    Legemaate J. (2001) Informed consent. https://www.knmg.nl/advies-richtlijnen/knmg-publicaties/informed-consent.htm. Accessed 10 Jan 2019.
  4. 4.
    Sokol DK (2015) Update on the UK law on consent. BMJ 350:h1481CrossRefGoogle Scholar
  5. 5.
    RAND Corporation (2018) Delphi Method. https://www.rand.org/topics/delphi-method.html. Accessed 10th Jan 2019.
  6. 6.
    de Mik SM, Stubenrouch FE, Legemate DA, et al (2018) Treatment of varicose veins, international consensus on which major complications to discuss with the patient: a Delphi study. Phlebology 268355518785482Google Scholar
  7. 7.
    Kunneman M, Pieterse AH, Stiggelbout AM et al (2015) Which benefits and harms of preoperative radiotherapy should be addressed? a Delphi consensus study among rectal cancer patients and radiation oncologists. Radiother Oncol 114:212–217CrossRefGoogle Scholar
  8. 8.
    Etzkorn JR, Gharavi NM, Carr DR et al (2018) Examining the relevance to patients of complications in the American College of Mohs Surgery registry: results of a Delphi Consensus process. Dermatol Surg 44:763–767CrossRefGoogle Scholar
  9. 9.
    Ubbink DT, Koelemay MJW (2018) Shared decision making in vascular surgery. Why would you? Eur J Vasc Endovasc Surg 56:749–750CrossRefGoogle Scholar
  10. 10.
    Keeney S, Hasson F, McKenna H (2006) Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 53:205–212CrossRefGoogle Scholar
  11. 11.
    Paravastu SC, Jayarajasingam R, Cottam R et al (2014) Endovascular repair of abdominal aortic aneurysm. Cochrane Database Syst Rev 2014:CD004178Google Scholar
  12. 12.
    Bonati LH, Lyrer P, Ederle J et al (2012) Percutaneous transluminal balloon angioplasty and stenting for carotid artery stenosis. Cochrane Database Syst Rev 9:CD000515Google Scholar
  13. 13.
    Dutch Society for Surgery (2015) Conceptrichtlijn Diagnostiek en Behandeling van Patiënten met Perifeer Arterieel Vaatlijden van de Onderste Extremiteit. https://heelkunde.nl/sites/heelkunde.nl/files/Bijlage-1-Conceptrichtlijn-Diagnostiek-en-Behandeling-van-Patienten-met-Perifeer-Arterieel-Vaatlijden.pdf. Accessed 24 Aug 2018.
  14. 14.
    Chaikof EL, Blankensteijn JD, Harris PL et al (2002) Reporting standards for endovascular aortic aneurysm repair. J Vasc Surg 35:1048–1060CrossRefGoogle Scholar
  15. 15.
    Timaran CH, McKinsey JF, Schneider PA et al (2011) Reporting standards for carotid interventions from the Society for Vascular Surgery. J Vasc Surg 53:1679–1695CrossRefGoogle Scholar
  16. 16.
    Stoner MC, Calligaro KD, Chaer RA et al (2016) Reporting standards of the Society for Vascular Surgery for endovascular treatment of chronic lower extremity peripheral artery disease: executive summary. J Vasc Surg 64:227–228CrossRefGoogle Scholar
  17. 17.
    Dindo D, Demartines N, Clavien PA (2004) Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 240:205–213CrossRefGoogle Scholar
  18. 18.
    George D, Mallery P (2003) In: SPSS for windows step by step: a simple guide and reference. 11.0 update. Allyn & Bacon, Boston, pp 1–386Google Scholar
  19. 19.
    Anderson OA, Wearne IM (2007) Informed consent for elective surgery—what is best practice? J R Soc Med 100:97–100Google Scholar
  20. 20.
    Masso Guijarro P, Aranaz Andres JM, Mira JJ et al (2010) Adverse events in hospitals: the patient's point of view. Qual Saf Health Care 19:144–147CrossRefGoogle Scholar
  21. 21.
    Mulley AG, Trimble C, Elwyn G (2012) Stop the silent misdiagnosis: patients' preferences matter. BMJ 345:e6572CrossRefGoogle Scholar
  22. 22.
    Stacey D, Legare F, Lewis K et al (2017) Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 4:CD001431Google Scholar
  23. 23.
    Stiggelbout AM, Van der Weijden T, De Wit MP et al (2012) Shared decision making: really putting patients at the centre of healthcare. BMJ 344:e256CrossRefGoogle Scholar
  24. 24.
    Clayton MJ (1997) Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 17:373–386CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • S. M. L. de Mik
    • 1
  • F. E. Stubenrouch
    • 1
  • D. A. Legemate
    • 1
  • R. Balm
    • 1
  • D. T. Ubbink
    • 1
    Email author
  • the DISCOVAR Study Group
    • 1
  1. 1.Department of Surgery, Amsterdam University Medical CentersUniversity of Amsterdam, Amsterdam Cardiovascular SciencesAmsterdamThe Netherlands

Personalised recommendations