Background

Because of several market withdrawals due to unsafe or ineffective devices, European medical device regulation is under scrutiny and will be re-regulated with stricter rules concerning requirements for clinical evidence for class III (active) and class IIb (inactive) implantable medical devices [1]. The criticism from health care providers [2, 3] as well as from pre-coverage health technology assessors (HTA) and payers [4–6], but also from patient groups [5] is nurtured by the fact that unsafe devices reach the European markets with pre-mature clinical data. Re-regulation details are still under debate: It has been suggested to reduce the number of European market authorisation agencies (“Notified Bodies/NB”) from the current 75 to a few certified ones to approve highly specialised devices or to even go for complete centralisation, as is the case with drugs. Another suggestion is to transparently document the approval process, the evidence requirements and the provided clinical data [7].

It is well known that new medical devices, including all high-risk cardio-vascular devices, receive the European market approval (CE/ Conformité Européenne mark) several years prior to USA market authorisation [2]. Due to the lack of transparency where the devices are approved and what the decentralised NBs receive as application information, the actual clinical evidence can only be speculated via clinical studies published after the CE mark was issued.

Still, because of pressure from physician groups wishing to offer patients early access to innovative medicine and from manufacturers wishing early market expansion, applications for coverage of those newly CE marked devices are submitted to reimbursement institutions only several months later [8]. In a recent analysis of seven devices from all medical disciplines, it could be shown that Austria is among the first countries where applications for uptake and inclusion into the benefit catalogue are put forward [8]. Since 2008, the Ludwig Boltzmann Institute for HTA (LBI-HTA) has been commissioned by the Austrian Ministry of Health (MoH) to assess new hospital interventions, whereby new high-risk cardio-vascular implantable devices have accounted for one-third of all primary assessments.

The aim of this study is to analyse what evidence was available for cardio-vascular devices at the time of CE marking using the evidence presented in the Austrian assessments as a proxy, and to compare this data with the data available at the time of FDA (Food and Drug Administration) approval. Thereby, we intend to challenge the argument that earlier provision of new devices is always of benefit to the patient.

Methods

Since no information is available on clinical data used by the NB for CE marking and because the Austrian pre-reimbursement assessments are rather close to CE marking, we assumed that the clinical evidence – provided by the manufacturer – used for CE mark could be considered a subset of the one used for the assessments produced for deciding on including the interventions in the hospital benefit catalogue. For this study we assumed the evidence is the same. Out of all pre-reimbursement assessments we selected those that dealt with cardio-vascular interventions. Since the intention of the analysis was to concentrate on the level of evidence of new medical devices, we excluded those assessments that dealt with a) interventions where no new (defined by us as max. 3–5 years prior to assessment) CE marked products were available at the time of assessment, b) expansion of established indications (new interventions) with already existing products, and c) cerebro-vascular interventions.

From the included assessments (published between 2006 and 2014) we extracted information on a) intervention, b) devices, c) indication and d) best available evidence in terms of number and types of studies and number of study participants. Furthermore, we searched for publicly available online information on CE marking (spring 2014) and, in cases of missing information, contacted the manufacturers directly, rather than consulting the NANDO (New Approach Notified and Designated Organisations) Database.

In the USA, new high-risk devices typically undergo a premarket approval (PMA) process based on an efficacy trial [9]. Devices can receive an IDE status (investigational device exemption) that allows the device to be used in clinical approval trials [10]. Moreover, by defining so-called HDE conditions (humanitarian device exemption), it is intended that patients with a disease manifested in fewer than 4,000 individuals benefit from a device [11]. To obtain information on the time of FDA approval and on available evidence at the time of approval, we searched for PMA, IDE, and HDE documents related to the selected interventions and devices (June to July 2014). Again, data on clinical evidence for PMA decisions and trial information were extracted.

Next, we assigned the levels of clinical evidence according to the Oxford Centre for Evidence-based Medicine hierarchy: 1: SR (systematic review) or MA (meta-analysis) based on several high quality RCTs (randomized clinical trials); 2: at least 1 RCT of high quality; 3: CTs (controlled trials) without randomisation; 4: prospective case–control and cohort studies; 5: case reports and retrospective case series [12].

Finally, we compared the market authorisation status and the level of evidence available around the time of market authorisation between Europe and the USA. Moreover, we analysed in more detail the differences between Europe and the USA in dealing with clinical evidence and with risk-benefit evaluations.

Results

Between 2008 and 2014, 15 (out of 48) pre-reimbursement assessments dealt with cardio-vascular interventions. From those, assessments on 10 cardio-vascular interventions were finally included into the analysis according to the pre-defined criteria (Table 1).

Table 1 Cardio-vascular interventions and medical devices, years of approval in Europe and in the USA, year of reimbursement-application in Austria (2008–2014)
Full size table

Market authorisation granted

For the 10 cardio-vascular interventions analysed, 27 newly approved CE marked medical devices were available (see details in Table 2). Of those, only 6 devices also hold a PMA status, 1 holds the HDE status, while 7 medical devices are under IDE. For 4 of the devices, the application in the USA was either suspended due to efficacy or safety concerns before/during/after the IDE trials (Cotovance™, Ventana™, Symplicity™), or market authorisation was denied (Watchman®) due to safety concerns. 12 CE marked cardio-vascular devices are neither PMA-approved, nor hold an IDE status (yet), meaning that they are produced solely for the European market or the IDE will be applied for at a later stage. For 1 device (CYPHER® Select), marketing was discontinued in Europe.

Table 2 Evidence available at time of pre-reimbursement assessment in Austria and at time of FDA approval, levels of evidence
Full size table

Time lag in market authorisation

For those 7 (6: PMA; 1: HDE) cardio-devices holding European and US licences, the time lag between market authorisation in Europe and in the USA amounted to 3 to 7 years.

Levels of evidence

The applications for coverage of new interventions and devices in cardio-vascular pathologies were submitted between 1 to 3 years after CE marking. In most of the 10 analysed interventions, the evidence available at the time of pre-reimbursement assessment was either level 4–5 (retrospective, sometimes prospective case series without control groups) or 2 (small RCTs/feasibility studies for individual devices/1st generation). Non-randomised controlled studies (level 3) or prospectively planned registries (level 4) were seldom available. On the contrary, IDE-accepted RCTs (level 2) or – in some rare medical conditions – prospective case series or cohort studies (level 3 and 4) were the basis for FDA approval decisions in most cases (Table 2).

A detailed analysis of the efficacy and safety assessments of individual devices seems to be more informative than the levels of evidence per se. Several differences in dealing with the evidence on benefit-risk ratios could be identified and the following patterns have appeared:

  1. 1.

    Requirement of extensive follow-up for high-risk devices: Melody™ was CE marked in 2006 and has held an HDE approval since 2010. An HDE application is not required to contain the final results of scientifically valid clinical investigations, but must contain sufficient information to determine that the device does not pose an unreasonable or significant risk and that the probable benefit to health outweighs the risks [13]. Therefore, two additional post-approval studies (150 patients, follow-up of 5 years and 100 new patients with primary analysis performed at 6 months) are required by the FDA for Melody™ [41]; these studies are ongoing. No such requirements exist in Europe, since registries are voluntary. Edwards SAPIEN XT™ and CoreValve® received a CE mark in 2010 and 2011, and a PMA in 2014. The latter is based on RCTs with 560 (SAPIEN XT™) and 656 patients (CoreValve®) respectively. This type of clinical data was not available at the time of European market authorisation. Follow-up data from post-approval studies on inoperable and extreme risk patients is required by the FDA for both devices. No such requirements were defined at the time of EU market authorisation.

  2. 2.

    Early approval in Europe and later demonstration of inefficacy in RCT: Symplicity™ received CE marking in 2008 based on a safety study (SYMPLICITY HTN-1). In 2014, the FDA approval study (SYMPLICITY HTN-3) failed to meet its primary efficacy endpoint [42]. As a consequence, the manufacturer is considering a suspension of enrolment in the already ongoing SYMPLICITY HTN-4 (IDE) trial.

  3. 3.

    Early approvals in Europe and safety concerns in the USA: Three different devices were approved for percutaneous left atrial appendage closure for the prevention of thromboembolic events in patients with atrial fibrillation in Europe between 2005 and 2013; none of them holds a PMA status. WATCHMAN® LAA Closure Technology was denied PMA in 2009 due to safety concerns. The FDA Circulatory System Devices Panel concluded (7 in favour, 5 opposed) that although short-term efficacy had been demonstrated by the data available from the PROTECT AF trial, longer term efficacy had not been adequately demonstrated due to the lack of available long-term data [39]. In late 2013, the PREVAIL data was presented; a PMA decision is pending [43].

  4. 4.

    Devices of unknown value: Optimizer™ received the CE mark in 2007 based on a feasibility trial (showing no improvement in primary endpoints [21–23] and proceeded on to the IDE-approved FIX-HF-5 trial that has been running since 2011. Results are to be expected in 2015.

    For PTA (percutaneous transluminal angioplasty) of peripheral arteries with drug- eluting balloon (DEB), 6 devices received the CE mark between 2009 and 2013. Only one of these (Lutonix DCB™) also received PMA in 2014; another (In.Pact™ Admiral IDE) is expected to be approved by the FDA in 2015 or later. One of the DEBs, Cotavance™, is CE marked for the treatment of peripheral arterial disease (PAD), as well as stenotic lesions in the iliac and infrainguinal arteries, but the IDE application was suspended in 2012 due to problems concerning drug adhesion to the balloon and the according safety concerns [44].

  5. 5.

    Devices of critical risk-benefit-ratio: MitraClip® received CE marking in 2008 on the basis of case series (EVEREST I); PMA followed in 2013. Although an RCT (EVEREST II), a prospectively planned registry (EVEREST II HRR) and a “Real World Expanded Multi-center Study” (REALISM HR) were submitted, FDA approval was cautious, with 5 votes to 3 on whether the benefits outweigh the risks and 4 votes to 5 on whether there is a reasonable assurance of efficacy.

Two devices for the endovascular repair of aortic aneurysms received CE marking in 2005 and 2013. Only one is also approved in the USA (2012); the other (Ventana™) holds an IDE, but the trial was suspended and enrolment was stopped because of a higher than expected number of re-interventions [45].

Discussion

Recent publications showed that cardiovascular devices receiving PMA (between 2000 and 2007) are often (63%) based on non-randomised studies that lack adequate strength and may be prone to bias [46], that effectiveness endpoints are more often reported than safety endpoints, and that patient comorbidities are only incompletely reported [47]. Recalls are not uncommon, especially for those devices that have been cleared via the “substantial equivalence” process [48]. The US watchdog institution Public Citizen called medical devices in the USA “substantially unsafe” [49]. In contrast, the European debate is being led – with the exemption of the detailed analysis in [2] – on a much more general level. There is only a general demand for stricter regulation, since no data are available for analysis and there is a lack of transparency concerning which NB gave market authorisation on the basis of what type of clinical evidence.

Because of the earlier market authorisation and the lack of requirements other than the performance evaluation of medical devices in Europe (the lack of a definition of “performance” is resulting in totally different interpretations), the clinical evidence available at the time of pre-reimbursement assessments is naturally lower than some years later for market authorisation in the USA [5, 50, 51]. Several unsafe and ineffective devices are approved in the EU, but not in the USA [52]. The perspective of manufacturers (and of some clinical experts) that early market access provides highly innovative medicine to suffering patients [53] is held against the perspective that little is known on the effectiveness and on the risk-benefit ratio at the time of European market authorisation. Patients are put at risk and health care systems are put under pressure to invest in interventions of unknown value [6]. Since some cardio-vascular devices are seemingly intended for the non-US market only, the uncertainty on their benefits and potential harms may never be resolved.

Based on our experiences from 7 years of pre-reimbursement assessments [54], our study contributes to the existing knowledge that not only earlier approvals are based on limited data, but that more mature data deriving from larger randomised trials and longer follow-ups might also change the perspective on the risk-benefit ratio entirely. The two most obvious examples of devices that have been considered for market approval on both continents and where different conclusions were drawn are Symplicity™ and Watchman ® . Others are less visible, but still there: For Optimizer™ (CE marked in 2007), no convincing evidence on efficacy has been demonstrated so far. In the case of Cotavance™ (CE marked in 2011), steps towards FDA approval were suspended due to problems concerning drug adhesion to the balloon. For Ventana™ (CE marked 2013), enrolment into a trial was stopped because of a higher than expected number of re-interventions. There may even be more examples we are not aware of. Further general deficiencies are well described by Fraser et al. [2].

This study has shortcomings: The biggest limitation is the fact that no data on the clinical evidence for the CE marking are available; we therefore had to rely on the information on available clinical studies derived from the Austrian pre-reimbursement assessments. Nevertheless, we think it is plausible that less rather than more clinical data were available at an earlier stage, though probably not all published. Additionally, only the devices for specific indications mentioned in the pre-reimbursement applications were assessed, meaning that other cardio-devices posing problems after CE marking such as ProRhytm® and HD Mesh Ablator® for the treatment of Atrial Fibrillation (AF) were not included in this analysis.

Conclusions

Our conclusions are that good clinical evidence at the time of market authorisation not only secures real patient benefit and safety, but also assures payers of investing only in truly innovative devices. In addition, good clinical evidence might ease market access for manufacturers and make coverage in (hospital) benefit catalogues more predictable. There is a strong need for stricter device regulation in Europe and compulsory, long-term follow-up in order not to expose European patients to (often) premature experimental devices.

Authors’ information

CW/Claudia Wild is director of the LBI-HTA and trained in communications and social medicine; IZ/Ingrid Zechmeister is deputy director of the LBI-HTA and trained in health economics; JE/Judit Erdos is research assistant at LBI-HTA and trained in International Health Care Management.