Introduction

Deep brain stimulation (DBS) is an established therapy for patients with advanced Parkinson’s disease (PD). Several clinical trials have demonstrated significant improvement of motor symptoms, daily ‘off’ time, medication use and quality of life in well-selected patients [1–10]. To determine eligibility for DBS, patients need to undergo a comprehensive assessment in a specialised centre for movement disorders. Partly due to the lack of clear and manageable criteria for pre-selection by referring neurologists, a substantial proportion of patients are rejected following the evaluation in DBS centres [11]. For the same reason, it is also conceivable that many patients who could benefit from DBS therapy are not referred for further evaluation. In order to support appropriate pre-selection, we developed and implemented an online screening tool, called Stimulus, and monitored its use in daily clinical practice. The monitoring project aims at determining the relationship between the tool recommendations and actual referral decisions, as well as the predictive value of the tool outcomes for final selection decisions in specialised DBS centres. Here we present the three-year results of this ongoing project.

Methods

Electronic screening tool

The development of the electronic screening tool is described elsewhere [12]. In summary, a panel of 12 European experts in movement disorders used the RAND Appropriateness Method [13, 14] to assess the appropriateness of referral for DBS for 972 hypothetical patient profiles. These profiles were mutually exclusive combinations of the values of seven clinical variables considered relevant to the decision as to whether a patient should be referred to a specialised centre for the consideration of DBS. Variables included age, disease duration, severity of off-symptoms, dyskinesias and tremor, presence of axial symptoms, and mental status. During a two-round process, panelists individually rated the appropriateness of referral using a nine-point scale. For each of the profiles included, an appropriateness statement (appropriate, inappropriate, or uncertain) was calculated on the basis of the median panel score and the extent of agreement [14]. The panel also formulated five absolute criteria a patient should meet to be considered for further DBS evaluation (diagnosis of idiopathic Parkinson’s disease, presence of troublesome symptoms despite optimal pharmacological treatment or intolerable side effects of anti-parkinsonian medication, motor improvement with levodopa, absence of medical conditions preventing surgery, and absence of significant medically-resistant mental diseases). Panel results were embedded in an electronic decision tool that allows the user to select a patient profile and to see the related panel recommendation on referral (Fig. 1). The Stimulus tool is available via http://test.stimulus-dbs.org.

Fig. 1
figure 1

User interface of the screening tool

Full size image

Implementation and monitoring

The tool was disseminated via regional educational meetings for general neurologists in six European countries. Meetings were moderated by an expert in movement disorders from the same region, and included an explanation of the development and functionalities of the tool. After the meeting, participants were asked to use the tool consecutively for a period of 4 months in patients having troublesome symptoms from Parkinson’s disease. The tool recommendation was not mandatory for the referral decision. Patient profiles and referral decisions were anonymously entered by participating neurologists in an online database. Six months after initial data entry, they were asked to add follow-up data for all patients for whom a positive referral intention was documented (verification of actual referral; if not referred, reasons behind the decision). For patients referred to a specialised centre for the consideration of DBS treatment, neurologists were asked to attach a printed tool report to the referral letter (see Fig. 2).

Fig. 2
figure 2

Flow chart of the data collection by general neurologists and by DBS centres

Full size image

Evaluation

In parallel with the data collection by general neurologists, regional DBS centres were asked to provide monthly reports on final selection decisions (acceptance/refusal, reasons for refusal) for all PD patients evaluated for DBS (Fig. 2). Using the form attached to the referral letter, a distinction could be made between patients referred with and without the use of the Stimulus tool. For patients with a completed screening form, appropriateness data were added to information about the final selection decision.

Data analysis

The association between tool recommendations and referral intentions was studied by cross-tabulation (χ 2 test). The same statistical approach was used to analyse differences in acceptance rates of patients screened or not screened by the tool. The predictive value of the tool recommendations (inappropriate versus appropriate/uncertain) was assessed using the common measures for diagnostic tests (sensitivity, specificity, positive and negative predictive value). To study differences in the reasons for refusal between screened and unscreened patients, the Fisher’s exact test was applied.

Standard protocol approvals, registrations, and patients consents

All patient data were anonymously entered in the database. Participants were instructed that the tool outcome (panel recommendation) should in no way interfere with their independent judgment and referral decision. The need for ethical approval differed by national regulations. In Spain, approval was obtained from the Andalusian Health Council, while in Germany no explicit requirements were considered to apply for this type of data collection.

Results

Since February 2007, the programme has run in six European countries involving 29 DBS centres and surrounding regions. Currently, more than 4,000 patient profiles have been entered in the online database. In the remainder of this section, we will restrict our discussion to regions and centres for which data collection has been entirely completed (14 regions/centres in Germany and 1 in Spain). The educational meetings were attended by around 250 neurologists, the majority (n = 208) of whom agreed to participate in the monitoring programme. As of February 2010, 3,128 patient profiles had been completed (mean: 15 patients per participating neurologist). Five-hundred and thirty-nine patients (17%) did not meet the absolute requirements of DBS consideration (first check of the screening tool). Of the remaining 2,589 patients, referral was deemed appropriate in 22%, inappropriate in 35%, and uncertain in 43%. The intention to refer a patient for DBS consideration was strongly associated with the recommendations on appropriateness made by the Stimulus programme: a positive intention was documented in 73, 32 and 2% of patients with, respectively, an appropriate, uncertain or inappropriate referral according to the tool (Pearson χ 2 = 830,613; P < 0.0001). Six months after initial data entry, 28% of patients for whom a positive referral intention was documented had actually been referred to a DBS centre. Principal reasons for not referring a potentially appropriate candidate were the patient’s reluctance to undergo brain surgery (50%) and patient indecision (41%).

For the observational survey in the DBS centres, the mean observation period was 16 months (range 6–24 months). Final selection decisions were documented for 791 patients, of whom 224 (28%) had been screened with the Stimulus tool (Table 1).

Table 1 Acceptance for DBS (final decisions by DBS centres) in screened versus unscreened patients
Full size table

The acceptance rate of the Stimulus group was significantly higher than that of patients outside the screening programme (77 vs. 48%; Pearson χ 2 = 57,15; P < 0.0001), and was most favourable in patients for whom the outcome of the tool was appropriate (83%). Of patients for whom referral was deemed inappropriate by the tool (n = 8), two were accepted for DBS.

Considering the Stimulus tool as a diagnostic test with the outcome positive (appropriate or uncertain) or negative (inappropriate), and taking the final decision of the DBS centres as the ‘gold standard’ (accepted/refused), the tool showed a high sensitivity and a low specificity (Table 2). The positive and negative predictive values were fairly similar.

Table 2 Tool outcomes compared to final selection decision by DBS centres
Full size table

A comparison of reasons for refusal (by DBS centres) between screened and unscreened patients is shown in Table 3. In both groups, the top two categories were ‘anti-parkinsonian medication not optimised yet’ and ‘neuropsychological and/or psychiatric disorders’, albeit their frequency was higher in the Stimulus group. None of the differences reached the level of statistical significance (Fisher’s exact test).

Table 3 Reasons behind refusal for DBS in patients screened and not screened using the Stimulus tool
Full size table

Discussion

Similar to observations in the United States [11], we found that in patients referred by general neurologists without the assistance of a screening tool, a substantial proportion (52%) were refused for DBS after evaluation in a specialised DBS centre. In patients for whom the Stimulus tool was applied, the acceptance rate was significantly higher (77%), with most favourable figures in patients for whom referral was deemed appropriate (83%). Taking the final selection decision by DBS centres as the reference value, good to reasonable values were found for the tool’s sensitivity (99%) and positive (79%) and negative (75%) predictive value, while specificity was low (12%). Although generally supporting the tool’s ability to improve the quality of pre-selection for DBS, these results should be considered within the context of the design and data collection of this monitoring project.

Our focus was on the tool’s applicability in daily practice with a minimum burden for participating neurologists. For that reason, data collection was limited and did not, for example, include a reassessment of patients for whom a negative referral decision was made (potential false-negatives). In addition, no follow-up data were collected for these patients, and it is conceivable that some of them were referred in a later phase.

Referral decisions were largely in line with the tool recommendations, and resulted in a low number of inappropriate referrals (n = 8). The low specificity of the tool (2 out of these 8 patients were eventually accepted for DBS) should therefore be interpreted with caution.

Despite the strong association between panel recommendations and referral ‘intentions’, only 28% of patients for whom a positive intention was documented were actually referred during the 6 month follow-up period. In the vast majority of cases (90%) this was related to the patients’ reluctance to undergo brain surgery or their wish to postpone the decision of referral. However, it needs to be considered that patients may change their decision over time if the disease progresses and/or their confidence in DBS increases with further information. Therefore, it is likely that a higher acceptance rate was found when the follow-up period was longer.

Potential shortcomings of the tool may be reflected in the reasons for refusal. For example, the principal reasons for refusal in Stimulus patients were non-optimal anti-parkinsonian medication and the presence of neuropsychological and/or psychiatric disorders. Though not significantly different from the non-Stimulus patients (presumably due to the low number of patients and events), refinement of these criteria by the inclusion of additional checklists or screening tools may be warranted. However, the benefits of a more specific tool should be balanced against the drawback of making it more time-consuming, potentially jeopardising its use in routine neurological care.

Notwithstanding these considerations, we feel that the diagnostic properties of the tool are sufficiently convincing to advocate its use to assist general neurologists in deciding on referral for DBS in patients with PD. Periodic reassessment of the tool is needed in light of changing insights and user experiences. For example, new insights into the role of age [10, 15] and disease duration [16] for patient selection may necessitate reconsideration of the criteria included. We therefore consider the Stimulus tool as a dynamic instrument of which the application could be improved by a continuous process of learning from both science and practice.