The methodologist’s point of view

Rita Banzi, Lorenzo Moja, Alessandro Liberati

In 2007, the first commentary hosted by this Cochrane’s Corner focussed on stroke units and discussed the principle that a valid combination of results from a series of unbiased primary studies can provide influential information that would not be otherwise available by individual studies [1]. We now present a case study of the role played by the stroke unit Cochrane review in the complicated process that led to changes in clinical practice and health policy. Our hypothesis is that the theoretical and pragmatic value of this systematic review goes beyond the mere increase in the number of patients considered in one study: robust results from a methodologically sound systematic review including trials performed in different setting and with different standards can increase the applicability of the conclusion.

Insight on the stroke unit Cochrane review

The Stroke Unit Trialists’ Collaboration is an international, collaborative group of trialists who (since 1984) committed themselves to systematically reviewing the field. In fact, the role of stroke unit has been controversial for over 30 years [2]. A complete systematic review was published in 1995 and 1997 in the Cochrane Library and the BMJ, respectively [3, 4]. This review can be consider as the benchmark of evidence availability on stroke unit. As any other Cochrane’s review, its regular updating guarantees a cumulative assessment of information: the last update published in 2007 includes 31 trials, involving 6,936 participants, and showed that patients who receive this care are more likely to survive their stroke, return home and become independent in looking after themselves [5]. Stroke unit care reduces the odds of death or institutionalised care (OR 0.82; 95% CI 0.73–0.92; P = 0.0006) and death or dependency (OR 0.82; 95% CI 0.73–0.92; P = 0.001). This effect remains of moderate statistical significance for case fatality (median 1 year follow-up OR 0.86; 95% CI 0.76–0.98; P = 0.02). The sensitivity analysis based on those trials which used an unequivocally blinded assessment suggested that such bias has not seriously influenced the results. Finally, the review highlighted the particular problems related to the complex and heterogeneous nature of the interventions and its potential interaction with other aspects of care: the review was unable to disentangle precisely what confers to stroke units an advantage (i.e. early mobilisation, better diagnostic procedures, prevention of complications, etc.) in improving patient outcomes.

Evaluating the impact of the stroke unit Cochrane review

Surrogate indicators of the impact of a systematic review can be obtained by analysing some quantitative impact categories. Regarding knowledge production, a rough bibliometric analysis (a branch of scientometrics and information science) performed through Google Scholar (searching by review title “Organised inpatient (stroke unit) care for stroke”) provided more than 750 citations. By searching more specifically through Web of Science, an online academic database provided by the Thomson Institute for Scientific Information [6], more than 560 publications citing one of the stroke unit systematic review versions emerged. The Cochrane review itself has been cited 149 times mainly in journal articles of clinical neurology and general and internal medicine (Table 1).

Table 1 Citation analysis of “Stroke Unit Trialists’ collaboration. Organised inpatient (stroke unit) care for stroke. Cochrane Database of Systematic Reviews” (all available versions within the Cochrane Library)
Full size table

Aside from this quantitative endeavour, a qualitative approach can lead us to in-depth describe how the systematic review findings have been implemented. Almost 50 out of the 149 bibliographic citation were reports of experimental studies. The Cochrane systematic review served as background in studies aimed to determine the effect size of stroke unit care in subgroups of patients [7], or in specific countries [8–10], or to further investigate stroke unit organisational features associated with better outcomes [11–13]. Some of these aspects, such as “…examining the potentially important components of stroke unit care and direct comparisons of different models of organised stroke unit care” were specifically mentioned in the Cochrane’s review section “Implication for research” [5]. The number of experimental studies, such as clinical trials, observational studies, surveys and other systematic reviews which have used the stroke unit systematic review as scientific background is another indicator of return in terms of helpful knowledge, research targeting and capacity building.

As a proxy of the impact on informing policy and product development the number of citation in guidelines and policy documents can be analysed. The systematic review seems to have had a considerable impact on policy at various levels. The Cochrane review and its related publications were cited in several evidence-based guidelines in many countries worldwide [14–17]. In all these documents it is strongly recommended that patients admitted to hospital because of an acute stroke are treated in an interdisciplinary stroke unit. The UK “National clinical guideline for diagnosis and initial management of acute stroke and transient ischemic attack (TIA)” published by NICE in 2008 specifically quoted “Patients with stroke admitted to organised stroke care (usually a stroke unit) are less likely to die and more likely to leave hospital independent than those who are cared for in general (usually medical and care of the elderly) wards. The evidence for this, documented in a systematic review initially in 1997, was the catalyst for a marked change in stroke service organisation across the NHS” [16]. In recent years, multiple activities supporting organised stroke care sprouted in Eastern European and developing countries, enlarging the borders of applied science outside the countries where the knowledge was produced [18, 19]. In Italy both regional and scientific society guidelines put emphasis on the implementation of stroke units [20, 21]. A large-scale controlled prospective trial aimed at assessing the effectiveness of stroke unit care in Italy (PROSIT trial) confirmed a long-term benefit for patients admitted to a stroke unit when compared with the conventional-ward care [8]. Between 2000 and 2006 the number of stroke units in seven Italian regions increased overall from 7 to 11%, reaching a 29% peak in Lombardy, demonstrating an effort to implement specialist stroke services within the National Health Service [22].

Our description of the impact of the stroke unit Cochrane review is a quick snapshot from what has been recognised as a multi-dimensional field: social, health and economic impacts of health research are complex issues at stake in evaluating research [23]. The evaluation of the impact on social health benefits and broader economic effects goes further the scope of this commentary.

A world without Cochrane reviews?

Measuring the impact and, to some extent, the benefits of health research is becoming of great interest to research funders, program managers, policymakers, researchers, policy analysts, and the public [24]. Given the considerable investment in health-related research, assessing its impact should ensure accountability, help identify ways to maximize impact and justify funding [25]. Scientific community often fails to reach a consensus around what is “valuable research”. Different target audiences could be interested in different measures of impact, able to answer to tailored needs, i.e. patient’s perspective. Irrespective of who sponsors research, any investment should be protected from the avoidable waste of inadequately producing and reporting research [26].

Not less important in any impact evaluation exercise is the difficulty in managing conflicting results: imagine that the findings of many ambiguous trials were not cumulated and that the benefit related to the intervention was still uncertain. How far would these changes have come about without this Cochrane review? It is possible that policy and practice would move in the same direction, driven for instance by the necessity to create an appropriate hospital setting to perform thrombolysis and closely monitor patients. We presume that this systematic review operated simultaneously to other maturation processes providing high-quality evidence and better knowledge on how benefit people in many parts of the world. Cochrane authors constantly support a collaborative and volunteer effort which made available an up-to-date, accurate information about the effects of healthcare readily available worldwide. Considerable impact in terms of publications, dissemination, policy and behaviour can be elicited for systematic reviews.

A clinician’s point of view

Gian Franco Gensini, Roberto Gusinu, Andrea A. Conti

Stroke units constitute a major progress in stroke patient care and international guidelines acknowledge their benefits in terms of disability reduction, home returning and survival after the acute event. The biomedical evidence underlying the recommendations included in clinical practice guidelines largely derives from systematic reviews, in particular the landmark quantitative review published in 1995 and in 1997 in the Cochrane Library and in the BMJ [3, 4], and recently updated [5], as underlined by Banzi et al. The Italian Stroke Prevention and Educational Awareness Diffusion (SPREAD) guidelines include a grade A recommendation which indicates the admission of people suffering acute stroke in stroke units [20]. These are dedicated structures in which, as reported in different syntheses of the same Italian guidelines, multi-inter-disciplinary teams of health professionals specifically committed to cerebrovascular diseases constantly follow stroke patients. An ever growing body of scientific evidence has in the last years demonstrated the efficacy and effectiveness of stroke units designed as environments including multi-competent staff, continuously and selectively educated in integrated medical, nursing and rehabilitative care. Last but not least, the specific education of patients and caregivers represents a further advancement in comprehensive and multi-dimensional acute stroke care carried out in stroke units [20]. Nursing care and leadership is a fundamental component of stroke units, and a recent interesting study clearly shows that a full comprehension of how health care operators perceive and face time, space and inter-professionalism in the setting of stroke units has the potential of further enhancing health care in stroke units [27]. With regard to this point, on methodological grounds, the role that systematic reviews have in triggering research appears relevant. In effect, besides providing the state-of-the-art scenario of the best evidence available, a key role of the Cochrane systematic reviews is, and should increasingly be in the future, to prompt targeted research in the less explored areas of even well studied clinical settings. With reference to stroke units, economic effects and social benefits may be carefully investigated to have a broader picture of the effects of these structure inside and outside hospitals. The title of this contribution indicates that measuring the impact of evidence requires time and attention. However, the model of stroke units already provides a good example of accurate and consolidated systematic reviewing and of high-quality modern and controlled health care.