Reporting of complex interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention interventions
- 7.9k Downloads
Interventions for preventing falls in older people often involve several components, multidisciplinary teams, and implementation in a variety of settings. We have developed a classification system (taxonomy) to describe interventions used to prevent falls in older people, with the aim of improving the design and reporting of clinical trials of fall-prevention interventions, and synthesis of evidence from these trials.
Thirty three international experts in falls prevention and health services research participated in a series of meetings to develop consensus. Robust techniques were used including literature reviews, expert presentations, and structured consensus workshops moderated by experienced facilitators. The taxonomy was refined using an international test panel of five health care practitioners. We assessed the chance corrected agreement of the final version by comparing taxonomy completion for 10 randomly selected published papers describing a variety of fall-prevention interventions.
The taxonomy consists of four domains, summarized as the "Approach", "Base", "Components" and "Descriptors" of an intervention. Sub-domains include; where participants are identified; the theoretical approach of the intervention; clinical targeting criteria; details on assessments; descriptions of the nature and intensity of interventions. Chance corrected agreement of the final version of the taxonomy was good to excellent for all items. Further independent evaluation of the taxonomy is required.
The taxonomy is a useful instrument for characterizing a broad range of interventions used in falls prevention. Investigators are encouraged to use the taxonomy to report their interventions.
KeywordsComplex Intervention International Expert Medical Subject Heading Service Delivery Model Medical Research Council Framework
Reduction of falls in older people is an important health care target in many countries. However, the aetiology of falls in older people is not fully understood. Although some falls result from a single factor, many are caused by an interaction between multiple factors . As a consequence, a wide variety of approaches to prevention and management have emerged, most of which are "complex interventions" . Complex interventions are usually described as interventions that contain several interacting components . Despite many clinical trials which have identified effective interventions, translation into clinical practice has been slow . There are many potential barriers to implementation of new interventions. Poor reporting of interventions in clinical trials can result in practitioners being unable to replicate the intervention. Researchers evaluating interventions in meta-analysis lack an internationally agreed framework for conceptualizing the main forms and important components of fall-prevention techniques, limiting the ability to draw meaningful comparisons and identify factors associated with success.
Several sets of international guidance have highlighted the importance of documenting and conceptualizing the influential components of an intervention, recognizing the importance of these steps in evaluation and understanding the potential for generalization of an intervention [4, 5]. However, this documentation and conceptualization is not simple, particularly when trials may be from countries with different health and social care services.
The aim was to develop a classification system to characterize the major influential components of fall-prevention interventions and promote consistency of reporting across international boundaries. We wished to encourage investigators to report all fall prevention interventions in a standardised and comprehensive manner, and to produce a taxonomy that could be used to classify existing and future interventions. We report the process of development, involving repeated testing and refinement, from the initial stages of design through to the final version of the taxonomy.
The study was developed through the Prevention of Falls Network Europe (ProFANE) project, a collaborative project to reduce the burden of fall injury in older people through excellence in research and promotion of best practice . The network activities were funded by the European Commission, but link clinicians, members of the public, and researchers worldwide.
Classification systems are of two major types-typologies and taxonomies. Typologies are theoretically driven classification schemes. A taxonomy necessitates identifying characteristics that can be objectively measured in real-world circumstances, and have sufficient variance to demonstrate meaningful differences . We aimed to develop an internationally accepted taxonomy. Sartorius  provides useful guidance on the development of taxonomies, highlighting the importance of broad agreement on the content and framework to ensure compliance with the intent and method of classification, and the need for an accompanying glossary defining operational terms. Our method was an iterative process, comprising a phase of development, followed by a period of refinement and finalisation. The process was informed and approved by international expert consensus throughout its span, the nature of the consensus building techniques varied during the project, but formal nominal group methods were implemented at key stages . The consensus panel included 33 representatives of the academic, policy, practice, and user communities drawn from a range of disciplines and countries in Australasia, Europe and North America (detailed at the end of the manuscript). Disciplines included were medicine, physiotherapy, occupational therapy, nursing, psychology, public health, exercise physiology, statistics and epidemiology.
Stage 1: Developing the taxonomy
Stage 2: Refining and agreeing the final version of the taxonomy and manual
Questions used to structure the modified nominal group.
Can current interventions be included (i.e. is the taxonomy complete)?
Is the proposed classification clear and meaningful? Does the typology include all potentially relevant and important factors, grouped in a meaningful manner?
Is the identification of factors likely to be influential in determining clinical outcome, generalisability and implementation (i.e. the model) acceptable?
Does the model adequately reflect the complexity of current interventions?
Can future interventions be included in the model?
Is the classification compatible with other classification systems?
Is it feasible to report the required information in articles/reports/
The taxonomy and manual were then tested using the remaining 27 articles retained after our search of trials published in the Cochrane review [9, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70]. Data on the intervention characteristics from these 27 papers were extracted by five academic health practitioners who had not been involved in development stages of the taxonomy (the test panel). These practitioners included disciplines of medicine, physiotherapy, occupational therapy, psychology and nursing. Each rater was provided with a copy of the taxonomy manual, and a recording form. In addition, a number of papers were replicated across the raters to enable estimation of the test re-test reliability of the taxonomy. A cut-off of poor agreement between raters (a cut of kappa <0.41 as per Landis and Koch ) was used to indicate areas of the taxonomy and manual requiring revision. We also worked with individual raters to determine areas of the manual and taxonomy that lacked clarity, were difficult to complete, or were incomplete. Results of the panel test were reported back to a group of 12 of the original members of the expert panel to agree the final format and manual for the taxonomy.
Finally, two independent reviewers completed a taxonomy for the interventions reported in ten randomised controlled trials selected at random from the systematic review of Gates et al [72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82], to determine the agreement and chance corrected agreement (kappa) for each sub-domain. Agreement was calculated over all categories included in each sub-domain, according to Landis and Koch . Chance corrected agreement (kappa) was then established using SPSS (version 17) for each sub-domain for all ten papers. For each paper we then determined the frequency of items (kappa values) contained within each kappa banding (poor, fair, moderate, good or very good) as described by Altman . The frequencies calculated were then added together across all ten papers and converted into a percentage. These results therefore show the percentage of items (kappa values) across all ten papers within each kappa banding.
Domains and sub-domains of the taxonomy of fall-prevention interventions
Approach: describes the theoretical approach in terms of the primary aims, whether a population or individual approach has been used, and whether selection or targeting criteria have been used to identify cases or populations.
Primary aims of the intervention being developed
Primary selection criteria used for case identification.
Base: describes where participants have been selected from, where the intervention is delivered and by whom
Recruitment or case identification: the site at which participants of the intervention were identified;
Primary site of delivery: the site at which the majority of the intervention is delivered or targeted.
Interventions delivered by : describes the individuals (professionals, trained professionals, etc) who deliver the majority of the intervention
Components: describes variations in assessments used for deciding treatments, the conceptual basis, and different methods of combining interventions
Assessments that are used
Combination of interventions
Descriptors: describes each of the components delivered in the control including sub-classifications that are considered potentially important
Description of control group or sham interventions
Description of the test interventions components
kappa agreement for the final extractions by two raters
Strength of agreement
0.81 - 1.00
0.61 - 0.80
0.41 - 0.60
0.21 - 0.40
The evaluation of complex interventions is an infant science. An inability to identify, define and communicate the potentially important components of falls prevention interventions has the potential to hinder development of research and uptake into clinical practice. We have identified key characteristics of interventions that we believe should be described alongside the publication of a randomised controlled trial of fall-prevention interventions. Many of the domains, sub-domains and categories are transferable to other similar interventions.
The taxonomy represents judgments in determining essential from non-essential information, and balances the need for brevity and simplicity against complexity and detail. There is no single accepted method to develop a taxonomy. Empirically based classification requires data, and in this case, data sources were publications of randomised controlled trials of fall-prevention interventions, meta-analyses and qualitative studies. This was supplemented by extensive involvement of experts, which is acknowledged as essential in developing taxonomies . Elicitation of expert knowledge from consensus methods can be problematic . We protected against predomination of individual's opinion by using a modified nominal group technique, guided by pre-specified questions and independent facilitation . The modified nominal group technique was selected in preference to the Delphi method, as it allows discussion between experts to gain consensus .
The taxonomy went through several iterations, being refined by a diverse group of international experts at different time points. There were a number of internationally agreed classification systems already in existence, and we used these wherever possible. These included the controlled vocabulary (Medical Subject Headings (MeSH)) used to index MEDLINE. The performance of the draft taxonomy was assessed using retrospective data extraction from published papers. The validation sample contained a variety of intervention types. Whilst we have allowed for the possibility of new areas emerging within the taxonomy, further refinements will be required. For example, further refinement of the selection criteria section might include data on screening tools used, for example whether the tool intends to capture single or repeat fallers, or aims to identify osteoporotic fracture risk. We also accept that the sub-domains of the taxonomy may not be entirely distinctive in all situations, but that for the majority of situations it is fit for purpose. We intend to review the taxonomy in five years time and welcome feedback on the structure and content.
The taxonomy is intended for a number of purposes, including to assist with data extraction alongside meta-analysis of research data, reporting of interventions tested in research studies, and in the process of development of interventions. The list is purposefully not exhaustive. The most recent update of the CONSORT guidelines for reporting clinical trials necessitates reporting on the main elements of complex interventions . The challenges of reporting complex interventions have been recognised for sometime. The Medical Research Council (MRC) first published a framework for reporting and evaluating complex interventions in 2000 . Although the MRC framework stresses the importance of accurate reporting, little operational guidance is provided. A recent suggestion has been to use graphical techniques to present key features of the timeline and content of interventions , a technique which can be expanded to a number of non-pharmacological interventions. In comparison, our method, which has more limited application as it focuses on fall-prevention interventions, is structured to ensure ease of complete reporting (by way of yes/no answers), uses standard internationally transferable set of definitions to describe the intervention components, settings and populations tested; and provides a more detailed description of intervention. Replication of interventions should be easier to achieve, although this assumption requires testing. We anticipate that similar methods could be developed in other fields. The taxonomy is complementary to but does not replace the Consort Guidance on the reporting of complex interventions .
Subsequently, the taxonomy has proved useful in a number of Cochrane and other high quality systematic reviews, and has enabled identification and pre-specification of important aspects of service configuration and intervention delivery [9, 72, 86, 87, 88]. We have also used the taxonomy as the framework for a UK national survey of falls services . Authors of trials are encouraged to register details of their interventions using the taxonomy in a prospective format (open registration is available at http://www.warwick.ac.uk/go/fallstaxonomy), and to utilize the taxonomy to ensure accurate, complete and useful intervention reporting.
We have developed a taxonomy to describe and classify fall-prevention interventions, with sufficient breadth to capture elements of the intervention that are thought influential in determining effectiveness. This was achieved through a mix of methods, including expert consensus, literature reviews, and validation by test panels.
Acknowledgements and funding
The authors were participants in the Prevention of Falls Network Europe (ProFaNE) Thematic Network, which was a project within Key Action 6 (The Ageing Population and Their Disabilities), part of the European Union's Quality of Life and Management of Living Resources funded by the European Commission (QLRT-2001-02705). The content of the manuscript does not represent the opinion of the European Community, and the Community is not responsible for any use that might be made of the information made in the text.
We thank all of the taxonomy investigators, all of whom participated in the expert consensus groups, panels or other aspects of the data analysis. Ballinger C, Becker C, Benevenuti F, Bramhall C, Bonnefoy M, Cameron ID, Davidson I,, Godehardt E, Gillespie LD, Grodzicki T, Hartikainen S, Harwood RH, Iwarsson S, Kivelä SL, Lamb SE, Lukasik MF, Mulder S, Ocetkiewicz T, Pfeiffer K, Potter R, Reinke T, Richardson S, Robertson MC, Rojano X, Rubenstein LZ, Salonoja M, Salva A, Sekman E, Skalska A, Skeleton D,, Stahl A,, Strach M, Todd CJ.
- 6.The Prevention of Falls Network Europe.[http://www.profane.eu.org/]
- 9.Gillespie LD, Robertson MC, Gillespie WJ, Lamb SE, Gates S, Cumming RG, Rowe BH: Interventions for preventing falls in elderly people. Cochrane Database Syst Rev. 2003, 4: CD007146-Google Scholar
- 11.Bischoff HA, Stähelin HB, Dick W, Akos R, Knecht M, Salis C, Nebiker M, Theiler R, Pfeifer M, Begerow B, Lew RA, Conzelmann M: Effects of vitamin D and calcium supplementation on falls: a randomized controlled trial. J Bone Miner Res. 2003, 18: 343-351. 10.1359/jbmr.2003.18.2.343.CrossRefPubMedGoogle Scholar
- 13.Carter ND, Khan KM, McKay HA, Petit MA, Waterman C, Heinonen A, Janssen PA, Donaldson MG, Mallinson A, Riddell L, Kruse K, Prior JC, Flicker L: Community-based exercise program reduces risk factors for falls in 65-75 year old women with osteoporosis; Randomized controlled trial. CMAJ. 2002, 167: 997-1004.PubMedPubMedCentralGoogle Scholar
- 22.Latham NK, Anderson CS, Lee A, Bennett DA, Moseley A, Cameron ID, Fitness Collaborative Group: A Randomized, Controlled Trial of Quadriceps Resistance Exercise and Vitamin D in Frail Older People: The Frailty Interventions Trial in Elderly Subjects (FITNESS). J Am Geriatr Soc. 2003, 51: 291-299. 10.1046/j.1532-5415.2003.51101.x.CrossRefPubMedGoogle Scholar
- 35.van Haastregt JC, Diedericks JP, van Rossum E, de Witte LP, Voorhoeve PM, Crebolder HF: Effects of a programme of multifactorial home visits on falls and mobility impairments in elderly people at risk: randomised controlled trial. BMJ. 2000, 321: 994-998. 10.1136/bmj.321.7267.994.CrossRefPubMedPubMedCentralGoogle Scholar
- 37.Wolf SL, Barnhart HX, Kutner NG, McNeely E, Coogler C, Xu T: Reducing frailty and falls in older persons: an investigation of Tai Chi and computerized balance training. Atlanta FICSIT Group. Frailty and Injuries: Cooperative Studies of Intervention Techniques. J Am Geriatr Soc. 1996, 44: 489-497.CrossRefPubMedGoogle Scholar
- 38.Lamb S, Gates S, Fisher J, Cooke M, Carter YH, McCabe C: Scoping Exercise on Fallers' Clinics. Report to the National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO). 2007,[http://www.sdo.nihr.ac.uk/files/project/139-final-report.pdf]Google Scholar
- 39.International Organization for Standardization: Technical Aids for Persons with Disabilities - Classification and Terminology (ISO9999). 2002, Geneva, 3Google Scholar
- 40.International Labour Office (ILO): International Standard Classification of Occupations: ISCO-88. 1988, GenevaGoogle Scholar
- 41.Organisation for Economic Co-operation and Development (OECD): Classification of Providers of Health Care (ICHA-HP). International Classification for Health Accounts 1.0. 2000, ParisGoogle Scholar
- 42.ATC/DDD Index.[http://www.whocc.no/atcddd/]
- 43.ICF - International Classification of Functioning, Disabilty and Health. [http://www.who.int/classification/icf/en]
- 66.Shaw FE, Bond J, Richardson DA, Dawson P, Steen IN, McKeith IG, Kenny RA: Multifactorial intervention after a fall in older people with cognitive impairment and dementia presenting to the accident and emergency department: randomised controlled trial. BMJ. 2003, 326: 73-75. 10.1136/bmj.326.7380.73.CrossRefPubMedPubMedCentralGoogle Scholar
- 72.Gates S, Fisher JD, Cooke MW, Carter YH, Lamb SE: Multifactorial assessment and targeted intervention for preventing falls and injuries among older people in community and emergency care settings: systematic review and meta-analysis. BMJ. 2008, 336: 130-133. 10.1136/bmj.39412.525243.BE.CrossRefPubMedGoogle Scholar
- 73.Bischoff-Ferrari HA, Orav EJ, Dawson-Hughes B: Additive benefit of higher testosterone levels and vitamin D plus calcium supplementation in regard to fall risk reduction among older men and women. Osteoporos Int. 2008, 19: 1307-1314. 10.1007/s00198-008-0573-7.CrossRefPubMedPubMedCentralGoogle Scholar
- 76.Carter ND, Khan KM, McKay HA, Petit MA, Waterman C, Heinonen A, Janssen PA, Donaldson MG, Mallinson A, Riddell L, Kruse K, Prior JC, Flicker L: Community-based exercise program reduces risk factors for falls in 65- to 75-year-old women with osteoporosis: randomized controlled trial. CMAJ. 2000, 167: 997-1004.Google Scholar
- 83.Altman DG: Practical Statistics for Medical Research. 1991, New York: Chapman and Hall, 404-1Google Scholar
- 85.A Framework for development and evaluation of RCTs for Complex Interventions to Improve Health.[http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC003372]
- 88.Gillespie LD, Robertson MC, Gillespie WJ, Lamb SE, Gates S, Cumming RG, Rowe BH: Interventions for preventing falls in elderly people. Cochrane Database Syst Rev. 2009, 15: CD007146-Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.